2024-05-08T22:55:12.934 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T22:55:12.934 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}} --verbose --owner scheduled_pdonnell@teuthology --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/orig.config.yaml 2024-05-08T22:55:12.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T22:55:13.081 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698148' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYq1kCo2lbEzSp69+JOpFnoxqac3DpM4v2MZV9JTbwoUs/Wcudfng48t8RQoNfXsgosB7/aCy0oQFxBlwJW84E= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYiog6BoFm6lZzZ1fdjbvX21f6d9TivvZuG+Qs9ENduZJPFSrhNzQG9iFGvqG7JqEC2eDbs6/pE33fi+jd6WpQ= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH6GoKraff7UaSdR9lVOfzflrRMWvf/VjNkynxjB44mDdRwIsBGr+gM7n4/N6prY+b6qG/EvpfMALy9glYF5OqI= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T22:55:13.081 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T22:55:13.082 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T22:55:13.082 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T22:55:13.084 INFO:teuthology.task.internal:Checking packages... 2024-05-08T22:55:13.112 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T22:55:13.112 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T22:55:13.113 INFO:teuthology.packaging:ref: None 2024-05-08T22:55:13.113 INFO:teuthology.packaging:tag: None 2024-05-08T22:55:13.113 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:55:13.113 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T22:55:13.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T22:55:13.339 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T22:55:13.340 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T22:55:13.349 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T22:55:13.349 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T22:55:13.357 INFO:teuthology.task.internal:Saving configuration 2024-05-08T22:55:13.380 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T22:55:13.391 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T22:55:13.409 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:49:48.650408', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYq1kCo2lbEzSp69+JOpFnoxqac3DpM4v2MZV9JTbwoUs/Wcudfng48t8RQoNfXsgosB7/aCy0oQFxBlwJW84E='} 2024-05-08T22:55:13.427 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:49:48.651649', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKYiog6BoFm6lZzZ1fdjbvX21f6d9TivvZuG+Qs9ENduZJPFSrhNzQG9iFGvqG7JqEC2eDbs6/pE33fi+jd6WpQ='} 2024-05-08T22:55:13.445 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 22:49:48.648976', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH6GoKraff7UaSdR9lVOfzflrRMWvf/VjNkynxjB44mDdRwIsBGr+gM7n4/N6prY+b6qG/EvpfMALy9glYF5OqI='} 2024-05-08T22:55:13.446 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T22:55:13.455 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T22:55:13.456 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T22:55:13.456 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T22:55:13.456 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T22:55:13.561 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6998dd1160>, signals=[15]) 2024-05-08T22:55:13.562 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T22:55:13.570 INFO:teuthology.task.internal:Opening connections... 2024-05-08T22:55:13.570 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T22:55:13.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:55:13.652 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-05-08T22:55:13.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:55:13.729 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-05-08T22:55:13.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T22:55:13.801 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T22:55:13.811 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-05-08T22:55:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-05-08T22:55:13.834 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:55:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:55:13.896 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:55:13.897 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-05-08T22:55:13.924 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-05-08T22:55:13.944 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-05-08T22:55:13.944 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-05-08T22:55:14.003 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.6" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:55:14.004 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:55:14.005 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:14.005 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:55:14.005 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:55:14.005 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:55:14.005 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:55:14.005 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-05-08T22:55:14.031 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-05-08T22:55:14.052 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-05-08T22:55:14.052 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-05-08T22:55:14.111 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T22:55:14.111 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T22:55:14.111 INFO:teuthology.orchestra.run.smithi189.stdout:ID="rhel" 2024-05-08T22:55:14.111 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="fedora" 2024-05-08T22:55:14.111 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="8.6" 2024-05-08T22:55:14.111 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el8" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T22:55:14.112 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T22:55:14.112 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-05-08T22:55:14.131 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T22:55:14.143 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T22:55:14.151 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T22:55:14.151 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:55:14.154 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:55:14.156 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-05-08T22:55:14.174 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T22:55:14.186 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T22:55:14.186 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:55:14.215 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:55:14.218 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-05-08T22:55:14.243 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T22:55:14.320 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T22:55:14.336 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T22:55:14.336 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T22:55:14.337 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:55:14.337 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:55:14.338 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T22:55:14.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:55:14.356 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-05-08T22:55:14.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:55:14.357 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-05-08T22:55:14.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:55:14.358 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-05-08T22:55:14.414 INFO:teuthology.orchestra.run.smithi189.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:14.414 INFO:teuthology.task.kernel:Running kernel on smithi189: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:14.415 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2024-05-08T22:55:14.417 INFO:teuthology.orchestra.run.smithi155.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:14.417 INFO:teuthology.task.kernel:Running kernel on smithi155: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:14.417 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum install -y kernel 2024-05-08T22:55:14.419 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:14.419 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:14.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-05-08T22:55:15.933 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:15.933 INFO:teuthology.orchestra.run.smithi155.stdout:Unable to read consumer identity 2024-05-08T22:55:15.983 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:15.984 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2024-05-08T22:55:15.993 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:15.994 INFO:teuthology.orchestra.run.smithi189.stdout:Unable to read consumer identity 2024-05-08T22:55:16.223 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:16.223 INFO:teuthology.orchestra.run.smithi155.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:16.223 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:16.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:16.236 INFO:teuthology.orchestra.run.smithi016.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:16.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:16.298 INFO:teuthology.orchestra.run.smithi189.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:17.860 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-08T22:55:17.868 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-05-08T22:55:18.065 INFO:teuthology.orchestra.run.smithi189.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-05-08T22:55:19.280 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-05-08T22:55:19.418 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-05-08T22:55:22.925 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 285 kB/s | 24 kB 00:00 2024-05-08T22:55:23.193 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 322 kB/s | 24 kB 00:00 2024-05-08T22:55:23.664 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T22:55:23.665 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T22:55:23.749 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-05-08T22:55:23.750 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-05-08T22:55:23.750 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-05-08T22:55:23.883 DEBUG:teuthology.orchestra.run.smithi155:> echo no | sudo yum reinstall kernel || true 2024-05-08T22:55:23.927 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T22:55:23.927 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T22:55:24.008 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-05-08T22:55:24.009 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-05-08T22:55:24.009 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-05-08T22:55:24.148 DEBUG:teuthology.orchestra.run.smithi189:> echo no | sudo yum reinstall kernel || true 2024-05-08T22:55:24.263 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:24.263 INFO:teuthology.orchestra.run.smithi155.stdout:Unable to read consumer identity 2024-05-08T22:55:24.469 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:24.470 INFO:teuthology.orchestra.run.smithi155.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:24.470 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:24.487 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:24.487 INFO:teuthology.orchestra.run.smithi189.stdout:Unable to read consumer identity 2024-05-08T22:55:24.659 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:24.659 INFO:teuthology.orchestra.run.smithi189.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:24.659 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:24.781 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 10:55:22 PM UTC. 2024-05-08T22:55:24.919 INFO:teuthology.orchestra.run.smithi155.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:55:24.921 INFO:teuthology.orchestra.run.smithi155.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:55:24.964 INFO:teuthology.orchestra.run.smithi155.stderr:Error: No packages marked for reinstall. 2024-05-08T22:55:24.966 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:01 ago on Wed 08 May 2024 10:55:23 PM UTC. 2024-05-08T22:55:25.014 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum reinstall -y kernel || true 2024-05-08T22:55:25.070 INFO:teuthology.orchestra.run.smithi189.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:55:25.077 INFO:teuthology.orchestra.run.smithi189.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:55:25.115 INFO:teuthology.orchestra.run.smithi189.stderr:Error: No packages marked for reinstall. 2024-05-08T22:55:25.162 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum reinstall -y kernel || true 2024-05-08T22:55:25.349 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:25.349 INFO:teuthology.orchestra.run.smithi155.stdout:Unable to read consumer identity 2024-05-08T22:55:25.499 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:25.499 INFO:teuthology.orchestra.run.smithi189.stdout:Unable to read consumer identity 2024-05-08T22:55:25.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:25.577 INFO:teuthology.orchestra.run.smithi155.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:25.577 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T22:55:25.729 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:25.730 INFO:teuthology.orchestra.run.smithi189.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:25.730 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T22:55:25.879 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 10:55:22 PM UTC. 2024-05-08T22:55:25.941 INFO:teuthology.orchestra.run.smithi155.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:55:25.941 INFO:teuthology.orchestra.run.smithi155.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:55:25.981 INFO:teuthology.orchestra.run.smithi155.stderr:Error: No packages marked for reinstall. 2024-05-08T22:55:26.029 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T22:55:26.030 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 10:55:23 PM UTC. 2024-05-08T22:55:26.096 INFO:teuthology.orchestra.run.smithi189.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:55:26.097 INFO:teuthology.orchestra.run.smithi189.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:55:26.129 INFO:teuthology.orchestra.run.smithi189.stderr:Error: No packages marked for reinstall. 2024-05-08T22:55:26.178 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T22:55:26.195 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:26.196 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:26.196 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T22:55:26.196 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T22:55:26.196 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T22:55:26.225 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-05-08T22:55:26.245 DEBUG:teuthology.parallel:result is None 2024-05-08T22:55:26.338 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:26.338 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:26.339 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T22:55:26.339 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-08T22:55:26.339 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T22:55:26.376 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-05-08T22:55:26.391 DEBUG:teuthology.parallel:result is None 2024-05-08T22:55:33.404 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 16 MB 00:15 2024-05-08T22:55:37.426 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 525 kB/s | 24 kB 00:00 2024-05-08T22:55:38.237 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T22:55:38.238 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T22:55:38.306 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-08T22:55:38.307 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-05-08T22:55:38.307 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-08T22:55:38.453 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2024-05-08T22:55:38.839 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:38.839 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2024-05-08T22:55:39.090 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:39.090 INFO:teuthology.orchestra.run.smithi016.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:39.090 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:39.452 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 10:55:37 PM UTC. 2024-05-08T22:55:39.537 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:55:39.543 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:55:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2024-05-08T22:55:39.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2024-05-08T22:55:40.020 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T22:55:40.020 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2024-05-08T22:55:40.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:40.269 INFO:teuthology.orchestra.run.smithi016.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T22:55:40.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T22:55:40.627 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 10:55:37 PM UTC. 2024-05-08T22:55:40.703 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T22:55:40.703 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T22:55:40.731 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2024-05-08T22:55:40.785 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T22:55:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:40.957 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T22:55:40.957 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T22:55:40.957 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T22:55:40.957 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T22:55:40.996 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2024-05-08T22:55:41.009 DEBUG:teuthology.parallel:result is None 2024-05-08T22:55:41.009 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T22:55:41.313 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T22:55:41.313 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:55:41.316 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:55:41.319 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T22:55:41.340 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T22:55:41.347 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T22:55:41.355 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T22:55:41.356 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:55:41.378 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:55:41.380 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T22:55:41.423 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T22:55:41.430 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T22:55:41.430 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:55:41.448 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:55:41.459 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T22:55:41.485 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:55:41.490 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:55:41.493 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:55:41.498 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:55:41.502 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:55:41.504 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T22:55:41.506 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T22:55:41.513 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T22:55:41.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:55:41.542 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:55:41.546 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T22:55:41.575 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T22:55:41.584 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T22:55:41.584 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:55:41.617 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:55:41.619 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T22:55:41.636 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:55:41.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:55:41.780 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:55:41.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:55:41.919 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T22:55:41.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:55:41.993 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:55:42.042 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:55:42.100 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:55:42.187 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:55:42.232 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T22:55:42.233 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:55:42.303 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:55:42.350 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T22:55:42.407 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:55:42.492 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T22:55:42.539 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T22:55:42.539 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T22:55:42.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-05-08T22:55:42.611 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-05-08T22:55:42.613 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-05-08T22:55:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:55:42.667 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:55:42.719 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T22:55:43.183 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T22:55:43.191 INFO:teuthology.task.internal:Starting timer... 2024-05-08T22:55:43.191 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T22:55:43.201 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T22:55:43.209 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T22:55:43.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-05-08T22:55:43.280 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-05-08T22:55:43.282 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-05-08T22:55:43.359 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-05-08T22:55:43.361 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-05-08T22:55:43.423 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-05-08T22:55:43.425 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T22:55:43.425 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-05-08T22:55:43.454 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-05-08T22:55:43.454 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-05-08T22:55:43.494 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-05-08T22:55:43.494 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-05-08T22:55:43.513 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-05-08T22:55:43.514 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-05-08T22:55:43.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:55:43.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:55:43.546 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:55:43.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:55:43.575 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T22:55:43.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T22:55:43.602 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T22:55:43.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:55:43.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:55:43.669 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-05-08T22:55:43.697 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T22:55:43.707 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T22:55:43.713 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T22:55:43.840 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T22:55:43.857 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T22:55:43.859 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi155.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T23:06:41.636 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-05-08T23:06:41.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-08T23:06:41.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:06:41.752 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-05-08T23:06:41.784 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-05-08T23:06:41.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-08T23:06:41.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:06:41.870 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-05-08T23:06:41.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-05-08T23:06:41.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-05-08T23:06:41.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T23:06:41.976 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-05-08T23:06:42.043 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-05-08T23:06:42.043 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T23:06:42.055 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T23:06:42.055 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:06:42.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:06:42.058 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:06:42.059 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:06:42.062 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T23:06:42.062 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:06:42.095 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:06:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:06:42.111 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:06:42.114 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:06:42.115 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T23:06:42.130 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T23:06:42.181 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-05-08T23:06:42.188 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-05-08T23:06:42.193 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-05-08T23:06:42.202 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-05-08T23:06:42.204 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-05-08T23:06:42.208 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:06:42.215 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-05-08T23:06:42.220 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:06:42.222 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:06:42.231 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T23:06:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:06:42.245 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T23:06:42.307 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2024-05-08T23:06:42.310 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:06:42.310 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-05-08T23:06:42.310 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.310 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.310 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.310 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.314 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2024-05-08T23:06:42.317 INFO:teuthology.orchestra.run.smithi189.stderr:bash: ntpq: command not found 2024-05-08T23:06:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:06:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-05-08T23:06:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.321 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:06:42.321 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-05-08T23:06:42.322 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.322 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.322 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.322 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:06:42.323 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T23:06:42.332 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T23:06:42.333 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:06:42.333 DEBUG:teuthology.orchestra.run.smithi016:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T23:06:42.333 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi155.front.sepia.ceph.com 2024-05-08T23:06:42.334 DEBUG:teuthology.orchestra.run.smithi155:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T23:06:42.334 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi189.front.sepia.ceph.com 2024-05-08T23:06:42.334 DEBUG:teuthology.orchestra.run.smithi189:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T23:06:42.731 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:42.742 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:42.793 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T23:06:44.565 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 kB/s | 4.1 kB 00:00 2024-05-08T23:06:44.645 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 18 kB/s | 4.1 kB 00:00 2024-05-08T23:06:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 17 kB/s | 4.5 kB 00:00 2024-05-08T23:06:44.961 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-05-08T23:06:44.961 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 kB/s | 4.5 kB 00:00 2024-05-08T23:06:45.428 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 kB/s | 4.1 kB 00:00 2024-05-08T23:06:47.906 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout:Resetting modules: 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: container-tools 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-05-08T23:06:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:06:48.000 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout:Resetting modules: 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-05-08T23:06:48.001 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:06:48.856 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-08T23:06:48.857 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-05-08T23:06:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-05-08T23:06:48.857 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-05-08T23:06:48.857 INFO:teuthology.orchestra.run.smithi016.stdout:Resetting modules: 2024-05-08T23:06:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: container-tools 2024-05-08T23:06:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:06:48.858 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-05-08T23:06:48.858 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-05-08T23:06:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:06:58.663 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-05-08T23:06:59.141 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:00.963 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-05-08T23:07:01.386 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-05-08T23:07:01.437 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-05-08T23:07:01.937 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:02.578 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-08T23:07:03.118 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:03.684 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 kB/s | 4.1 kB 00:00 2024-05-08T23:07:04.036 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-05-08T23:07:04.432 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-05-08T23:07:04.599 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================================== 2024-05-08T23:07:04.599 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-05-08T23:07:04.599 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================================== 2024-05-08T23:07:04.599 INFO:teuthology.orchestra.run.smithi155.stdout:Installing group/module packages: 2024-05-08T23:07:04.599 INFO:teuthology.orchestra.run.smithi155.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T23:07:04.600 INFO:teuthology.orchestra.run.smithi155.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout:Installing module profiles: 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools/common 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout: container-tools rhel8 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================================== 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout:Install 13 Packages 2024-05-08T23:07:04.601 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:07:04.602 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 26 M 2024-05-08T23:07:04.602 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 87 M 2024-05-08T23:07:04.602 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-05-08T23:07:04.747 INFO:teuthology.orchestra.run.smithi155.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 171 kB/s | 25 kB 00:00 2024-05-08T23:07:04.772 INFO:teuthology.orchestra.run.smithi155.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 553 kB/s | 94 kB 00:00 2024-05-08T23:07:04.831 INFO:teuthology.orchestra.run.smithi155.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 172 kB/s | 39 kB 00:00 2024-05-08T23:07:04.856 INFO:teuthology.orchestra.run.smithi155.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 377 kB/s | 41 kB 00:00 2024-05-08T23:07:04.898 INFO:teuthology.orchestra.run.smithi155.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 8.0 MB/s | 1.0 MB 00:00 2024-05-08T23:07:05.015 INFO:teuthology.orchestra.run.smithi155.stdout:(6/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.0 MB/s | 239 kB 00:00 2024-05-08T23:07:05.076 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-05-08T23:07:05.082 INFO:teuthology.orchestra.run.smithi155.stdout:(7/13): cockpit-podman-75-1.module+el8.9.0+2169 3.2 MB/s | 739 kB 00:00 2024-05-08T23:07:05.196 INFO:teuthology.orchestra.run.smithi155.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T23:07:05.291 INFO:teuthology.orchestra.run.smithi155.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 736 kB/s | 154 kB 00:00 2024-05-08T23:07:05.366 INFO:teuthology.orchestra.run.smithi155.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 11 MB/s | 3.7 MB 00:00 2024-05-08T23:07:05.441 INFO:teuthology.orchestra.run.smithi155.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 17 MB/s | 2.5 MB 00:00 2024-05-08T23:07:05.491 INFO:teuthology.orchestra.run.smithi155.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 395 kB/s | 49 kB 00:00 2024-05-08T23:07:05.600 INFO:teuthology.orchestra.run.smithi155.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T23:07:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:07:05.601 INFO:teuthology.orchestra.run.smithi155.stdout:Total 26 MB/s | 26 MB 00:00 2024-05-08T23:07:05.878 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-05-08T23:07:05.878 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 23 kB/s | 4.1 kB 00:00 2024-05-08T23:07:05.899 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-05-08T23:07:05.899 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-05-08T23:07:06.127 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-05-08T23:07:06.135 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-05-08T23:07:06.885 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-05-08T23:07:06.923 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-05-08T23:07:07.084 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================================== 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================================== 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout:Installing group/module packages: 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T23:07:07.085 INFO:teuthology.orchestra.run.smithi189.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout:Installing module profiles: 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: container-tools/common 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout:Enabling module streams: 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: container-tools rhel8 2024-05-08T23:07:07.086 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:07:07.087 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-05-08T23:07:07.087 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================================== 2024-05-08T23:07:07.087 INFO:teuthology.orchestra.run.smithi189.stdout:Install 13 Packages 2024-05-08T23:07:07.087 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:07:07.088 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 26 M 2024-05-08T23:07:07.088 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 87 M 2024-05-08T23:07:07.088 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-05-08T23:07:07.212 INFO:teuthology.orchestra.run.smithi189.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 199 kB/s | 25 kB 00:00 2024-05-08T23:07:07.272 INFO:teuthology.orchestra.run.smithi189.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 214 kB/s | 39 kB 00:00 2024-05-08T23:07:07.296 INFO:teuthology.orchestra.run.smithi189.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 450 kB/s | 94 kB 00:00 2024-05-08T23:07:07.346 INFO:teuthology.orchestra.run.smithi189.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 306 kB/s | 41 kB 00:00 2024-05-08T23:07:07.371 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T23:07:07.463 INFO:teuthology.orchestra.run.smithi189.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 5.3 MB/s | 1.0 MB 00:00 2024-05-08T23:07:07.543 INFO:teuthology.orchestra.run.smithi189.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 3.7 MB/s | 739 kB 00:00 2024-05-08T23:07:07.544 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T23:07:07.672 INFO:teuthology.orchestra.run.smithi189.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T23:07:07.694 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T23:07:07.697 INFO:teuthology.orchestra.run.smithi189.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.0 MB/s | 239 kB 00:00 2024-05-08T23:07:07.806 INFO:teuthology.orchestra.run.smithi189.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 14 MB/s | 3.7 MB 00:00 2024-05-08T23:07:07.856 INFO:teuthology.orchestra.run.smithi189.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 837 kB/s | 154 kB 00:00 2024-05-08T23:07:07.896 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T23:07:07.965 INFO:teuthology.orchestra.run.smithi189.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 16 MB/s | 2.5 MB 00:00 2024-05-08T23:07:07.990 INFO:teuthology.orchestra.run.smithi189.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 370 kB/s | 49 kB 00:00 2024-05-08T23:07:08.091 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T23:07:08.115 INFO:teuthology.orchestra.run.smithi189.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 20 MB/s | 8.1 MB 00:00 2024-05-08T23:07:08.116 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:07:08.116 INFO:teuthology.orchestra.run.smithi189.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-08T23:07:08.293 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T23:07:08.397 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-05-08T23:07:08.418 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-05-08T23:07:08.418 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-05-08T23:07:08.553 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T23:07:08.647 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-05-08T23:07:08.656 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-05-08T23:07:09.190 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================================== 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================================== 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout:Installing group/module packages: 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:07:09.376 INFO:teuthology.orchestra.run.smithi016.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout:Installing module profiles: 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: container-tools/common 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling module streams: 2024-05-08T23:07:09.377 INFO:teuthology.orchestra.run.smithi016.stdout: container-tools rhel8 2024-05-08T23:07:09.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:07:09.378 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-05-08T23:07:09.378 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================================== 2024-05-08T23:07:09.378 INFO:teuthology.orchestra.run.smithi016.stdout:Install 13 Packages 2024-05-08T23:07:09.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:07:09.378 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 26 M 2024-05-08T23:07:09.379 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 87 M 2024-05-08T23:07:09.379 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-05-08T23:07:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-05-08T23:07:09.456 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:07:09.483 INFO:teuthology.orchestra.run.smithi016.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 909 kB/s | 94 kB 00:00 2024-05-08T23:07:09.533 INFO:teuthology.orchestra.run.smithi016.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 161 kB/s | 25 kB 00:00 2024-05-08T23:07:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 220 kB/s | 39 kB 00:00 2024-05-08T23:07:09.601 INFO:teuthology.orchestra.run.smithi016.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 349 kB/s | 41 kB 00:00 2024-05-08T23:07:09.700 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.0 MB/s | 1.0 MB 00:00 2024-05-08T23:07:09.768 INFO:teuthology.orchestra.run.smithi016.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.3 MB/s | 739 kB 00:00 2024-05-08T23:07:09.876 INFO:teuthology.orchestra.run.smithi016.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 28 MB/s | 8.8 MB 00:00 2024-05-08T23:07:09.901 INFO:teuthology.orchestra.run.smithi016.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.2 MB/s | 239 kB 00:00 2024-05-08T23:07:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T23:07:10.010 INFO:teuthology.orchestra.run.smithi016.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.1 MB/s | 154 kB 00:00 2024-05-08T23:07:10.070 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T23:07:10.085 INFO:teuthology.orchestra.run.smithi016.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 12 MB/s | 3.7 MB 00:00 2024-05-08T23:07:10.097 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T23:07:10.252 INFO:teuthology.orchestra.run.smithi016.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 23 MB/s | 8.1 MB 00:00 2024-05-08T23:07:10.278 INFO:teuthology.orchestra.run.smithi016.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 258 kB/s | 49 kB 00:00 2024-05-08T23:07:10.291 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T23:07:10.378 INFO:teuthology.orchestra.run.smithi016.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 6.9 MB/s | 2.5 MB 00:00 2024-05-08T23:07:10.378 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:07:10.378 INFO:teuthology.orchestra.run.smithi016.stdout:Total 26 MB/s | 26 MB 00:01 2024-05-08T23:07:10.509 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T23:07:10.679 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T23:07:10.686 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-05-08T23:07:10.710 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-05-08T23:07:10.710 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-05-08T23:07:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T23:07:10.953 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T23:07:10.967 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-05-08T23:07:10.977 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-05-08T23:07:11.149 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T23:07:11.157 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T23:07:11.340 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:07:11.738 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:07:11.742 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:07:11.743 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T23:07:11.743 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T23:07:11.743 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T23:07:12.063 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:07:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T23:07:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:07:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T23:07:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T23:07:12.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T23:07:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T23:07:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T23:07:13.179 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T23:07:13.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T23:07:13.600 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T23:07:13.828 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T23:07:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:07:14.332 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:07:14.436 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T23:07:14.437 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T23:07:14.536 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:07:14.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T23:07:15.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T23:07:16.164 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T23:07:16.326 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T23:07:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T23:07:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T23:07:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T23:07:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T23:07:25.082 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T23:07:25.082 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:25.642 INFO:teuthology.orchestra.run.smithi155.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:07:25.643 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-05-08T23:07:25.843 DEBUG:teuthology.parallel:result is None 2024-05-08T23:07:25.882 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T23:07:25.882 INFO:teuthology.orchestra.run.smithi189.stdout:Installed products updated. 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:26.476 INFO:teuthology.orchestra.run.smithi189.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:07:26.477 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-05-08T23:07:26.660 DEBUG:teuthology.parallel:result is None 2024-05-08T23:07:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T23:07:29.535 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2024-05-08T23:07:30.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:07:30.177 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-05-08T23:07:30.177 INFO:teuthology.orchestra.run.smithi016.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:30.177 INFO:teuthology.orchestra.run.smithi016.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T23:07:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T23:07:30.179 INFO:teuthology.orchestra.run.smithi016.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T23:07:30.179 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:07:30.179 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-08T23:07:30.358 DEBUG:teuthology.parallel:result is None 2024-05-08T23:07:30.358 INFO:teuthology.run_tasks:Running task install... 2024-05-08T23:07:30.368 DEBUG:teuthology.task.install:project ceph 2024-05-08T23:07:30.368 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T23:07:30.368 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T23:07:30.368 INFO:teuthology.task.install:Using flavor: default 2024-05-08T23:07:30.377 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T23:07:30.377 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T23:07:30.377 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:07:30.378 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:07:30.380 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:07:30.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:07:30.381 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T23:07:30.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:07:30.534 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:07:30.534 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:07:30.539 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:07:30.539 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:07:30.606 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T23:07:30.606 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T23:07:30.711 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:07:30.711 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:07:30.711 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:07:30.714 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:07:30.714 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:07:30.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:07:30.719 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T23:07:30.719 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:07:30.719 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T23:07:30.747 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:07:30.747 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:07:30.749 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:07:30.750 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:07:30.753 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T23:07:30.754 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T23:07:30.818 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:07:30.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:07:30.826 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T23:07:30.900 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:07:30.909 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:07:30.912 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T23:07:30.944 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-05-08T23:07:30.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-05-08T23:07:30.967 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-05-08T23:07:30.970 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-05-08T23:07:30.978 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-05-08T23:07:30.980 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-05-08T23:07:31.348 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:31.365 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:31.381 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:32.537 INFO:teuthology.orchestra.run.smithi189.stdout:63 files removed 2024-05-08T23:07:32.589 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:07:32.728 INFO:teuthology.orchestra.run.smithi155.stdout:63 files removed 2024-05-08T23:07:32.736 INFO:teuthology.orchestra.run.smithi016.stdout:63 files removed 2024-05-08T23:07:32.779 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:07:32.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T23:07:32.928 INFO:teuthology.orchestra.run.smithi189.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:33.118 INFO:teuthology.orchestra.run.smithi155.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:33.176 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2024-05-08T23:07:35.136 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 106 kB/s | 80 kB 00:00 2024-05-08T23:07:35.280 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 141 kB/s | 80 kB 00:00 2024-05-08T23:07:35.312 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 153 kB/s | 80 kB 00:00 2024-05-08T23:07:35.670 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-05-08T23:07:35.792 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-05-08T23:07:35.813 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-05-08T23:07:36.179 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2024-05-08T23:07:36.330 INFO:teuthology.orchestra.run.smithi189.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-05-08T23:07:36.372 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 2.8 kB/s | 1.5 kB 00:00 2024-05-08T23:07:36.401 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2024-05-08T23:07:36.524 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-05-08T23:07:36.585 INFO:teuthology.orchestra.run.smithi155.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T23:07:38.238 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-05-08T23:07:38.597 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2024-05-08T23:07:39.215 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 16 MB 00:02 2024-05-08T23:07:41.894 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 357 kB/s | 24 kB 00:00 2024-05-08T23:07:42.239 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 286 kB/s | 24 kB 00:00 2024-05-08T23:07:42.940 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-08T23:07:43.230 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 67 MB 00:01 2024-05-08T23:07:43.291 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 433 kB/s | 24 kB 00:00 2024-05-08T23:07:44.801 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 61 MB 00:01 2024-05-08T23:07:46.342 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 MB/s | 67 MB 00:01 2024-05-08T23:07:53.563 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-05-08T23:07:56.582 INFO:teuthology.orchestra.run.smithi155.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 61 MB 00:01 2024-05-08T23:07:57.102 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-05-08T23:07:57.518 INFO:teuthology.orchestra.run.smithi189.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 61 MB 00:01 2024-05-08T23:08:00.287 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-05-08T23:08:12.346 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:08:12.347 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:08:12.365 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:08:12.366 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:08:12.368 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:08:12.369 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:08:12.372 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:08:12.375 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:08:12.377 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:08:12.595 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-05-08T23:08:12.609 INFO:teuthology.orchestra.run.smithi155.stdout:=================================================================================================================================== 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout:=================================================================================================================================== 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:08:12.610 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:08:12.611 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:08:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:08:12.614 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:08:12.615 INFO:teuthology.orchestra.run.smithi155.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:08:12.616 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:08:12.617 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:08:12.618 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:08:12.619 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:08:12.620 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout:Enabling module streams: 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql 10 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout:=================================================================================================================================== 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout:Install 130 Packages 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 16 Packages 2024-05-08T23:08:12.621 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:08:12.622 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 215 M 2024-05-08T23:08:12.622 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-05-08T23:08:13.107 INFO:teuthology.orchestra.run.smithi155.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2024-05-08T23:08:13.384 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:08:13.385 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:08:13.404 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:08:13.406 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:08:13.407 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:08:13.409 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:08:13.412 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:08:13.416 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:08:13.417 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:08:13.623 INFO:teuthology.orchestra.run.smithi155.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.6 MB/s | 860 kB 00:00 2024-05-08T23:08:13.638 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout:=================================================================================================================================== 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout:=================================================================================================================================== 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:08:13.652 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:08:13.653 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:08:13.654 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:08:13.655 INFO:teuthology.orchestra.run.smithi189.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:08:13.656 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:08:13.657 INFO:teuthology.orchestra.run.smithi189.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:08:13.658 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:08:13.659 INFO:teuthology.orchestra.run.smithi189.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:08:13.660 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:08:13.661 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:08:13.662 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:08:13.663 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:08:13.664 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout:Enabling module streams: 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql 10 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout:=================================================================================================================================== 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout:Install 130 Packages 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 16 Packages 2024-05-08T23:08:13.665 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:08:13.666 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 215 M 2024-05-08T23:08:13.666 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-05-08T23:08:14.201 INFO:teuthology.orchestra.run.smithi189.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2024-05-08T23:08:14.590 INFO:teuthology.orchestra.run.smithi155.stdout:(3/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 3.1 MB/s | 5.5 MB 00:01 2024-05-08T23:08:14.616 INFO:teuthology.orchestra.run.smithi155.stdout:(4/146): ceph-immutable-object-cache-18.2.2-132 167 kB/s | 166 kB 00:00 2024-05-08T23:08:15.751 INFO:teuthology.orchestra.run.smithi189.stdout:(2/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 2.9 MB/s | 5.5 MB 00:01 2024-05-08T23:08:15.857 INFO:teuthology.orchestra.run.smithi155.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 1.3 MB/s | 1.6 MB 00:01 2024-05-08T23:08:15.876 INFO:teuthology.orchestra.run.smithi189.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.3 MB/s | 166 kB 00:00 2024-05-08T23:08:16.401 INFO:teuthology.orchestra.run.smithi189.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 4.3 MB/s | 2.3 MB 00:00 2024-05-08T23:08:16.426 INFO:teuthology.orchestra.run.smithi155.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.2 MB/s | 2.3 MB 00:01 2024-05-08T23:08:16.793 INFO:teuthology.orchestra.run.smithi189.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 4.0 MB/s | 1.6 MB 00:00 2024-05-08T23:08:16.951 INFO:teuthology.orchestra.run.smithi189.stdout:(6/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 313 kB/s | 860 kB 00:02 2024-05-08T23:08:17.313 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:08:17.314 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T23:08:17.335 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T23:08:17.337 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T23:08:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T23:08:17.340 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T23:08:17.343 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T23:08:17.348 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T23:08:17.349 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T23:08:17.605 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-05-08T23:08:17.621 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================================================== 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================================================== 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T23:08:17.622 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T23:08:17.623 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T23:08:17.624 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T23:08:17.625 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T23:08:17.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T23:08:17.627 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T23:08:17.628 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T23:08:17.629 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T23:08:17.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T23:08:17.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T23:08:17.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling module streams: 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql 10 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================================================== 2024-05-08T23:08:17.633 INFO:teuthology.orchestra.run.smithi016.stdout:Install 130 Packages 2024-05-08T23:08:17.634 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 16 Packages 2024-05-08T23:08:17.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:08:17.634 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 215 M 2024-05-08T23:08:17.634 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-05-08T23:08:17.727 INFO:teuthology.orchestra.run.smithi189.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 5.2 MB/s | 4.8 MB 00:00 2024-05-08T23:08:18.179 INFO:teuthology.orchestra.run.smithi016.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 12 kB/s | 6.4 kB 00:00 2024-05-08T23:08:19.204 INFO:teuthology.orchestra.run.smithi016.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 839 kB/s | 860 kB 00:01 2024-05-08T23:08:19.366 INFO:teuthology.orchestra.run.smithi155.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 1.4 MB/s | 4.8 MB 00:03 2024-05-08T23:08:19.396 INFO:teuthology.orchestra.run.smithi016.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 867 kB/s | 166 kB 00:00 2024-05-08T23:08:20.302 INFO:teuthology.orchestra.run.smithi189.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 5.1 MB/s | 13 MB 00:02 2024-05-08T23:08:20.410 INFO:teuthology.orchestra.run.smithi189.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 225 kB/s | 24 kB 00:00 2024-05-08T23:08:20.621 INFO:teuthology.orchestra.run.smithi016.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 1.9 MB/s | 5.5 MB 00:02 2024-05-08T23:08:21.288 INFO:teuthology.orchestra.run.smithi016.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 2.3 MB/s | 1.6 MB 00:00 2024-05-08T23:08:21.355 INFO:teuthology.orchestra.run.smithi016.stdout:(6/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.2 MB/s | 2.3 MB 00:01 2024-05-08T23:08:21.641 INFO:teuthology.orchestra.run.smithi155.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 2.5 MB/s | 22 MB 00:08 2024-05-08T23:08:21.758 INFO:teuthology.orchestra.run.smithi155.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 209 kB/s | 24 kB 00:00 2024-05-08T23:08:23.605 INFO:teuthology.orchestra.run.smithi016.stdout:(7/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 4.8 MB 00:02 2024-05-08T23:08:24.113 INFO:teuthology.orchestra.run.smithi016.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 3.4 MB/s | 22 MB 00:06 2024-05-08T23:08:24.239 INFO:teuthology.orchestra.run.smithi016.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 195 kB/s | 24 kB 00:00 2024-05-08T23:08:33.549 INFO:teuthology.orchestra.run.smithi155.stdout:(10/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 945 kB/s | 13 MB 00:14 2024-05-08T23:08:33.782 INFO:teuthology.orchestra.run.smithi155.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.1 MB/s | 261 kB 00:00 2024-05-08T23:08:33.882 INFO:teuthology.orchestra.run.smithi155.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 310 kB/s | 31 kB 00:00 2024-05-08T23:08:34.334 INFO:teuthology.orchestra.run.smithi189.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 1.1 MB/s | 18 MB 00:17 2024-05-08T23:08:34.466 INFO:teuthology.orchestra.run.smithi155.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 1.3 MB/s | 782 kB 00:00 2024-05-08T23:08:34.584 INFO:teuthology.orchestra.run.smithi189.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 1.0 MB/s | 261 kB 00:00 2024-05-08T23:08:34.633 INFO:teuthology.orchestra.run.smithi155.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 851 kB/s | 142 kB 00:00 2024-05-08T23:08:34.676 INFO:teuthology.orchestra.run.smithi189.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 337 kB/s | 31 kB 00:00 2024-05-08T23:08:34.817 INFO:teuthology.orchestra.run.smithi155.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 758 kB/s | 139 kB 00:00 2024-05-08T23:08:35.242 INFO:teuthology.orchestra.run.smithi155.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 1.1 MB/s | 474 kB 00:00 2024-05-08T23:08:35.276 INFO:teuthology.orchestra.run.smithi189.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 1.3 MB/s | 782 kB 00:00 2024-05-08T23:08:35.460 INFO:teuthology.orchestra.run.smithi189.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 774 kB/s | 142 kB 00:00 2024-05-08T23:08:35.635 INFO:teuthology.orchestra.run.smithi189.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 795 kB/s | 139 kB 00:00 2024-05-08T23:08:36.152 INFO:teuthology.orchestra.run.smithi189.stdout:(16/146): ceph-common-18.2.2-1322.gd21e4f96.el8 996 kB/s | 22 MB 00:22 2024-05-08T23:08:36.178 INFO:teuthology.orchestra.run.smithi189.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 875 kB/s | 474 kB 00:00 2024-05-08T23:08:36.269 INFO:teuthology.orchestra.run.smithi189.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 488 kB/s | 45 kB 00:00 2024-05-08T23:08:36.436 INFO:teuthology.orchestra.run.smithi189.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 774 kB/s | 129 kB 00:00 2024-05-08T23:08:36.653 INFO:teuthology.orchestra.run.smithi189.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 908 kB/s | 197 kB 00:00 2024-05-08T23:08:36.989 INFO:teuthology.orchestra.run.smithi189.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.1 MB/s | 370 kB 00:00 2024-05-08T23:08:37.295 INFO:teuthology.orchestra.run.smithi189.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 1.2 MB/s | 365 kB 00:00 2024-05-08T23:08:37.462 INFO:teuthology.orchestra.run.smithi189.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 641 kB/s | 107 kB 00:00 2024-05-08T23:08:37.554 INFO:teuthology.orchestra.run.smithi189.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.0 MB/s | 95 kB 00:00 2024-05-08T23:08:38.638 INFO:teuthology.orchestra.run.smithi189.stdout:(25/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 2.6 MB/s | 6.5 MB 00:02 2024-05-08T23:08:39.008 INFO:teuthology.orchestra.run.smithi155.stdout:(17/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 835 kB/s | 18 MB 00:22 2024-05-08T23:08:39.117 INFO:teuthology.orchestra.run.smithi155.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 414 kB/s | 45 kB 00:00 2024-05-08T23:08:39.154 INFO:teuthology.orchestra.run.smithi016.stdout:(10/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 862 kB/s | 13 MB 00:15 2024-05-08T23:08:39.409 INFO:teuthology.orchestra.run.smithi189.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 235 kB/s | 181 kB 00:00 2024-05-08T23:08:39.471 INFO:teuthology.orchestra.run.smithi016.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 826 kB/s | 261 kB 00:00 2024-05-08T23:08:39.580 INFO:teuthology.orchestra.run.smithi016.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 285 kB/s | 31 kB 00:00 2024-05-08T23:08:39.599 INFO:teuthology.orchestra.run.smithi155.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 267 kB/s | 129 kB 00:00 2024-05-08T23:08:39.630 INFO:teuthology.orchestra.run.smithi189.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 110 kB/s | 24 kB 00:00 2024-05-08T23:08:39.821 INFO:teuthology.orchestra.run.smithi189.stdout:(28/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 1.5 MB/s | 3.5 MB 00:02 2024-05-08T23:08:39.856 INFO:teuthology.orchestra.run.smithi189.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 615 kB/s | 138 kB 00:00 2024-05-08T23:08:39.934 INFO:teuthology.orchestra.run.smithi155.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 588 kB/s | 197 kB 00:00 2024-05-08T23:08:40.393 INFO:teuthology.orchestra.run.smithi155.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 808 kB/s | 370 kB 00:00 2024-05-08T23:08:40.455 INFO:teuthology.orchestra.run.smithi016.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 894 kB/s | 782 kB 00:00 2024-05-08T23:08:40.614 INFO:teuthology.orchestra.run.smithi016.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 896 kB/s | 142 kB 00:00 2024-05-08T23:08:40.676 INFO:teuthology.orchestra.run.smithi155.stdout:(22/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 1.2 MB/s | 6.5 MB 00:05 2024-05-08T23:08:40.764 INFO:teuthology.orchestra.run.smithi016.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 928 kB/s | 139 kB 00:00 2024-05-08T23:08:40.793 INFO:teuthology.orchestra.run.smithi155.stdout:(23/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 911 kB/s | 365 kB 00:00 2024-05-08T23:08:40.819 INFO:teuthology.orchestra.run.smithi155.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 752 kB/s | 107 kB 00:00 2024-05-08T23:08:40.886 INFO:teuthology.orchestra.run.smithi155.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.0 MB/s | 95 kB 00:00 2024-05-08T23:08:40.964 INFO:teuthology.orchestra.run.smithi189.stdout:(30/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 2.3 MB/s | 47 MB 00:20 2024-05-08T23:08:41.019 INFO:teuthology.orchestra.run.smithi155.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.3 MB/s | 181 kB 00:00 2024-05-08T23:08:41.081 INFO:teuthology.orchestra.run.smithi189.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.1 MB/s | 245 kB 00:00 2024-05-08T23:08:41.103 INFO:teuthology.orchestra.run.smithi155.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 291 kB/s | 24 kB 00:00 2024-05-08T23:08:41.173 INFO:teuthology.orchestra.run.smithi189.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 537 kB/s | 49 kB 00:00 2024-05-08T23:08:41.189 INFO:teuthology.orchestra.run.smithi016.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 1.1 MB/s | 474 kB 00:00 2024-05-08T23:08:41.253 INFO:teuthology.orchestra.run.smithi155.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 925 kB/s | 138 kB 00:00 2024-05-08T23:08:41.265 INFO:teuthology.orchestra.run.smithi189.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 178 kB/s | 16 kB 00:00 2024-05-08T23:08:41.365 INFO:teuthology.orchestra.run.smithi189.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.6 MB/s | 263 kB 00:00 2024-05-08T23:08:41.490 INFO:teuthology.orchestra.run.smithi189.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.3 MB/s | 170 kB 00:00 2024-05-08T23:08:41.582 INFO:teuthology.orchestra.run.smithi189.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 197 kB/s | 18 kB 00:00 2024-05-08T23:08:41.749 INFO:teuthology.orchestra.run.smithi189.stdout:(37/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 1.8 MB/s | 3.5 MB 00:01 2024-05-08T23:08:42.024 INFO:teuthology.orchestra.run.smithi189.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 693 kB/s | 306 kB 00:00 2024-05-08T23:08:42.041 INFO:teuthology.orchestra.run.smithi189.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 202 kB/s | 59 kB 00:00 2024-05-08T23:08:42.100 INFO:teuthology.orchestra.run.smithi189.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 998 kB/s | 75 kB 00:00 2024-05-08T23:08:42.167 INFO:teuthology.orchestra.run.smithi189.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 653 kB/s | 44 kB 00:00 2024-05-08T23:08:42.267 INFO:teuthology.orchestra.run.smithi189.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2024-05-08T23:08:42.293 INFO:teuthology.orchestra.run.smithi189.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2024-05-08T23:08:42.335 INFO:teuthology.orchestra.run.smithi189.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 493 kB/s | 33 kB 00:00 2024-05-08T23:08:42.360 INFO:teuthology.orchestra.run.smithi189.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2024-05-08T23:08:42.418 INFO:teuthology.orchestra.run.smithi189.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-05-08T23:08:42.452 INFO:teuthology.orchestra.run.smithi189.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2024-05-08T23:08:42.519 INFO:teuthology.orchestra.run.smithi189.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 811 kB/s | 81 kB 00:00 2024-05-08T23:08:42.531 INFO:teuthology.orchestra.run.smithi016.stdout:(17/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 890 kB/s | 18 MB 00:21 2024-05-08T23:08:42.544 INFO:teuthology.orchestra.run.smithi189.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2024-05-08T23:08:42.586 INFO:teuthology.orchestra.run.smithi189.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2024-05-08T23:08:42.631 INFO:teuthology.orchestra.run.smithi016.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 448 kB/s | 45 kB 00:00 2024-05-08T23:08:42.661 INFO:teuthology.orchestra.run.smithi189.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 650 kB/s | 49 kB 00:00 2024-05-08T23:08:42.703 INFO:teuthology.orchestra.run.smithi189.stdout:(52/146): python3-kubernetes-11.0.0-6.el8.noarc 7.3 MB/s | 1.2 MB 00:00 2024-05-08T23:08:42.768 INFO:teuthology.orchestra.run.smithi189.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 551 kB/s | 59 kB 00:00 2024-05-08T23:08:42.773 INFO:teuthology.orchestra.run.smithi016.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 912 kB/s | 129 kB 00:00 2024-05-08T23:08:42.804 INFO:teuthology.orchestra.run.smithi189.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2024-05-08T23:08:42.896 INFO:teuthology.orchestra.run.smithi189.stdout:(55/146): ceph-mgr-diskprediction-local-18.2.2- 2.4 MB/s | 7.4 MB 00:03 2024-05-08T23:08:42.920 INFO:teuthology.orchestra.run.smithi155.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 2.1 MB/s | 3.5 MB 00:01 2024-05-08T23:08:42.921 INFO:teuthology.orchestra.run.smithi189.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2024-05-08T23:08:42.938 INFO:teuthology.orchestra.run.smithi189.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2024-05-08T23:08:42.963 INFO:teuthology.orchestra.run.smithi189.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2024-05-08T23:08:42.973 INFO:teuthology.orchestra.run.smithi016.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 983 kB/s | 197 kB 00:00 2024-05-08T23:08:43.031 INFO:teuthology.orchestra.run.smithi189.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 693 kB/s | 64 kB 00:00 2024-05-08T23:08:43.064 INFO:teuthology.orchestra.run.smithi189.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2024-05-08T23:08:43.098 INFO:teuthology.orchestra.run.smithi189.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 367 kB/s | 24 kB 00:00 2024-05-08T23:08:43.131 INFO:teuthology.orchestra.run.smithi189.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 436 kB/s | 29 kB 00:00 2024-05-08T23:08:43.173 INFO:teuthology.orchestra.run.smithi189.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 367 kB/s | 27 kB 00:00 2024-05-08T23:08:43.223 INFO:teuthology.orchestra.run.smithi189.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2024-05-08T23:08:43.240 INFO:teuthology.orchestra.run.smithi016.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.4 MB/s | 370 kB 00:00 2024-05-08T23:08:43.310 INFO:teuthology.orchestra.run.smithi189.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2024-05-08T23:08:43.340 INFO:teuthology.orchestra.run.smithi189.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 523 kB/s | 61 kB 00:00 2024-05-08T23:08:43.357 INFO:teuthology.orchestra.run.smithi189.stdout:(67/146): python3-routes-2.4.1-12.el8.noarch.rp 451 kB/s | 196 kB 00:00 2024-05-08T23:08:43.383 INFO:teuthology.orchestra.run.smithi189.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2024-05-08T23:08:43.387 INFO:teuthology.orchestra.run.smithi155.stdout:(30/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 1.3 MB/s | 3.5 MB 00:02 2024-05-08T23:08:43.408 INFO:teuthology.orchestra.run.smithi189.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 342 kB/s | 23 kB 00:00 2024-05-08T23:08:43.441 INFO:teuthology.orchestra.run.smithi189.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2024-05-08T23:08:43.491 INFO:teuthology.orchestra.run.smithi016.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 1.4 MB/s | 365 kB 00:00 2024-05-08T23:08:43.533 INFO:teuthology.orchestra.run.smithi189.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 386 kB/s | 48 kB 00:00 2024-05-08T23:08:43.579 INFO:teuthology.orchestra.run.smithi155.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 1.3 MB/s | 245 kB 00:00 2024-05-08T23:08:43.584 INFO:teuthology.orchestra.run.smithi189.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 126 kB/s | 18 kB 00:00 2024-05-08T23:08:43.616 INFO:teuthology.orchestra.run.smithi016.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 853 kB/s | 107 kB 00:00 2024-05-08T23:08:43.626 INFO:teuthology.orchestra.run.smithi189.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2024-05-08T23:08:43.651 INFO:teuthology.orchestra.run.smithi189.stdout:(74/146): python3-jwt-1.6.1-2.el8.noarch.rpm 362 kB/s | 43 kB 00:00 2024-05-08T23:08:43.671 INFO:teuthology.orchestra.run.smithi155.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 536 kB/s | 49 kB 00:00 2024-05-08T23:08:43.691 INFO:teuthology.orchestra.run.smithi016.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.2 MB/s | 95 kB 00:00 2024-05-08T23:08:43.751 INFO:teuthology.orchestra.run.smithi189.stdout:(75/146): python3-oauthlib-2.1.0-1.el8.noarch.r 927 kB/s | 155 kB 00:00 2024-05-08T23:08:43.771 INFO:teuthology.orchestra.run.smithi155.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 163 kB/s | 16 kB 00:00 2024-05-08T23:08:43.793 INFO:teuthology.orchestra.run.smithi189.stdout:(76/146): python3-pycparser-2.14-14.el8.noarch. 653 kB/s | 109 kB 00:00 2024-05-08T23:08:43.852 INFO:teuthology.orchestra.run.smithi189.stdout:(77/146): python3-requests-oauthlib-1.0.0-1.el8 219 kB/s | 43 kB 00:00 2024-05-08T23:08:43.869 INFO:teuthology.orchestra.run.smithi189.stdout:(78/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 160 kB/s | 19 kB 00:00 2024-05-08T23:08:43.894 INFO:teuthology.orchestra.run.smithi189.stdout:(79/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 436 kB/s | 44 kB 00:00 2024-05-08T23:08:44.013 INFO:teuthology.orchestra.run.smithi155.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.1 MB/s | 263 kB 00:00 2024-05-08T23:08:44.019 INFO:teuthology.orchestra.run.smithi189.stdout:(80/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2024-05-08T23:08:44.036 INFO:teuthology.orchestra.run.smithi189.stdout:(81/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-05-08T23:08:44.112 INFO:teuthology.orchestra.run.smithi189.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2024-05-08T23:08:44.128 INFO:teuthology.orchestra.run.smithi189.stdout:(83/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 524 kB/s | 48 kB 00:00 2024-05-08T23:08:44.154 INFO:teuthology.orchestra.run.smithi189.stdout:(84/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.5 MB/s | 200 kB 00:00 2024-05-08T23:08:44.188 INFO:teuthology.orchestra.run.smithi155.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 974 kB/s | 170 kB 00:00 2024-05-08T23:08:44.271 INFO:teuthology.orchestra.run.smithi189.stdout:(85/146): libsepol-devel-2.9-3.el8.x86_64.rpm 548 kB/s | 87 kB 00:00 2024-05-08T23:08:44.289 INFO:teuthology.orchestra.run.smithi155.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 181 kB/s | 18 kB 00:00 2024-05-08T23:08:44.296 INFO:teuthology.orchestra.run.smithi189.stdout:(86/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 345 kB/s | 58 kB 00:00 2024-05-08T23:08:44.352 INFO:teuthology.orchestra.run.smithi189.stdout:(87/146): libcom_err-devel-1.45.6-4.el8.x86_64. 194 kB/s | 39 kB 00:00 2024-05-08T23:08:44.438 INFO:teuthology.orchestra.run.smithi189.stdout:(88/146): fuse-2.9.7-17.el8.x86_64.rpm 497 kB/s | 83 kB 00:00 2024-05-08T23:08:44.472 INFO:teuthology.orchestra.run.smithi189.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.6 MB/s | 645 kB 00:00 2024-05-08T23:08:44.541 INFO:teuthology.orchestra.run.smithi155.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2024-05-08T23:08:44.555 INFO:teuthology.orchestra.run.smithi189.stdout:(90/146): libquadmath-8.5.0-20.el8.x86_64.rpm 850 kB/s | 172 kB 00:00 2024-05-08T23:08:44.598 INFO:teuthology.orchestra.run.smithi155.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2024-05-08T23:08:44.639 INFO:teuthology.orchestra.run.smithi189.stdout:(91/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 435 kB/s | 73 kB 00:00 2024-05-08T23:08:44.672 INFO:teuthology.orchestra.run.smithi189.stdout:(92/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 MB/s | 238 kB 00:00 2024-05-08T23:08:44.706 INFO:teuthology.orchestra.run.smithi189.stdout:(93/146): elfutils-libelf-devel-0.189-3.el8.x86 410 kB/s | 62 kB 00:00 2024-05-08T23:08:44.723 INFO:teuthology.orchestra.run.smithi155.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 600 kB/s | 75 kB 00:00 2024-05-08T23:08:44.815 INFO:teuthology.orchestra.run.smithi189.stdout:(94/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.1 MB/s | 562 kB 00:00 2024-05-08T23:08:44.823 INFO:teuthology.orchestra.run.smithi155.stdout:(40/146): python3-asyncssh-2.7.0-2.el8.noarch.r 4.7 MB/s | 476 kB 00:00 2024-05-08T23:08:44.831 INFO:teuthology.orchestra.run.smithi189.stdout:(95/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 376 kB/s | 47 kB 00:00 2024-05-08T23:08:44.874 INFO:teuthology.orchestra.run.smithi155.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 872 kB/s | 44 kB 00:00 2024-05-08T23:08:44.915 INFO:teuthology.orchestra.run.smithi189.stdout:(96/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 9.6 MB/s | 2.3 MB 00:00 2024-05-08T23:08:44.933 INFO:teuthology.orchestra.run.smithi155.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2024-05-08T23:08:44.982 INFO:teuthology.orchestra.run.smithi189.stdout:(97/146): python3-cryptography-3.2.1-7.el8_9.x8 3.3 MB/s | 559 kB 00:00 2024-05-08T23:08:44.983 INFO:teuthology.orchestra.run.smithi155.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 660 kB/s | 33 kB 00:00 2024-05-08T23:08:45.016 INFO:teuthology.orchestra.run.smithi189.stdout:(98/146): NetworkManager-initscripts-updown-1.4 785 kB/s | 144 kB 00:00 2024-05-08T23:08:45.033 INFO:teuthology.orchestra.run.smithi189.stdout:(99/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 853 kB/s | 100 kB 00:00 2024-05-08T23:08:45.034 INFO:teuthology.orchestra.run.smithi155.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 327 kB/s | 16 kB 00:00 2024-05-08T23:08:45.100 INFO:teuthology.orchestra.run.smithi155.stdout:(45/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2024-05-08T23:08:45.148 INFO:teuthology.orchestra.run.smithi189.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 121 kB/s | 20 kB 00:00 2024-05-08T23:08:45.159 INFO:teuthology.orchestra.run.smithi155.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2024-05-08T23:08:45.175 INFO:teuthology.orchestra.run.smithi189.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 277 kB/s | 44 kB 00:00 2024-05-08T23:08:45.217 INFO:teuthology.orchestra.run.smithi189.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 599 kB/s | 110 kB 00:00 2024-05-08T23:08:45.217 INFO:teuthology.orchestra.run.smithi155.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2024-05-08T23:08:45.260 INFO:teuthology.orchestra.run.smithi155.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2024-05-08T23:08:45.301 INFO:teuthology.orchestra.run.smithi155.stdout:(49/146): python3-jaraco-functools-2.0-4.el8.no 424 kB/s | 18 kB 00:00 2024-05-08T23:08:45.334 INFO:teuthology.orchestra.run.smithi016.stdout:(25/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 1.6 MB/s | 6.5 MB 00:04 2024-05-08T23:08:45.351 INFO:teuthology.orchestra.run.smithi189.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 131 kB/s | 27 kB 00:00 2024-05-08T23:08:45.376 INFO:teuthology.orchestra.run.smithi189.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 629 kB/s | 126 kB 00:00 2024-05-08T23:08:45.401 INFO:teuthology.orchestra.run.smithi155.stdout:(50/146): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2024-05-08T23:08:45.435 INFO:teuthology.orchestra.run.smithi189.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2024-05-08T23:08:45.444 INFO:teuthology.orchestra.run.smithi155.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2024-05-08T23:08:45.467 INFO:teuthology.orchestra.run.smithi016.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.3 MB/s | 181 kB 00:00 2024-05-08T23:08:45.526 INFO:teuthology.orchestra.run.smithi189.stdout:(106/146): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2024-05-08T23:08:45.527 INFO:teuthology.orchestra.run.smithi155.stdout:(52/146): ceph-mgr-diskprediction-local-18.2.2- 2.8 MB/s | 7.4 MB 00:02 2024-05-08T23:08:45.544 INFO:teuthology.orchestra.run.smithi155.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 585 kB/s | 59 kB 00:00 2024-05-08T23:08:45.550 INFO:teuthology.orchestra.run.smithi016.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 292 kB/s | 24 kB 00:00 2024-05-08T23:08:45.551 INFO:teuthology.orchestra.run.smithi189.stdout:(107/146): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2024-05-08T23:08:45.585 INFO:teuthology.orchestra.run.smithi189.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2024-05-08T23:08:45.619 INFO:teuthology.orchestra.run.smithi189.stdout:(109/146): libtool-2.4.6-25.el8.x86_64.rpm 3.8 MB/s | 709 kB 00:00 2024-05-08T23:08:45.619 INFO:teuthology.orchestra.run.smithi155.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2024-05-08T23:08:45.662 INFO:teuthology.orchestra.run.smithi155.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 373 kB/s | 16 kB 00:00 2024-05-08T23:08:45.678 INFO:teuthology.orchestra.run.smithi155.stdout:(56/146): python3-natsort-7.1.1-2.el8.noarch.rp 400 kB/s | 60 kB 00:00 2024-05-08T23:08:45.692 INFO:teuthology.orchestra.run.smithi016.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 979 kB/s | 138 kB 00:00 2024-05-08T23:08:45.704 INFO:teuthology.orchestra.run.smithi155.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 800 kB/s | 34 kB 00:00 2024-05-08T23:08:45.711 INFO:teuthology.orchestra.run.smithi189.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2024-05-08T23:08:45.736 INFO:teuthology.orchestra.run.smithi189.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 425 kB/s | 64 kB 00:00 2024-05-08T23:08:45.788 INFO:teuthology.orchestra.run.smithi155.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 768 kB/s | 64 kB 00:00 2024-05-08T23:08:45.794 INFO:teuthology.orchestra.run.smithi189.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 590 kB/s | 103 kB 00:00 2024-05-08T23:08:45.814 INFO:teuthology.orchestra.run.smithi155.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2024-05-08T23:08:45.834 INFO:teuthology.orchestra.run.smithi016.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 1.6 MB/s | 3.5 MB 00:02 2024-05-08T23:08:45.838 INFO:teuthology.orchestra.run.smithi155.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2024-05-08T23:08:45.861 INFO:teuthology.orchestra.run.smithi189.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 4.2 MB/s | 538 kB 00:00 2024-05-08T23:08:45.864 INFO:teuthology.orchestra.run.smithi155.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2024-05-08T23:08:45.889 INFO:teuthology.orchestra.run.smithi155.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2024-05-08T23:08:45.914 INFO:teuthology.orchestra.run.smithi155.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 546 kB/s | 27 kB 00:00 2024-05-08T23:08:45.956 INFO:teuthology.orchestra.run.smithi155.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2024-05-08T23:08:45.995 INFO:teuthology.orchestra.run.smithi189.stdout:(114/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-05-08T23:08:46.029 INFO:teuthology.orchestra.run.smithi189.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.9 MB/s | 323 kB 00:00 2024-05-08T23:08:46.031 INFO:teuthology.orchestra.run.smithi155.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2024-05-08T23:08:46.057 INFO:teuthology.orchestra.run.smithi155.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2024-05-08T23:08:46.099 INFO:teuthology.orchestra.run.smithi155.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2024-05-08T23:08:46.120 INFO:teuthology.orchestra.run.smithi189.stdout:(116/146): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2024-05-08T23:08:46.124 INFO:teuthology.orchestra.run.smithi155.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 344 kB/s | 23 kB 00:00 2024-05-08T23:08:46.145 INFO:teuthology.orchestra.run.smithi189.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2024-05-08T23:08:46.149 INFO:teuthology.orchestra.run.smithi155.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T23:08:46.213 INFO:teuthology.orchestra.run.smithi189.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.9 MB/s | 354 kB 00:00 2024-05-08T23:08:46.267 INFO:teuthology.orchestra.run.smithi155.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2024-05-08T23:08:46.296 INFO:teuthology.orchestra.run.smithi189.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-05-08T23:08:46.342 INFO:teuthology.orchestra.run.smithi155.stdout:(71/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 93 kB/s | 18 kB 00:00 2024-05-08T23:08:46.380 INFO:teuthology.orchestra.run.smithi189.stdout:(120/146): openblas-threads-0.3.15-6.el8.x86_64 20 MB/s | 5.2 MB 00:00 2024-05-08T23:08:46.409 INFO:teuthology.orchestra.run.smithi155.stdout:(72/146): python3-jwt-1.6.1-2.el8.noarch.rpm 300 kB/s | 43 kB 00:00 2024-05-08T23:08:46.497 INFO:teuthology.orchestra.run.smithi189.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2024-05-08T23:08:46.518 INFO:teuthology.orchestra.run.smithi155.stdout:(73/146): python3-oauthlib-2.1.0-1.el8.noarch.r 884 kB/s | 155 kB 00:00 2024-05-08T23:08:46.530 INFO:teuthology.orchestra.run.smithi189.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 3.0 MB/s | 714 kB 00:00 2024-05-08T23:08:46.556 INFO:teuthology.orchestra.run.smithi189.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-05-08T23:08:46.593 INFO:teuthology.orchestra.run.smithi155.stdout:(74/146): python3-pycparser-2.14-14.el8.noarch. 595 kB/s | 109 kB 00:00 2024-05-08T23:08:46.707 INFO:teuthology.orchestra.run.smithi189.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 963 kB/s | 202 kB 00:00 2024-05-08T23:08:46.760 INFO:teuthology.orchestra.run.smithi155.stdout:(75/146): python3-requests-oauthlib-1.0.0-1.el8 176 kB/s | 43 kB 00:00 2024-05-08T23:08:46.777 INFO:teuthology.orchestra.run.smithi155.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 102 kB/s | 19 kB 00:00 2024-05-08T23:08:46.798 INFO:teuthology.orchestra.run.smithi189.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 203 kB/s | 54 kB 00:00 2024-05-08T23:08:46.823 INFO:teuthology.orchestra.run.smithi189.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-08T23:08:46.869 INFO:teuthology.orchestra.run.smithi155.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 404 kB/s | 44 kB 00:00 2024-05-08T23:08:46.902 INFO:teuthology.orchestra.run.smithi155.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2024-05-08T23:08:46.994 INFO:teuthology.orchestra.run.smithi155.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2024-05-08T23:08:47.028 INFO:teuthology.orchestra.run.smithi155.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-05-08T23:08:47.040 INFO:teuthology.orchestra.run.smithi189.stdout:(127/146): python3-scipy-1.0.0-21.module+el8.9. 28 MB/s | 14 MB 00:00 2024-05-08T23:08:47.070 INFO:teuthology.orchestra.run.smithi155.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2024-05-08T23:08:47.074 INFO:teuthology.orchestra.run.smithi189.stdout:(128/146): postgresql-contrib-10.23-4.module+el 3.2 MB/s | 812 kB 00:00 2024-05-08T23:08:47.116 INFO:teuthology.orchestra.run.smithi189.stdout:(129/146): postgresql-10.23-4.module+el8.9.0+21 4.7 MB/s | 1.5 MB 00:00 2024-05-08T23:08:47.145 INFO:teuthology.orchestra.run.smithi155.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 411 kB/s | 48 kB 00:00 2024-05-08T23:08:47.229 INFO:teuthology.orchestra.run.smithi155.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 549 kB/s | 87 kB 00:00 2024-05-08T23:08:47.283 INFO:teuthology.orchestra.run.smithi189.stdout:(130/146): postgresql-server-10.23-4.module+el8 21 MB/s | 5.1 MB 00:00 2024-05-08T23:08:47.298 INFO:teuthology.orchestra.run.smithi155.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 378 kB/s | 58 kB 00:00 2024-05-08T23:08:47.407 INFO:teuthology.orchestra.run.smithi155.stdout:(85/146): libcom_err-devel-1.45.6-4.el8.x86_64. 217 kB/s | 39 kB 00:00 2024-05-08T23:08:47.416 INFO:teuthology.orchestra.run.smithi189.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-05-08T23:08:47.440 INFO:teuthology.orchestra.run.smithi155.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 585 kB/s | 83 kB 00:00 2024-05-08T23:08:47.524 INFO:teuthology.orchestra.run.smithi155.stdout:(87/146): libgfortran-8.5.0-20.el8.x86_64.rpm 5.4 MB/s | 645 kB 00:00 2024-05-08T23:08:47.568 INFO:teuthology.orchestra.run.smithi016.stdout:(30/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 2.0 MB/s | 47 MB 00:23 2024-05-08T23:08:47.626 INFO:teuthology.orchestra.run.smithi016.stdout:(31/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 1.8 MB/s | 3.5 MB 00:01 2024-05-08T23:08:47.649 INFO:teuthology.orchestra.run.smithi155.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 827 kB/s | 172 kB 00:00 2024-05-08T23:08:47.650 INFO:teuthology.orchestra.run.smithi189.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.3 MB/s | 303 kB 00:00 2024-05-08T23:08:47.674 INFO:teuthology.orchestra.run.smithi155.stdout:(89/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2024-05-08T23:08:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 536 kB/s | 49 kB 00:00 2024-05-08T23:08:47.750 INFO:teuthology.orchestra.run.smithi155.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 725 kB/s | 73 kB 00:00 2024-05-08T23:08:47.819 INFO:teuthology.orchestra.run.smithi016.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 163 kB/s | 16 kB 00:00 2024-05-08T23:08:47.833 INFO:teuthology.orchestra.run.smithi189.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.6 MB/s | 854 kB 00:00 2024-05-08T23:08:47.858 INFO:teuthology.orchestra.run.smithi155.stdout:(91/146): elfutils-libelf-devel-0.189-3.el8.x86 336 kB/s | 62 kB 00:00 2024-05-08T23:08:47.886 INFO:teuthology.orchestra.run.smithi016.stdout:(34/146): ceph-mgr-modules-core-18.2.2-1322.gd2 774 kB/s | 245 kB 00:00 2024-05-08T23:08:47.942 INFO:teuthology.orchestra.run.smithi155.stdout:(92/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.9 MB/s | 562 kB 00:00 2024-05-08T23:08:48.009 INFO:teuthology.orchestra.run.smithi189.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.0 MB/s | 711 kB 00:00 2024-05-08T23:08:48.011 INFO:teuthology.orchestra.run.smithi016.stdout:(35/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 1.3 MB/s | 263 kB 00:00 2024-05-08T23:08:48.061 INFO:teuthology.orchestra.run.smithi016.stdout:(36/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 971 kB/s | 170 kB 00:00 2024-05-08T23:08:48.067 INFO:teuthology.orchestra.run.smithi155.stdout:(93/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-08T23:08:48.093 INFO:teuthology.orchestra.run.smithi155.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 313 kB/s | 47 kB 00:00 2024-05-08T23:08:48.094 INFO:teuthology.orchestra.run.smithi016.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 216 kB/s | 18 kB 00:00 2024-05-08T23:08:48.184 INFO:teuthology.orchestra.run.smithi189.stdout:(135/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-08T23:08:48.234 INFO:teuthology.orchestra.run.smithi189.stdout:(136/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 3.2 MB/s | 3.6 MB 00:01 2024-05-08T23:08:48.237 INFO:teuthology.orchestra.run.smithi016.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 432 kB/s | 59 kB 00:00 2024-05-08T23:08:48.259 INFO:teuthology.orchestra.run.smithi155.stdout:(95/146): python3-cryptography-3.2.1-7.el8_9.x8 2.8 MB/s | 559 kB 00:00 2024-05-08T23:08:48.260 INFO:teuthology.orchestra.run.smithi189.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 4.3 MB/s | 333 kB 00:00 2024-05-08T23:08:48.276 INFO:teuthology.orchestra.run.smithi155.stdout:(96/146): NetworkManager-initscripts-updown-1.4 787 kB/s | 144 kB 00:00 2024-05-08T23:08:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2024-05-08T23:08:48.295 INFO:teuthology.orchestra.run.smithi016.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.2 MB/s | 75 kB 00:00 2024-05-08T23:08:48.346 INFO:teuthology.orchestra.run.smithi016.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 874 kB/s | 44 kB 00:00 2024-05-08T23:08:48.371 INFO:teuthology.orchestra.run.smithi016.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 5.0 MB/s | 476 kB 00:00 2024-05-08T23:08:48.376 INFO:teuthology.orchestra.run.smithi189.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-05-08T23:08:48.393 INFO:teuthology.orchestra.run.smithi189.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 221 kB 00:00 2024-05-08T23:08:48.402 INFO:teuthology.orchestra.run.smithi155.stdout:(97/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 704 kB/s | 100 kB 00:00 2024-05-08T23:08:48.413 INFO:teuthology.orchestra.run.smithi016.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2024-05-08T23:08:48.427 INFO:teuthology.orchestra.run.smithi155.stdout:(98/146): python3-packaging-16.8-9.el8.noarch.r 321 kB/s | 48 kB 00:00 2024-05-08T23:08:48.447 INFO:teuthology.orchestra.run.smithi016.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 437 kB/s | 33 kB 00:00 2024-05-08T23:08:48.460 INFO:teuthology.orchestra.run.smithi189.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 3.0 MB/s | 254 kB 00:00 2024-05-08T23:08:48.472 INFO:teuthology.orchestra.run.smithi016.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 277 kB/s | 16 kB 00:00 2024-05-08T23:08:48.489 INFO:teuthology.orchestra.run.smithi016.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 4.0 MB/s | 173 kB 00:00 2024-05-08T23:08:48.527 INFO:teuthology.orchestra.run.smithi189.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 237 kB 00:00 2024-05-08T23:08:48.528 INFO:teuthology.orchestra.run.smithi155.stdout:(99/146): python3-prettytable-0.7.2-14.el8.noar 439 kB/s | 44 kB 00:00 2024-05-08T23:08:48.540 INFO:teuthology.orchestra.run.smithi016.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2024-05-08T23:08:48.565 INFO:teuthology.orchestra.run.smithi016.stdout:(48/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2024-05-08T23:08:48.569 INFO:teuthology.orchestra.run.smithi155.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 120 kB/s | 20 kB 00:00 2024-05-08T23:08:48.590 INFO:teuthology.orchestra.run.smithi016.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2024-05-08T23:08:48.615 INFO:teuthology.orchestra.run.smithi016.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2024-05-08T23:08:48.627 INFO:teuthology.orchestra.run.smithi189.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 180 kB 00:00 2024-05-08T23:08:48.661 INFO:teuthology.orchestra.run.smithi189.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 741 kB/s | 99 kB 00:00 2024-05-08T23:08:48.674 INFO:teuthology.orchestra.run.smithi016.stdout:(51/146): python3-kubernetes-11.0.0-6.el8.noarc 14 MB/s | 1.2 MB 00:00 2024-05-08T23:08:48.686 INFO:teuthology.orchestra.run.smithi155.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 695 kB/s | 110 kB 00:00 2024-05-08T23:08:48.720 INFO:teuthology.orchestra.run.smithi016.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 467 kB/s | 49 kB 00:00 2024-05-08T23:08:48.750 INFO:teuthology.orchestra.run.smithi016.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 780 kB/s | 59 kB 00:00 2024-05-08T23:08:48.753 INFO:teuthology.orchestra.run.smithi155.stdout:(102/146): python3-jsonpatch-1.21-2.el8.noarch. 144 kB/s | 27 kB 00:00 2024-05-08T23:08:48.775 INFO:teuthology.orchestra.run.smithi016.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.1 MB/s | 60 kB 00:00 2024-05-08T23:08:48.795 INFO:teuthology.orchestra.run.smithi189.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 15 MB/s | 2.5 MB 00:00 2024-05-08T23:08:48.800 INFO:teuthology.orchestra.run.smithi016.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2024-05-08T23:08:48.820 INFO:teuthology.orchestra.run.smithi189.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 648 kB/s | 103 kB 00:00 2024-05-08T23:08:48.826 INFO:teuthology.orchestra.run.smithi016.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-05-08T23:08:48.851 INFO:teuthology.orchestra.run.smithi016.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2024-05-08T23:08:48.876 INFO:teuthology.orchestra.run.smithi016.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2024-05-08T23:08:48.886 INFO:teuthology.orchestra.run.smithi155.stdout:(103/146): python3-werkzeug-0.12.2-4.el8.noarch 3.4 MB/s | 457 kB 00:00 2024-05-08T23:08:48.902 INFO:teuthology.orchestra.run.smithi016.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2024-05-08T23:08:48.912 INFO:teuthology.orchestra.run.smithi155.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 561 kB/s | 126 kB 00:00 2024-05-08T23:08:48.927 INFO:teuthology.orchestra.run.smithi016.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2024-05-08T23:08:48.952 INFO:teuthology.orchestra.run.smithi016.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2024-05-08T23:08:48.978 INFO:teuthology.orchestra.run.smithi016.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 584 kB/s | 29 kB 00:00 2024-05-08T23:08:48.987 INFO:teuthology.orchestra.run.smithi189.stdout:(146/146): librados2-18.2.2-1322.gd21e4f96.el8. 1.9 MB/s | 3.7 MB 00:01 2024-05-08T23:08:48.990 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:08:48.990 INFO:teuthology.orchestra.run.smithi189.stdout:Total 6.1 MB/s | 215 MB 00:35 2024-05-08T23:08:49.003 INFO:teuthology.orchestra.run.smithi016.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 547 kB/s | 27 kB 00:00 2024-05-08T23:08:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2024-05-08T23:08:49.053 INFO:teuthology.orchestra.run.smithi016.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2024-05-08T23:08:49.079 INFO:teuthology.orchestra.run.smithi016.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2024-05-08T23:08:49.096 INFO:teuthology.orchestra.run.smithi155.stdout:(105/146): bison-3.0.4-10.el8.x86_64.rpm 3.2 MB/s | 688 kB 00:00 2024-05-08T23:08:49.104 INFO:teuthology.orchestra.run.smithi016.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2024-05-08T23:08:49.129 INFO:teuthology.orchestra.run.smithi155.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 1.4 MB/s | 320 kB 00:00 2024-05-08T23:08:49.130 INFO:teuthology.orchestra.run.smithi016.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2024-05-08T23:08:49.146 INFO:teuthology.orchestra.run.smithi016.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-05-08T23:08:49.197 INFO:teuthology.orchestra.run.smithi016.stdout:(70/146): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2024-05-08T23:08:49.263 INFO:teuthology.orchestra.run.smithi155.stdout:(107/146): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2024-05-08T23:08:49.280 INFO:teuthology.orchestra.run.smithi155.stdout:(108/146): python3-markupsafe-0.23-19.el8.x86_6 258 kB/s | 39 kB 00:00 2024-05-08T23:08:49.372 INFO:teuthology.orchestra.run.smithi016.stdout:(71/146): python3-prettytable-0.7.2-14.el8.noar 264 kB/s | 44 kB 00:00 2024-05-08T23:08:49.389 INFO:teuthology.orchestra.run.smithi016.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 82 kB/s | 20 kB 00:00 2024-05-08T23:08:49.397 INFO:teuthology.orchestra.run.smithi155.stdout:(109/146): uuid-1.6.2-43.el8.x86_64.rpm 550 kB/s | 64 kB 00:00 2024-05-08T23:08:49.430 INFO:teuthology.orchestra.run.smithi155.stdout:(110/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2024-05-08T23:08:49.473 INFO:teuthology.orchestra.run.smithi016.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2024-05-08T23:08:49.490 INFO:teuthology.orchestra.run.smithi016.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 264 kB/s | 27 kB 00:00 2024-05-08T23:08:49.523 INFO:teuthology.orchestra.run.smithi016.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2024-05-08T23:08:49.547 INFO:teuthology.orchestra.run.smithi155.stdout:(111/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 689 kB/s | 103 kB 00:00 2024-05-08T23:08:49.632 INFO:teuthology.orchestra.run.smithi016.stdout:(76/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.1 MB/s | 457 kB 00:00 2024-05-08T23:08:49.706 INFO:teuthology.orchestra.run.smithi155.stdout:(112/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-05-08T23:08:49.707 INFO:teuthology.orchestra.run.smithi016.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2024-05-08T23:08:49.740 INFO:teuthology.orchestra.run.smithi155.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 2.7 MB/s | 538 kB 00:00 2024-05-08T23:08:49.774 INFO:teuthology.orchestra.run.smithi016.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-05-08T23:08:49.833 INFO:teuthology.orchestra.run.smithi016.stdout:(79/146): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2024-05-08T23:08:49.923 INFO:teuthology.orchestra.run.smithi155.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2024-05-08T23:08:49.948 INFO:teuthology.orchestra.run.smithi155.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-05-08T23:08:49.967 INFO:teuthology.orchestra.run.smithi016.stdout:(80/146): ceph-mgr-diskprediction-local-18.2.2- 1.8 MB/s | 7.4 MB 00:04 2024-05-08T23:08:49.995 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-05-08T23:08:50.008 INFO:teuthology.orchestra.run.smithi016.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 166 kB/s | 39 kB 00:00 2024-05-08T23:08:50.042 INFO:teuthology.orchestra.run.smithi016.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-05-08T23:08:50.049 INFO:teuthology.orchestra.run.smithi155.stdout:(116/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2024-05-08T23:08:50.118 INFO:teuthology.orchestra.run.smithi016.stdout:(83/146): uuid-1.6.2-43.el8.x86_64.rpm 426 kB/s | 64 kB 00:00 2024-05-08T23:08:50.132 INFO:teuthology.orchestra.run.smithi155.stdout:(117/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.9 MB/s | 354 kB 00:00 2024-05-08T23:08:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 592 kB/s | 103 kB 00:00 2024-05-08T23:08:50.243 INFO:teuthology.orchestra.run.smithi016.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 4.2 MB/s | 538 kB 00:00 2024-05-08T23:08:50.249 INFO:teuthology.orchestra.run.smithi155.stdout:(118/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-05-08T23:08:50.310 INFO:teuthology.orchestra.run.smithi016.stdout:(86/146): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2024-05-08T23:08:50.391 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-05-08T23:08:50.391 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-05-08T23:08:50.402 INFO:teuthology.orchestra.run.smithi016.stdout:(87/146): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2024-05-08T23:08:50.427 INFO:teuthology.orchestra.run.smithi016.stdout:(88/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2024-05-08T23:08:50.471 INFO:teuthology.orchestra.run.smithi155.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 12 MB/s | 5.2 MB 00:00 2024-05-08T23:08:50.502 INFO:teuthology.orchestra.run.smithi016.stdout:(89/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-05-08T23:08:50.536 INFO:teuthology.orchestra.run.smithi016.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2024-05-08T23:08:50.595 INFO:teuthology.orchestra.run.smithi016.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-05-08T23:08:50.687 INFO:teuthology.orchestra.run.smithi016.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 20 MB/s | 5.2 MB 00:00 2024-05-08T23:08:50.729 INFO:teuthology.orchestra.run.smithi016.stdout:(93/146): automake-1.16.1-8.el8.noarch.rpm 5.2 MB/s | 714 kB 00:00 2024-05-08T23:08:50.846 INFO:teuthology.orchestra.run.smithi016.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-05-08T23:08:50.871 INFO:teuthology.orchestra.run.smithi016.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2024-05-08T23:08:50.896 INFO:teuthology.orchestra.run.smithi016.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-05-08T23:08:50.917 INFO:teuthology.orchestra.run.smithi155.stdout:(120/146): ceph-test-18.2.2-1322.gd21e4f96.el8. 1.6 MB/s | 47 MB 00:29 2024-05-08T23:08:50.921 INFO:teuthology.orchestra.run.smithi016.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 720 kB/s | 54 kB 00:00 2024-05-08T23:08:50.958 INFO:teuthology.orchestra.run.smithi155.stdout:(121/146): automake-1.16.1-8.el8.noarch.rpm 1.4 MB/s | 714 kB 00:00 2024-05-08T23:08:50.997 INFO:teuthology.orchestra.run.smithi016.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-08T23:08:51.042 INFO:teuthology.orchestra.run.smithi155.stdout:(122/146): openblas-0.3.15-6.el8.x86_64.rpm 6.4 MB/s | 5.0 MB 00:00 2024-05-08T23:08:51.059 INFO:teuthology.orchestra.run.smithi155.stdout:(123/146): jq-1.6-7.el8.x86_64.rpm 2.1 MB/s | 202 kB 00:00 2024-05-08T23:08:51.076 INFO:teuthology.orchestra.run.smithi155.stdout:(124/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-05-08T23:08:51.148 INFO:teuthology.orchestra.run.smithi016.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 6.7 MB/s | 1.5 MB 00:00 2024-05-08T23:08:51.160 INFO:teuthology.orchestra.run.smithi155.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 463 kB/s | 54 kB 00:00 2024-05-08T23:08:51.189 INFO:teuthology.orchestra.run.smithi016.stdout:(100/146): postgresql-contrib-10.23-4.module+el 4.1 MB/s | 812 kB 00:00 2024-05-08T23:08:51.318 INFO:teuthology.orchestra.run.smithi155.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 778 kB/s | 188 kB 00:00 2024-05-08T23:08:51.381 INFO:teuthology.orchestra.run.smithi016.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 27 MB/s | 14 MB 00:00 2024-05-08T23:08:51.427 INFO:teuthology.orchestra.run.smithi155.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 5.6 MB/s | 1.5 MB 00:00 2024-05-08T23:08:51.469 INFO:teuthology.orchestra.run.smithi155.stdout:(128/146): postgresql-contrib-10.23-4.module+el 5.3 MB/s | 812 kB 00:00 2024-05-08T23:08:51.531 INFO:teuthology.orchestra.run.smithi016.stdout:(102/146): postgresql-server-10.23-4.module+el8 13 MB/s | 5.1 MB 00:00 2024-05-08T23:08:51.557 INFO:teuthology.orchestra.run.smithi016.stdout:(103/146): python3-packaging-16.8-9.el8.noarch. 131 kB/s | 48 kB 00:00 2024-05-08T23:08:51.582 INFO:teuthology.orchestra.run.smithi016.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 89 kB/s | 18 kB 00:00 2024-05-08T23:08:51.599 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-05-08T23:08:51.609 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-05-08T23:08:51.636 INFO:teuthology.orchestra.run.smithi155.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 24 MB/s | 14 MB 00:00 2024-05-08T23:08:51.674 INFO:teuthology.orchestra.run.smithi016.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 299 kB/s | 43 kB 00:00 2024-05-08T23:08:51.741 INFO:teuthology.orchestra.run.smithi016.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 842 kB/s | 155 kB 00:00 2024-05-08T23:08:51.769 INFO:teuthology.orchestra.run.smithi155.stdout:(130/146): postgresql-server-10.23-4.module+el8 15 MB/s | 5.1 MB 00:00 2024-05-08T23:08:51.799 INFO:teuthology.orchestra.run.smithi016.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 502 kB/s | 109 kB 00:00 2024-05-08T23:08:51.825 INFO:teuthology.orchestra.run.smithi016.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 284 kB/s | 43 kB 00:00 2024-05-08T23:08:51.892 INFO:teuthology.orchestra.run.smithi016.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 124 kB/s | 19 kB 00:00 2024-05-08T23:08:51.911 INFO:teuthology.orchestra.run.smithi155.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T23:08:51.917 INFO:teuthology.orchestra.run.smithi016.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 374 kB/s | 44 kB 00:00 2024-05-08T23:08:52.034 INFO:teuthology.orchestra.run.smithi016.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2024-05-08T23:08:52.059 INFO:teuthology.orchestra.run.smithi016.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-05-08T23:08:52.093 INFO:teuthology.orchestra.run.smithi016.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2024-05-08T23:08:52.153 INFO:teuthology.orchestra.run.smithi155.stdout:(132/146): librados2-18.2.2-1322.gd21e4f96.el8. 5.4 MB/s | 3.7 MB 00:00 2024-05-08T23:08:52.178 INFO:teuthology.orchestra.run.smithi155.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.1 MB/s | 303 kB 00:00 2024-05-08T23:08:52.227 INFO:teuthology.orchestra.run.smithi016.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 288 kB/s | 48 kB 00:00 2024-05-08T23:08:52.243 INFO:teuthology.orchestra.run.smithi016.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 579 kB/s | 87 kB 00:00 2024-05-08T23:08:52.269 INFO:teuthology.orchestra.run.smithi016.stdout:(116/146): libselinux-devel-2.9-5.el8.x86_64.rp 853 kB/s | 200 kB 00:00 2024-05-08T23:08:52.312 INFO:teuthology.orchestra.run.smithi155.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.2 MB/s | 711 kB 00:00 2024-05-08T23:08:52.346 INFO:teuthology.orchestra.run.smithi155.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.3 MB/s | 854 kB 00:00 2024-05-08T23:08:52.353 INFO:teuthology.orchestra.run.smithi016.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 460 kB/s | 58 kB 00:00 2024-05-08T23:08:52.421 INFO:teuthology.orchestra.run.smithi155.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 14 MB/s | 1.5 MB 00:00 2024-05-08T23:08:52.478 INFO:teuthology.orchestra.run.smithi016.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 474 kB/s | 83 kB 00:00 2024-05-08T23:08:52.479 INFO:teuthology.orchestra.run.smithi016.stdout:(119/146): libcom_err-devel-1.45.6-4.el8.x86_64 171 kB/s | 39 kB 00:00 2024-05-08T23:08:52.521 INFO:teuthology.orchestra.run.smithi155.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-05-08T23:08:52.562 INFO:teuthology.orchestra.run.smithi016.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.0 MB/s | 645 kB 00:00 2024-05-08T23:08:52.568 INFO:teuthology.orchestra.run.smithi155.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-05-08T23:08:52.587 INFO:teuthology.orchestra.run.smithi016.stdout:(121/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2024-05-08T23:08:52.613 INFO:teuthology.orchestra.run.smithi016.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2024-05-08T23:08:52.641 INFO:teuthology.orchestra.run.smithi155.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.8 MB/s | 221 kB 00:00 2024-05-08T23:08:52.713 INFO:teuthology.orchestra.run.smithi016.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 482 kB/s | 73 kB 00:00 2024-05-08T23:08:52.713 INFO:teuthology.orchestra.run.smithi155.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 254 kB 00:00 2024-05-08T23:08:52.746 INFO:teuthology.orchestra.run.smithi016.stdout:(124/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.1 MB/s | 562 kB 00:00 2024-05-08T23:08:52.781 INFO:teuthology.orchestra.run.smithi155.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 237 kB 00:00 2024-05-08T23:08:52.805 INFO:teuthology.orchestra.run.smithi016.stdout:(125/146): elfutils-libelf-devel-0.189-3.el8.x8 284 kB/s | 62 kB 00:00 2024-05-08T23:08:52.872 INFO:teuthology.orchestra.run.smithi016.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 375 kB/s | 47 kB 00:00 2024-05-08T23:08:52.881 INFO:teuthology.orchestra.run.smithi155.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.0 MB/s | 180 kB 00:00 2024-05-08T23:08:52.915 INFO:teuthology.orchestra.run.smithi155.stdout:(143/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 740 kB/s | 99 kB 00:00 2024-05-08T23:08:52.956 INFO:teuthology.orchestra.run.smithi016.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 9.6 MB/s | 2.3 MB 00:00 2024-05-08T23:08:52.982 INFO:teuthology.orchestra.run.smithi155.stdout:(144/146): util-linux-2.32.1-44.el8_9.1.x86_64. 25 MB/s | 2.5 MB 00:00 2024-05-08T23:08:52.989 INFO:teuthology.orchestra.run.smithi016.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 3.0 MB/s | 559 kB 00:00 2024-05-08T23:08:53.015 INFO:teuthology.orchestra.run.smithi016.stdout:(129/146): NetworkManager-initscripts-updown-1. 1.0 MB/s | 144 kB 00:00 2024-05-08T23:08:53.057 INFO:teuthology.orchestra.run.smithi155.stdout:(145/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 2.5 MB/s | 3.6 MB 00:01 2024-05-08T23:08:53.073 INFO:teuthology.orchestra.run.smithi016.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 852 kB/s | 100 kB 00:00 2024-05-08T23:08:53.082 INFO:teuthology.orchestra.run.smithi155.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 616 kB/s | 103 kB 00:00 2024-05-08T23:08:53.084 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:08:53.084 INFO:teuthology.orchestra.run.smithi155.stdout:Total 5.3 MB/s | 215 MB 00:40 2024-05-08T23:08:53.165 INFO:teuthology.orchestra.run.smithi016.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 2.5 MB/s | 232 kB 00:00 2024-05-08T23:08:53.290 INFO:teuthology.orchestra.run.smithi016.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.4 MB/s | 303 kB 00:00 2024-05-08T23:08:53.432 INFO:teuthology.orchestra.run.smithi016.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-05-08T23:08:53.532 INFO:teuthology.orchestra.run.smithi016.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 6.9 MB/s | 711 kB 00:00 2024-05-08T23:08:53.716 INFO:teuthology.orchestra.run.smithi016.stdout:(135/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.0 MB/s | 1.5 MB 00:00 2024-05-08T23:08:53.875 INFO:teuthology.orchestra.run.smithi016.stdout:(136/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 4.1 MB/s | 3.6 MB 00:00 2024-05-08T23:08:53.908 INFO:teuthology.orchestra.run.smithi016.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.7 MB/s | 333 kB 00:00 2024-05-08T23:08:54.059 INFO:teuthology.orchestra.run.smithi016.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-05-08T23:08:54.092 INFO:teuthology.orchestra.run.smithi016.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.2 MB/s | 221 kB 00:00 2024-05-08T23:08:54.104 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-05-08T23:08:54.201 INFO:teuthology.orchestra.run.smithi016.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 254 kB 00:00 2024-05-08T23:08:54.309 INFO:teuthology.orchestra.run.smithi016.stdout:(141/146): librados2-18.2.2-1322.gd21e4f96.el8. 2.8 MB/s | 3.7 MB 00:01 2024-05-08T23:08:54.326 INFO:teuthology.orchestra.run.smithi016.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.0 MB/s | 237 kB 00:00 2024-05-08T23:08:54.352 INFO:teuthology.orchestra.run.smithi016.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.2 MB/s | 180 kB 00:00 2024-05-08T23:08:54.435 INFO:teuthology.orchestra.run.smithi016.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 790 kB/s | 99 kB 00:00 2024-05-08T23:08:54.499 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-05-08T23:08:54.499 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-05-08T23:08:54.510 INFO:teuthology.orchestra.run.smithi016.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 13 MB/s | 2.5 MB 00:00 2024-05-08T23:08:54.611 INFO:teuthology.orchestra.run.smithi016.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 397 kB/s | 103 kB 00:00 2024-05-08T23:08:54.615 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:08:54.615 INFO:teuthology.orchestra.run.smithi016.stdout:Total 5.8 MB/s | 215 MB 00:36 2024-05-08T23:08:55.707 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-05-08T23:08:55.709 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-05-08T23:08:55.717 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-05-08T23:08:56.161 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-05-08T23:08:56.161 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-05-08T23:08:57.444 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-05-08T23:08:57.490 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-05-08T23:08:57.501 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-05-08T23:08:57.578 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:08:57.865 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:08:58.077 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:08:58.266 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:08:58.487 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:08:58.562 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:08:58.762 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:08:58.824 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:08:59.067 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:08:59.269 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T23:08:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:08:59.555 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:08:59.754 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:09:00.159 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:09:00.404 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:09:00.779 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:09:00.954 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:09:01.093 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-05-08T23:09:01.107 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:09:01.227 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:09:01.366 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:09:01.461 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:09:01.503 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:09:01.723 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:09:01.793 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:09:01.878 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:09:02.066 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:09:02.087 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:09:02.141 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:09:02.152 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:09:02.291 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:09:02.361 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:09:02.402 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:09:02.553 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:09:02.694 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:09:02.781 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T23:09:02.899 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:09:02.911 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-05-08T23:09:02.966 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:09:03.058 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T23:09:03.059 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T23:09:03.252 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:09:03.273 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:09:03.274 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:09:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T23:09:03.515 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:09:03.684 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:09:03.686 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:09:03.692 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T23:09:03.788 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:09:03.862 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:09:03.915 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:09:03.923 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T23:09:04.003 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:09:04.075 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:09:04.204 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:09:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T23:09:04.275 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T23:09:04.306 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:09:04.482 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:09:04.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T23:09:04.545 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T23:09:04.644 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:09:04.662 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:09:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T23:09:04.782 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T23:09:04.844 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:09:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T23:09:04.940 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:09:04.961 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:09:05.109 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:09:05.195 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:09:05.231 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:09:05.403 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:09:05.524 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:09:05.598 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:09:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T23:09:05.797 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:09:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:09:06.077 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:09:06.205 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:09:06.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:09:06.258 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:09:06.378 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:09:06.420 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:09:06.549 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:09:06.636 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:09:06.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:09:06.825 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:09:06.869 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:09:07.004 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:09:07.015 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:09:07.115 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:09:07.144 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:09:07.228 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:09:07.282 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:09:07.356 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:09:07.395 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:09:07.490 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T23:09:07.518 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T23:09:07.731 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T23:09:07.773 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T23:09:07.919 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:09:07.985 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T23:09:08.028 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T23:09:08.151 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T23:09:08.206 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:09:08.209 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T23:09:08.329 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T23:09:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T23:09:08.490 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:09:08.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T23:09:08.713 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:09:08.750 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T23:09:08.886 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:09:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:09:09.003 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:09:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:09:09.100 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T23:09:09.203 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:09:09.273 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:09:09.289 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T23:09:09.314 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:09:09.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:09:09.549 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:09:09.579 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T23:09:09.728 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:09:09.865 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T23:09:10.040 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:09:10.418 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T23:09:10.578 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:09:10.721 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T23:09:11.013 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T23:09:11.405 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:09:11.405 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T23:09:11.405 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T23:09:11.411 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:09:11.466 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:09:11.663 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:09:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T23:09:11.809 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:09:12.068 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:09:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T23:09:12.312 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:09:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T23:09:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T23:09:12.684 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T23:09:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T23:09:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T23:09:13.040 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T23:09:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T23:09:13.305 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T23:09:13.531 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T23:09:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:09:13.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T23:09:13.821 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:09:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T23:09:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:09:14.054 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T23:09:14.058 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T23:09:14.251 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:09:14.433 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T23:09:14.435 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:09:14.542 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:09:14.802 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:09:14.869 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:09:15.079 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T23:09:15.114 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:09:15.354 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:09:15.370 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T23:09:15.426 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T23:09:15.585 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:09:15.714 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T23:09:15.744 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T23:09:15.915 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:09:15.986 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T23:09:16.023 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:09:16.266 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:09:16.419 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:09:16.446 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:09:16.466 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:09:16.652 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:09:16.693 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:09:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:09:16.879 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:09:17.046 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T23:09:17.072 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:09:17.147 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:09:17.302 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T23:09:17.389 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:09:17.441 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:09:17.535 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:09:17.609 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:09:17.670 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:09:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T23:09:17.866 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:09:17.885 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T23:09:17.967 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T23:09:18.072 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:09:18.169 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T23:09:18.192 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T23:09:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/162 2024-05-08T23:09:18.422 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T23:09:18.505 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:09:18.624 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/162 2024-05-08T23:09:18.659 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T23:09:18.790 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/162 2024-05-08T23:09:18.822 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:09:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T23:09:18.851 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:09:18.978 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T23:09:19.013 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/162 2024-05-08T23:09:19.030 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:09:19.095 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T23:09:19.177 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:09:19.247 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/162 2024-05-08T23:09:19.258 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T23:09:19.390 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T23:09:19.460 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:09:19.535 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T23:09:19.588 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/162 2024-05-08T23:09:19.630 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T23:09:19.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T23:09:19.805 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:09:19.872 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T23:09:19.873 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/162 2024-05-08T23:09:19.939 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T23:09:20.070 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T23:09:20.114 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/162 2024-05-08T23:09:20.127 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:09:20.236 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T23:09:20.298 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/162 2024-05-08T23:09:20.448 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T23:09:20.555 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/162 2024-05-08T23:09:20.588 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T23:09:20.673 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-08T23:09:20.859 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-08T23:09:20.861 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:09:20.882 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T23:09:20.935 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/162 2024-05-08T23:09:21.037 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-08T23:09:21.057 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T23:09:21.244 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T23:09:21.252 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-08T23:09:21.353 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 67/162 2024-05-08T23:09:21.433 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-08T23:09:21.433 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T23:09:21.607 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T23:09:21.616 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-08T23:09:21.624 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 68/162 2024-05-08T23:09:21.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T23:09:21.867 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 69/162 2024-05-08T23:09:21.898 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-08T23:09:22.027 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 70/162 2024-05-08T23:09:22.102 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T23:09:22.141 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-08T23:09:22.368 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-08T23:09:22.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T23:09:22.404 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/162 2024-05-08T23:09:22.541 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-08T23:09:22.599 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T23:09:22.663 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/162 2024-05-08T23:09:22.754 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T23:09:22.756 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-08T23:09:22.943 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/162 2024-05-08T23:09:22.987 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T23:09:23.095 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-08T23:09:23.211 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/162 2024-05-08T23:09:23.347 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T23:09:23.374 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-08T23:09:23.382 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/162 2024-05-08T23:09:23.642 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T23:09:23.661 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-08T23:09:23.711 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/162 2024-05-08T23:09:23.912 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-08T23:09:23.938 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T23:09:23.946 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T23:09:24.092 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:09:24.105 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-08T23:09:24.161 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T23:09:24.318 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:09:24.358 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-08T23:09:24.359 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T23:09:24.602 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T23:09:24.623 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:09:24.641 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-08T23:09:24.836 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T23:09:24.879 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-08T23:09:24.891 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:09:25.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T23:09:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:09:25.123 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-08T23:09:25.213 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T23:09:25.220 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:09:25.317 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-08T23:09:25.407 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T23:09:25.468 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:09:25.563 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-08T23:09:25.704 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:09:25.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T23:09:25.820 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:09:25.947 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:09:25.948 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-16.8-9.el8.noarch 70/162 2024-05-08T23:09:26.022 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:09:26.143 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T23:09:26.170 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:09:26.229 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-05-08T23:09:26.325 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:09:26.397 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:09:26.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-05-08T23:09:26.577 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:09:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:09:26.754 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-05-08T23:09:26.758 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:09:26.837 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:09:26.918 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T23:09:26.924 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:09:26.981 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-05-08T23:09:27.039 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:09:27.134 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-05-08T23:09:27.164 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:09:27.318 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-05-08T23:09:27.328 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:09:27.391 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:09:27.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-05-08T23:09:27.591 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:09:27.677 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:09:27.677 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T23:09:27.799 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:09:27.865 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:09:27.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T23:09:28.026 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:09:28.066 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:09:28.214 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T23:09:28.316 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:09:28.417 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T23:09:28.557 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:09:28.597 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T23:09:28.759 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:09:28.763 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T23:09:28.965 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T23:09:29.048 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:09:29.158 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T23:09:29.161 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:09:29.286 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:09:29.417 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T23:09:29.486 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:09:29.504 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:09:29.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T23:09:29.740 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:09:29.761 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:09:29.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T23:09:29.978 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T23:09:30.070 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:09:30.173 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T23:09:30.308 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:09:30.341 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T23:09:30.574 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:09:30.605 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T23:09:30.844 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:09:30.851 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T23:09:31.035 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T23:09:31.149 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:09:31.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T23:09:31.360 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:09:31.366 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:09:31.662 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:09:31.928 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:09:31.999 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:09:32.091 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:09:32.161 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:09:32.283 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:09:32.518 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T23:09:32.794 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T23:09:32.916 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:09:33.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T23:09:33.070 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:09:33.145 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:09:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T23:09:33.303 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:09:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T23:09:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T23:09:34.588 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T23:09:34.742 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:09:34.820 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T23:09:35.001 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T23:09:36.097 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:36.390 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:36.983 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:37.015 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:37.286 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:37.876 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:38.198 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:09:38.281 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:09:38.410 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:09:38.465 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:09:39.080 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:09:39.239 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:09:39.520 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:39.758 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:40.418 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T23:09:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:09:40.810 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T23:09:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:09:57.694 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /run 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /dev 2024-05-08T23:09:57.695 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:09:58.075 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:09:58.572 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:09:58.572 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /dev 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /run 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /dev 2024-05-08T23:09:58.573 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:09:58.954 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:09:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:09:59.936 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:10:00.154 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:10:00.316 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:10:00.598 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:10:00.794 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:10:00.822 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:10:00.974 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:10:00.991 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:10:01.043 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:10:01.194 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:10:01.671 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:10:01.861 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:10:01.879 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:10:02.747 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:10:02.913 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:10:03.038 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:10:03.116 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T23:10:03.116 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-05-08T23:10:03.116 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-05-08T23:10:03.116 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2024-05-08T23:10:03.116 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2024-05-08T23:10:03.117 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-05-08T23:10:03.117 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-05-08T23:10:03.117 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-05-08T23:10:03.117 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-05-08T23:10:03.117 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2024-05-08T23:10:03.117 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:10:03.291 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:10:03.481 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:10:03.544 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:10:03.736 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:10:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:10:04.026 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:10:04.246 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:10:04.515 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:10:04.574 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:10:04.810 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:10:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T23:10:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:10:05.543 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:10:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T23:10:05.774 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:10:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:10:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:10:05.939 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:10:06.032 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:10:06.372 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:10:06.392 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T23:10:06.552 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:10:06.554 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:10:06.575 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T23:10:06.589 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:10:06.744 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:10:06.756 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:10:06.767 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:10:06.935 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:10:07.029 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:10:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:10:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:10:07.677 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:10:07.868 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:10:07.880 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:10:08.015 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:10:08.232 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:10:08.266 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:10:08.456 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:10:08.499 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:10:08.664 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:10:08.687 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:10:08.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T23:10:08.937 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T23:10:09.011 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:10:09.141 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:10:09.161 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T23:10:09.198 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:10:09.329 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:10:09.363 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:10:09.397 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:10:09.585 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:10:09.628 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:10:09.818 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:10:10.166 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:10:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:10:10.547 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T23:10:10.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:10:11.876 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T23:10:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:10:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T23:10:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T23:10:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:10:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T23:10:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:10:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T23:10:13.646 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:10:14.350 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T23:10:14.548 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:10:14.589 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T23:10:14.620 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:10:14.770 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:10:14.821 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T23:10:14.826 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:10:14.926 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T23:10:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:10:15.196 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T23:10:15.375 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T23:10:15.562 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:10:15.616 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T23:10:15.833 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:10:15.915 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T23:10:16.156 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:10:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T23:10:16.168 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:10:16.382 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:10:16.466 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T23:10:16.567 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:10:16.739 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:10:16.862 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T23:10:17.236 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T23:10:17.503 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:10:17.517 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T23:10:17.761 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:10:17.988 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:10:18.143 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:10:19.007 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:19.338 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:19.375 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:19.762 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:10:20.111 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:10:20.316 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:10:20.499 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:20.503 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T23:10:20.529 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:10:20.720 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:10:20.892 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:20.930 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:20.978 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T23:10:20.991 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:10:21.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T23:10:21.341 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:10:21.349 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:10:21.541 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:10:21.545 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:10:21.626 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:10:21.782 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:10:21.854 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:10:21.868 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:10:22.012 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:10:22.021 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:10:22.037 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:10:22.173 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T23:10:22.242 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:10:22.353 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T23:10:22.412 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:10:22.500 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:10:22.606 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:10:22.616 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T23:10:22.774 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:10:22.775 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:10:22.829 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:10:22.869 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T23:10:23.060 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:10:23.090 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:10:23.090 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:10:23.153 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:10:23.265 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T23:10:23.316 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:10:23.316 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:10:23.407 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:10:23.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T23:10:23.487 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T23:10:23.560 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:10:23.636 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T23:10:23.733 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:10:23.747 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T23:10:23.814 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:10:23.964 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:10:23.966 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:10:23.976 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T23:10:24.160 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T23:10:24.214 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:10:24.214 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:24.373 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:10:24.473 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:24.473 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:10:24.607 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:10:24.689 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:10:24.690 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:24.794 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:10:24.795 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:10:24.914 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:24.914 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:10:25.134 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:10:25.134 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:10:25.156 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:10:25.156 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:10:25.345 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:10:25.372 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:25.548 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:10:25.548 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:10:25.639 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:10:25.884 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:10:26.110 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:10:26.187 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:26.376 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:10:26.376 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:26.486 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:26.507 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:10:26.507 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:10:26.507 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:10:26.507 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:10:26.507 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:10:26.508 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:10:26.509 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:10:26.510 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:10:26.511 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:10:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-08T23:10:26.513 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-08T23:10:26.514 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T23:10:26.515 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T23:10:26.516 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:10:26.517 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:10:26.518 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:26.519 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:10:26.526 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T23:10:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:10:26.924 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T23:10:26.968 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:10:26.968 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:27.264 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T23:10:27.284 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:27.284 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:10:27.482 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T23:10:27.551 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:10:27.551 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:27.645 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T23:10:27.809 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:27.837 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T23:10:27.960 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:10:27.960 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:10:28.115 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T23:10:28.395 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T23:10:28.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:10:28.580 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T23:10:28.801 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T23:10:28.934 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:10:28.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:10:28.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:10:28.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:10:28.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:10:28.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:10:28.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:10:28.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:10:28.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T23:10:28.939 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/162 2024-05-08T23:10:28.940 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 93/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 94/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 95/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 96/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 97/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 98/162 2024-05-08T23:10:28.941 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 99/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T23:10:28.942 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T23:10:28.943 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:10:28.944 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:10:28.945 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:28.946 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:10:29.034 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:10:29.122 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T23:10:29.378 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:10:29.589 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T23:10:29.589 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T23:10:29.762 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T23:10:29.922 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T23:10:29.922 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:10:30.163 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T23:10:30.163 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:10:30.389 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T23:10:30.389 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:10:30.609 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T23:10:30.820 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T23:10:30.974 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:10:31.187 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T23:10:31.187 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:10:31.662 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T23:10:31.663 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:31.896 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:31.896 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:10:32.137 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T23:10:32.137 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:32.529 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T23:10:32.529 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:10:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T23:10:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T23:10:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T23:10:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T23:10:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T23:10:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T23:10:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T23:10:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T23:10:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T23:10:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T23:10:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T23:10:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T23:10:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T23:10:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-05-08T23:10:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-05-08T23:10:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-05-08T23:10:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-05-08T23:10:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 103/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T23:10:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T23:10:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T23:10:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T23:10:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T23:10:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T23:10:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T23:10:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T23:10:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T23:10:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T23:10:43.146 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:10:43.147 INFO:teuthology.orchestra.run.smithi155.stdout:Installed products updated. 2024-05-08T23:10:43.750 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:10:43.750 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:43.751 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.752 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.753 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.754 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:10:43.755 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:10:43.756 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:10:43.757 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:10:43.758 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:10:43.759 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:10:43.760 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:10:43.761 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:10:43.762 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:10:43.762 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-05-08T23:10:43.762 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-05-08T23:10:43.975 DEBUG:teuthology.parallel:result is None 2024-05-08T23:10:44.365 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:10:44.365 INFO:teuthology.orchestra.run.smithi189.stdout:Installed products updated. 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.976 INFO:teuthology.orchestra.run.smithi189.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:10:44.977 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.978 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:10:44.979 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:10:44.980 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:10:44.981 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.982 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:10:44.983 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:10:44.984 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:10:44.985 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:10:44.986 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:10:44.987 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:10:44.987 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:10:44.987 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:10:44.987 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-05-08T23:10:45.187 DEBUG:teuthology.parallel:result is None 2024-05-08T23:10:50.100 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T23:10:50.100 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2024-05-08T23:10:50.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:10:50.760 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-05-08T23:10:50.760 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:50.761 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.762 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.763 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-7.el8.x86_64 2024-05-08T23:10:50.764 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.765 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T23:10:50.766 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T23:10:50.767 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T23:10:50.768 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T23:10:50.769 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T23:10:50.770 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T23:10:50.771 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T23:10:50.772 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T23:10:50.773 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T23:10:50.773 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T23:10:50.773 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T23:10:50.773 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T23:10:50.773 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T23:10:50.773 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:10:50.774 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-05-08T23:10:51.007 DEBUG:teuthology.parallel:result is None 2024-05-08T23:10:51.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:10:51.155 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:10:51.183 INFO:teuthology.orchestra.run.smithi016.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:10:51.183 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:10:51.184 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:10:51.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:10:51.453 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:10:51.482 INFO:teuthology.orchestra.run.smithi155.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:10:51.482 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:10:51.482 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:10:51.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:10:51.795 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T23:10:51.823 INFO:teuthology.orchestra.run.smithi189.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T23:10:51.823 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T23:10:51.824 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T23:10:51.826 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T23:10:51.826 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:10:51.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:10:51.880 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:10:51.880 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:10:51.931 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:10:51.931 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T23:10:51.980 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T23:10:51.980 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:10:51.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:10:52.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:10:52.090 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:10:52.091 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:10:52.123 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:10:52.195 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:10:52.195 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T23:10:52.227 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T23:10:52.299 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T23:10:52.299 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:10:52.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:10:52.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:10:52.408 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:10:52.408 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:10:52.440 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:10:52.510 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:10:52.510 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T23:10:52.542 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T23:10:52.611 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T23:10:52.612 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:10:52.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:10:52.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:10:52.720 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:10:52.720 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:10:52.753 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:10:52.826 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:10:52.826 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T23:10:52.856 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T23:10:52.925 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T23:10:53.026 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2}} 2024-05-08T23:10:53.026 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:10:53.027 INFO:tasks.cephadm:Cluster fsid is 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:10:53.027 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T23:10:53.027 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.155', 'mon.c': '172.21.15.189'} 2024-05-08T23:10:53.027 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-05-08T23:10:53.027 INFO:tasks.cephadm:First mgr is x 2024-05-08T23:10:53.027 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T23:10:53.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-05-08T23:10:53.063 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-05-08T23:10:53.095 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-05-08T23:10:53.125 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T23:10:53.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:10:53.263 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T23:10:53.400 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:10:53.401 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T23:10:53.401 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:53.898 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 23:10 /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:53.899 DEBUG:teuthology.orchestra.run.smithi155:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.244 INFO:teuthology.orchestra.run.smithi155.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 23:10 /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.245 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.587 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 23:10 /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.588 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.611 DEBUG:teuthology.orchestra.run.smithi155:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.633 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T23:10:54.671 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T23:10:54.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T23:10:54.673 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T23:10:54.676 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T23:10:55.543 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:10:55.694 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:10:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:11:47.736 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-08T23:11:47.737 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:11:47.737 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T23:11:47.737 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-05-08T23:11:47.737 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T23:11:47.737 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-05-08T23:11:47.737 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:11:48.428 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-05-08T23:11:48.470 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-05-08T23:11:48.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-05-08T23:11:48.529 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-05-08T23:11:48.563 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-05-08T23:11:48.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-05-08T23:11:48.626 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-05-08T23:11:48.655 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-05-08T23:11:48.682 INFO:tasks.cephadm:Writing seed config... 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T23:11:48.683 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T23:11:48.684 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T23:11:48.685 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T23:11:48.686 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T23:11:48.687 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:11:48.687 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T23:11:48.708 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 378de17a-0d90-11ef-bc97-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T23:11:48.708 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T23:11:48.751 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T23:11:48.785 INFO:journalctl@ceph.mon.a.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:11:48.786 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T23:11:48.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:11:48.818 INFO:journalctl@ceph.mgr.x.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:11:48.975 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-05-08T23:11:48.975 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', '378de17a-0d90-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-05-08T23:11:49.018 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.6.1 2024-05-08T23:11:49.018 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T23:11:49.019 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-05-08T23:11:49.056 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.6.1 2024-05-08T23:11:49.056 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-05-08T23:11:49.057 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-05-08T23:11:49.065 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T23:11:49.065 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T23:11:49.071 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T23:11:49.072 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-08T23:11:49.078 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-05-08T23:11:49.085 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-05-08T23:11:49.085 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-05-08T23:11:49.085 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-05-08T23:11:49.119 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 4.6.1 2024-05-08T23:11:49.119 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T23:11:49.119 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-05-08T23:11:49.120 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-05-08T23:11:49.127 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T23:11:49.127 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T23:11:49.133 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T23:11:49.133 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-08T23:11:49.140 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-05-08T23:11:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-05-08T23:11:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-05-08T23:11:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-05-08T23:11:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 139719176268264 on /run/cephadm/378de17a-0d90-11ef-bc97-c7b262605968.lock 2024-05-08T23:11:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 139719176268264 acquired on /run/cephadm/378de17a-0d90-11ef-bc97-c7b262605968.lock 2024-05-08T23:11:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-05-08T23:11:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-05-08T23:11:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-05-08T23:11:49.152 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-05-08T23:11:49.153 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-05-08T23:11:49.156 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T23:11:49.156 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-05-08T23:11:49.156 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1784sec hoplimit 64 pref medium 2024-05-08T23:11:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T23:11:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T23:11:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T23:11:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T23:11:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-05-08T23:11:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T23:11:49.160 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-05-08T23:11:49.160 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-05-08T23:11:49.160 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T23:11:49.161 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T23:11:49.162 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T23:11:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T23:11:50.252 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:11:50.252 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T23:11:50.252 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T23:11:50.845 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-05-08T23:11:50.845 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-05-08T23:11:51.435 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQC3BjxmTbwTBhAA6tjSFmIiDmsPt7kxfymXnw== 2024-05-08T23:11:52.017 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQC3Bjxmnl/CKBAAq1/WmDHzL5TuaCXsDN9HdA== 2024-05-08T23:11:52.613 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQC4BjxmvhJrEBAApj1u3AxKS5PMVWwvwchO5A== 2024-05-08T23:11:52.614 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-05-08T23:11:53.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T23:11:53.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T23:11:53.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:53.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T23:11:53.247 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T23:11:53.248 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2024-05-08T23:11:53.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:53.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T23:11:53.248 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:11:53.248 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-05-08T23:11:53.847 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-05-08T23:11:53.995 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T23:11:54.094 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T23:11:54.215 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-378de17a-0d90-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-378de17a-0d90-11ef-bc97-c7b262605968.target. 2024-05-08T23:11:54.216 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-378de17a-0d90-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-378de17a-0d90-11ef-bc97-c7b262605968.target. 2024-05-08T23:11:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a 2024-05-08T23:11:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T23:11:54.495 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-378de17a-0d90-11ef-bc97-c7b262605968.target.wants/ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-378de17a-0d90-11ef-bc97-c7b262605968@.service. 2024-05-08T23:11:54.901 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:11:54.902 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-08T23:11:54.911 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:11:54.911 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-08T23:11:54.911 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-08T23:11:54.911 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T23:11:54.911 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-05-08T23:11:54.911 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-05-08T23:11:55.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:54 smithi016 ceph-mon[42054]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T23:11:55.724 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.490591s) 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T23:11:55.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.336+0000 7f145da12700 1 Processor -- start 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.336+0000 7f145da12700 1 -- start start 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.336+0000 7f145da12700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f1458106900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.336+0000 7f145da12700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1458106ed0 con 0x7f14581064e0 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1456ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f1458106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1456ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f1458106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56948/0 (socket says 172.21.15.16:56948) 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1456ffd700 1 -- 172.21.15.16:0/900881466 learned_addr learned my addr 172.21.15.16:0/900881466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1456ffd700 1 -- 172.21.15.16:0/900881466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14581076e0 con 0x7f14581064e0 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1456ffd700 1 --2- 172.21.15.16:0/900881466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f1458106900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f14480098a0 tx=0x7f1448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d557e029a5b8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1455ffb700 1 -- 172.21.15.16:0/900881466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1448004160 con 0x7f14581064e0 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1455ffb700 1 -- 172.21.15.16:0/900881466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14480042c0 con 0x7f14581064e0 2024-05-08T23:11:55.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.337+0000 7f1455ffb700 1 -- 172.21.15.16:0/900881466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14480045a0 con 0x7f14581064e0 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 -- 172.21.15.16:0/900881466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 msgr2=0x7f1458106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 --2- 172.21.15.16:0/900881466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f1458106900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f14480098a0 tx=0x7f1448009bb0 comp rx=0 tx=0).stop 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 -- 172.21.15.16:0/900881466 shutdown_connections 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 --2- 172.21.15.16:0/900881466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f1458106900 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 -- 172.21.15.16:0/900881466 >> 172.21.15.16:0/900881466 conn(0x7f1458101a80 msgr2=0x7f1458103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 -- 172.21.15.16:0/900881466 shutdown_connections 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.338+0000 7f145da12700 1 -- 172.21.15.16:0/900881466 wait complete. 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f145da12700 1 Processor -- start 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f145da12700 1 -- start start 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f145da12700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f145807bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f145da12700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145807a2c0 con 0x7f14581064e0 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f1456ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f145807bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:55.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f1456ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f145807bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56952/0 (socket says 172.21.15.16:56952) 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.339+0000 7f1456ffd700 1 -- 172.21.15.16:0/1333283597 learned_addr learned my addr 172.21.15.16:0/1333283597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f1456ffd700 1 -- 172.21.15.16:0/1333283597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1448009550 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f1456ffd700 1 --2- 172.21.15.16:0/1333283597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f145807bd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1448004000 tx=0x7f1448004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1448011630 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448011790 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1448011a30 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145807a4c0 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145807a960 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1448021070 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.340+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f144801b440 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.341+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1458193660 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.344+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f144801a460 con 0x7f14581064e0 2024-05-08T23:11:55.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.420+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1458107a50 con 0x7f14581064e0 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.420+0000 7f145ca10700 1 -- 172.21.15.16:0/1333283597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f144801a600 con 0x7f14581064e0 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 msgr2=0x7f145807bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 --2- 172.21.15.16:0/1333283597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f145807bd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1448004000 tx=0x7f1448004750 comp rx=0 tx=0).stop 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 shutdown_connections 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 --2- 172.21.15.16:0/1333283597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14581064e0 0x7f145807bd20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 >> 172.21.15.16:0/1333283597 conn(0x7f1458101a80 msgr2=0x7f14581907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 shutdown_connections 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:55.421+0000 7f145da12700 1 -- 172.21.15.16:0/1333283597 wait complete. 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-05-08T23:11:55.729 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T23:11:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:55 smithi016 ceph-mon[42054]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T23:11:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:55 smithi016 ceph-mon[42054]: monmap e1: 1 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:11:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:55 smithi016 ceph-mon[42054]: fsmap 2024-05-08T23:11:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:55 smithi016 ceph-mon[42054]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T23:11:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:55 smithi016 ceph-mon[42054]: mgrmap e1: no daemons active 2024-05-08T23:11:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:55 smithi016 ceph-mon[42054]: from='client.? 172.21.15.16:0/1333283597' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T23:11:56.534 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T23:11:56.535 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.094+0000 7f5173bfc700 1 Processor -- start 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.094+0000 7f5173bfc700 1 -- start start 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.094+0000 7f5173bfc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.094+0000 7f5173bfc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c106ee0 con 0x7f516c1064f0 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5171998700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5171998700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56968/0 (socket says 172.21.15.16:56968) 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5171998700 1 -- 172.21.15.16:0/1632321646 learned_addr learned my addr 172.21.15.16:0/1632321646 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5171998700 1 -- 172.21.15.16:0/1632321646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c1076f0 con 0x7f516c1064f0 2024-05-08T23:11:56.536 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5171998700 1 --2- 172.21.15.16:0/1632321646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c106910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f515c0098a0 tx=0x7f515c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28591ff0c51bf9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5170996700 1 -- 172.21.15.16:0/1632321646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f515c004160 con 0x7f516c1064f0 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5170996700 1 -- 172.21.15.16:0/1632321646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c0042c0 con 0x7f516c1064f0 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.095+0000 7f5170996700 1 -- 172.21.15.16:0/1632321646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f515c0045a0 con 0x7f516c1064f0 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 -- 172.21.15.16:0/1632321646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 msgr2=0x7f516c106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 --2- 172.21.15.16:0/1632321646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c106910 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f515c0098a0 tx=0x7f515c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 -- 172.21.15.16:0/1632321646 shutdown_connections 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 --2- 172.21.15.16:0/1632321646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c106910 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 -- 172.21.15.16:0/1632321646 >> 172.21.15.16:0/1632321646 conn(0x7f516c101ad0 msgr2=0x7f516c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 -- 172.21.15.16:0/1632321646 shutdown_connections 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.096+0000 7f5173bfc700 1 -- 172.21.15.16:0/1632321646 wait complete. 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.097+0000 7f5173bfc700 1 Processor -- start 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.097+0000 7f5173bfc700 1 -- start start 2024-05-08T23:11:56.537 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.097+0000 7f5173bfc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.097+0000 7f5173bfc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c199f70 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.097+0000 7f5171998700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.097+0000 7f5171998700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56978/0 (socket says 172.21.15.16:56978) 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5171998700 1 -- 172.21.15.16:0/299456298 learned_addr learned my addr 172.21.15.16:0/299456298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5171998700 1 -- 172.21.15.16:0/299456298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c009550 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5171998700 1 --2- 172.21.15.16:0/299456298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c199a30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f515c004000 tx=0x7f515c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f515c011630 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c011790 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f515c011a30 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c19a170 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.098+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c19acd0 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.099+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f515c022070 con 0x7f516c1064f0 2024-05-08T23:11:56.538 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.099+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f515c01c450 con 0x7f516c1064f0 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.099+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c0625d0 con 0x7f516c1064f0 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.102+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f515c01b450 con 0x7f516c1064f0 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.175+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f516c066f40 con 0x7f516c1064f0 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.179+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f515c01b5f0 con 0x7f516c1064f0 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.179+0000 7f5162ffd700 1 -- 172.21.15.16:0/299456298 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f515c024560 con 0x7f516c1064f0 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 msgr2=0x7f516c199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 --2- 172.21.15.16:0/299456298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c199a30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f515c004000 tx=0x7f515c004750 comp rx=0 tx=0).stop 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 shutdown_connections 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 --2- 172.21.15.16:0/299456298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f516c1064f0 0x7f516c199a30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 >> 172.21.15.16:0/299456298 conn(0x7f516c101ad0 msgr2=0x7f516c190800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 shutdown_connections 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.180+0000 7f5173bfc700 1 -- 172.21.15.16:0/299456298 wait complete. 2024-05-08T23:11:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-05-08T23:11:57.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:56 smithi016 ceph-mon[42054]: from='client.? 172.21.15.16:0/299456298' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T23:11:57.248 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:56 smithi016 ceph-mon[42054]: from='client.? 172.21.15.16:0/299456298' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff9764700 1 Processor -- start 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff9764700 1 -- start start 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff9764700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff4107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff9764700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4108500 con 0x7f4ff4107b10 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff4107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff4107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37886/0 (socket says 172.21.15.16:37886) 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.910+0000 7f4ff2ffd700 1 -- 172.21.15.16:0/4183147105 learned_addr learned my addr 172.21.15.16:0/4183147105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.911+0000 7f4ff2ffd700 1 -- 172.21.15.16:0/4183147105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff4108640 con 0x7f4ff4107b10 2024-05-08T23:11:57.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.911+0000 7f4ff2ffd700 1 --2- 172.21.15.16:0/4183147105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff4107f30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40098a0 tx=0x7f4fe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22a5949abf12fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.911+0000 7f4ff1ffb700 1 -- 172.21.15.16:0/4183147105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4004160 con 0x7f4ff4107b10 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.911+0000 7f4ff1ffb700 1 -- 172.21.15.16:0/4183147105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4fe40042c0 con 0x7f4ff4107b10 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.911+0000 7f4ff1ffb700 1 -- 172.21.15.16:0/4183147105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4011940 con 0x7f4ff4107b10 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 -- 172.21.15.16:0/4183147105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 msgr2=0x7f4ff4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 --2- 172.21.15.16:0/4183147105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff4107f30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40098a0 tx=0x7f4fe4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 -- 172.21.15.16:0/4183147105 shutdown_connections 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 --2- 172.21.15.16:0/4183147105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff4107f30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 -- 172.21.15.16:0/4183147105 >> 172.21.15.16:0/4183147105 conn(0x7f4ff41000f0 msgr2=0x7f4ff4102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 -- 172.21.15.16:0/4183147105 shutdown_connections 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 -- 172.21.15.16:0/4183147105 wait complete. 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.912+0000 7f4ff9764700 1 Processor -- start 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff9764700 1 -- start start 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff9764700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff41072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff9764700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff4105860 con 0x7f4ff4107b10 2024-05-08T23:11:57.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff41072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff41072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37902/0 (socket says 172.21.15.16:37902) 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff2ffd700 1 -- 172.21.15.16:0/3712281245 learned_addr learned my addr 172.21.15.16:0/3712281245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff2ffd700 1 -- 172.21.15.16:0/3712281245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009550 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.913+0000 7f4ff2ffd700 1 --2- 172.21.15.16:0/3712281245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff41072c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4003f20 tx=0x7f4fe4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe40036a0 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4fe4003c70 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff4105a60 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe401be80 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff4105f00 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4fe4023790 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.914+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4fe402cd50 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.915+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff4103e00 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.917+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4fe401b440 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.991+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4ff4105150 con 0x7f4ff4107b10 2024-05-08T23:11:57.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.991+0000 7f4fdbfff700 1 -- 172.21.15.16:0/3712281245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f4fe402c3f0 con 0x7f4ff4107b10 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.992+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 msgr2=0x7f4ff41072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.992+0000 7f4ff9764700 1 --2- 172.21.15.16:0/3712281245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff41072c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4003f20 tx=0x7f4fe4011f00 comp rx=0 tx=0).stop 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.992+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 shutdown_connections 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.992+0000 7f4ff9764700 1 --2- 172.21.15.16:0/3712281245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff4107b10 0x7f4ff41072c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.992+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 >> 172.21.15.16:0/3712281245 conn(0x7f4ff41000f0 msgr2=0x7f4ff4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.992+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 shutdown_connections 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:56.993+0000 7f4ff9764700 1 -- 172.21.15.16:0/3712281245 wait complete. 2024-05-08T23:11:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-05-08T23:11:57.565 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:57 smithi016 systemd[1]: Stopping Ceph mon.a for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:11:57.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:57 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a[42031]: 2024-05-08T23:11:57.564+0000 7fd07ff93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:11:57.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:57 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a[42031]: 2024-05-08T23:11:57.565+0000 7fd07ff93700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T23:11:57.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:57 smithi016 bash[42381]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a 2024-05-08T23:11:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 systemd[1]: ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T23:11:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 systemd[1]: Stopped Ceph mon.a for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:11:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 systemd[1]: Starting Ceph mon.a for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:11:58.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 bash[42503]: 180b5f12c5cd0d6e51469844f3c7458c4672fe0574362df093a0ecc0b9745ff4 2024-05-08T23:11:58.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T23:11:58.455 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T23:11:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T23:11:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: pidfile_write: ignore empty --pid-file 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 systemd[1]: Started Ceph mon.a for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: load: jerasure load: lrc 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: RocksDB version: 7.9.2 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Git sha 0 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: DB SUMMARY 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: DB Session ID: JKYPCUC5UVRHIDK14IOB 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: CURRENT file: CURRENT 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 96322 ; 2024-05-08T23:11:58.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.error_if_exists: 0 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.create_if_missing: 0 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.paranoid_checks: 1 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.env: 0x557b54202820 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.info_log: 0x557b56049520 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.statistics: (nil) 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.use_fsync: 0 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_log_file_size: 0 2024-05-08T23:11:58.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.allow_fallocate: 1 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.use_direct_reads: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.db_log_dir: 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.wal_dir: 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T23:11:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.write_buffer_manager: 0x557b552de320 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T23:11:58.868 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.unordered_write: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.row_cache: None 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.wal_filter: None 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.two_write_queues: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.wal_compression: 0 2024-05-08T23:11:58.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.atomic_flush: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.log_readahead_size: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_background_jobs: 2 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_background_compactions: -1 2024-05-08T23:11:58.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_subcompactions: 1 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_open_files: -1 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T23:11:58.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_background_flushes: -1 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Compression algorithms supported: 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kZSTD supported: 0 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kXpressCompression supported: 0 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kZlibCompression supported: 1 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kSnappyCompression supported: 1 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kLZ4Compression supported: 1 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: kBZip2Compression supported: 0 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T23:11:58.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.merge_operator: 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_filter: None 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557b56049620) 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-05-08T23:11:58.873 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x557b553611f0 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-05-08T23:11:58.874 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-05-08T23:11:58.875 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression: NoCompression 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.num_levels: 7 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T23:11:58.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T23:11:58.877 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T23:11:58.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T23:11:58.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T23:11:58.880 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T23:11:58.880 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T23:11:58.880 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.table_properties_collectors: 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.inplace_update_support: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.bloom_locality: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.max_successive_merges: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T23:11:58.881 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.ttl: 2592000 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.enable_blob_files: false 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.min_blob_size: 0 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T23:11:58.882 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b613a1c0-abf7-43f9-a99e-b991e8c642d3 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209918461248, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209918463241, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91821, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 303, "table_properties": {"data_size": 89843, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13318, "raw_average_key_size": 50, "raw_value_size": 82582, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 266, "num_filter_entries": 266, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715209918, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b613a1c0-abf7-43f9-a99e-b991e8c642d3", "db_session_id": "JKYPCUC5UVRHIDK14IOB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209918463380, "job": 1, "event": "recovery_finished"} 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T23:11:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557b553fe000 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: DB pointer 0x557b553ec000 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:11:58.884 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 91.50 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 91.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 68.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 68.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-08T23:11:58.885 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 15.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 15.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x557b553611f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-08T23:11:58.886 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???) e1 preinit fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).mds e1 new map 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).mds e1 print_map 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:11:58.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:11:58.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:58 smithi016 ceph-mon[42541]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T23:11:59.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.815+0000 7f4eef9bf700 1 Processor -- start 2024-05-08T23:11:59.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.815+0000 7f4eef9bf700 1 -- start start 2024-05-08T23:11:59.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.816+0000 7f4eef9bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee8109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.284 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.816+0000 7f4eef9bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee8079e90 con 0x7f4ee8106f70 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.816+0000 7f4eed75b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee8109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.816+0000 7f4eed75b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee8109360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37918/0 (socket says 172.21.15.16:37918) 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.816+0000 7f4eed75b700 1 -- 172.21.15.16:0/2107091180 learned_addr learned my addr 172.21.15.16:0/2107091180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.816+0000 7f4eed75b700 1 -- 172.21.15.16:0/2107091180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee81098a0 con 0x7f4ee8106f70 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4eed75b700 1 --2- 172.21.15.16:0/2107091180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee8109360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40098a0 tx=0x7f4ee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427d8896752f8828 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4edffff700 1 -- 172.21.15.16:0/2107091180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ee4004160 con 0x7f4ee8106f70 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4edffff700 1 -- 172.21.15.16:0/2107091180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4ee40042c0 con 0x7f4ee8106f70 2024-05-08T23:11:59.285 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4edffff700 1 -- 172.21.15.16:0/2107091180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ee4011940 con 0x7f4ee8106f70 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4eef9bf700 1 -- 172.21.15.16:0/2107091180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 msgr2=0x7f4ee8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4eef9bf700 1 --2- 172.21.15.16:0/2107091180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee8109360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40098a0 tx=0x7f4ee4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4eef9bf700 1 -- 172.21.15.16:0/2107091180 shutdown_connections 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4eef9bf700 1 --2- 172.21.15.16:0/2107091180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee8109360 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.817+0000 7f4eef9bf700 1 -- 172.21.15.16:0/2107091180 >> 172.21.15.16:0/2107091180 conn(0x7f4ee8100960 msgr2=0x7f4ee8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.818+0000 7f4eef9bf700 1 -- 172.21.15.16:0/2107091180 shutdown_connections 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.818+0000 7f4eef9bf700 1 -- 172.21.15.16:0/2107091180 wait complete. 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.818+0000 7f4eef9bf700 1 Processor -- start 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.818+0000 7f4eef9bf700 1 -- start start 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.818+0000 7f4eef9bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.818+0000 7f4eef9bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee819a6e0 con 0x7f4ee8106f70 2024-05-08T23:11:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eed75b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eed75b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37928/0 (socket says 172.21.15.16:37928) 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eed75b700 1 -- 172.21.15.16:0/400203896 learned_addr learned my addr 172.21.15.16:0/400203896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eed75b700 1 -- 172.21.15.16:0/400203896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4009550 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eed75b700 1 --2- 172.21.15.16:0/400203896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee819a1a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4003f20 tx=0x7f4ee4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ee40037d0 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4ee401b440 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee819a8e0 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ee4023590 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.819+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee819b440 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.820+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4ee4023b00 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.820+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ee402ce40 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.820+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee8104650 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.823+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f4ee401b5b0 con 0x7f4ee8106f70 2024-05-08T23:11:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.896+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f4ee810a860 con 0x7f4ee8106f70 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.898+0000 7f4ede7fc700 1 -- 172.21.15.16:0/400203896 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4ee401b750 con 0x7f4ee8106f70 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 msgr2=0x7f4ee819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 --2- 172.21.15.16:0/400203896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee819a1a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4003f20 tx=0x7f4ee4011f00 comp rx=0 tx=0).stop 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 shutdown_connections 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 --2- 172.21.15.16:0/400203896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8106f70 0x7f4ee819a1a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 >> 172.21.15.16:0/400203896 conn(0x7f4ee8100960 msgr2=0x7f4ee8101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 shutdown_connections 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:11:58.899+0000 7f4eef9bf700 1 -- 172.21.15.16:0/400203896 wait complete. 2024-05-08T23:11:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T23:11:59.305 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:11:59.305 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-05-08T23:11:59.305 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T23:11:59.306 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T23:11:59.549 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x 2024-05-08T23:11:59.550 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T23:11:59.652 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-378de17a-0d90-11ef-bc97-c7b262605968.target.wants/ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-378de17a-0d90-11ef-bc97-c7b262605968@.service. 2024-05-08T23:11:59.659 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:11:59 smithi016 systemd[1]: Starting Ceph mgr.x for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:11:59.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:59 smithi016 ceph-mon[42541]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T23:11:59.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:59 smithi016 ceph-mon[42541]: monmap e1: 1 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:11:59.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:59 smithi016 ceph-mon[42541]: fsmap 2024-05-08T23:11:59.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:59 smithi016 ceph-mon[42541]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T23:11:59.659 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:59 smithi016 ceph-mon[42541]: mgrmap e1: no daemons active 2024-05-08T23:11:59.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:11:59 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/400203896' entity='client.admin' 2024-05-08T23:12:00.077 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:12:00.077 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-08T23:12:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:12:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-08T23:12:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-08T23:12:00.086 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T23:12:00.096 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:12:00.096 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-08T23:12:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:12:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-08T23:12:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-08T23:12:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T23:12:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-05-08T23:12:00.105 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-05-08T23:12:00.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:00 smithi016 bash[42778]: 06b8e61a7de0abb6a3c97c848214fcc45cabd78c951247204d0ed0acdf8f067d 2024-05-08T23:12:00.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:00 smithi016 systemd[1]: Started Ceph mgr.x for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:12:00.823 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:00 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/682346476' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:12:00.823 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:00.518+0000 7fa7221f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:12:00.823 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:00.625+0000 7fa7221f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:12:00.823 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:00.728+0000 7fa7221f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:12:00.823 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:00.823+0000 7fa7221f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:12:00.930 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:12:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:12:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:12:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:00.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:12:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:12:00.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:11:54.945737+0000", 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.499+0000 7f8b7abe2700 1 Processor -- start 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.499+0000 7f8b7abe2700 1 -- start start 2024-05-08T23:12:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.500+0000 7f8b7abe2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b741092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.500+0000 7f8b7abe2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74079ed0 con 0x7f8b74106ed0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.500+0000 7f8b7897e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b741092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.500+0000 7f8b7897e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b741092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37960/0 (socket says 172.21.15.16:37960) 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.500+0000 7f8b7897e700 1 -- 172.21.15.16:0/268859861 learned_addr learned my addr 172.21.15.16:0/268859861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.501+0000 7f8b7897e700 1 -- 172.21.15.16:0/268859861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74109800 con 0x7f8b74106ed0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.501+0000 7f8b7897e700 1 --2- 172.21.15.16:0/268859861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b741092c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b680098a0 tx=0x7f8b68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b983062dac10393 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.501+0000 7f8b737fe700 1 -- 172.21.15.16:0/268859861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b68004160 con 0x7f8b74106ed0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.501+0000 7f8b737fe700 1 -- 172.21.15.16:0/268859861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8b680042c0 con 0x7f8b74106ed0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.501+0000 7f8b737fe700 1 -- 172.21.15.16:0/268859861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b68011940 con 0x7f8b74106ed0 2024-05-08T23:12:00.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 -- 172.21.15.16:0/268859861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 msgr2=0x7f8b741092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 --2- 172.21.15.16:0/268859861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b741092c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b680098a0 tx=0x7f8b68009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 -- 172.21.15.16:0/268859861 shutdown_connections 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 --2- 172.21.15.16:0/268859861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b741092c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 -- 172.21.15.16:0/268859861 >> 172.21.15.16:0/268859861 conn(0x7f8b74100920 msgr2=0x7f8b74102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 -- 172.21.15.16:0/268859861 shutdown_connections 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.502+0000 7f8b7abe2700 1 -- 172.21.15.16:0/268859861 wait complete. 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7abe2700 1 Processor -- start 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7abe2700 1 -- start start 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7abe2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b7419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7abe2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419c930 con 0x7f8b74106ed0 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7897e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b7419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:00.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7897e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b7419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37968/0 (socket says 172.21.15.16:37968) 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7897e700 1 -- 172.21.15.16:0/682346476 learned_addr learned my addr 172.21.15.16:0/682346476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.503+0000 7f8b7897e700 1 -- 172.21.15.16:0/682346476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68009550 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.504+0000 7f8b7897e700 1 --2- 172.21.15.16:0/682346476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b7419c3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006af0 tx=0x7f8b68011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.504+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b68003bf0 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.504+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8b6801b8b0 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.504+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7419cb30 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.504+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b68023800 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.504+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7419d690 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.505+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f8b68023a70 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.505+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8b6802ce40 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.505+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b741045f0 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.507+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f8b6801bd70 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.585+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8b7410a5a0 con 0x7f8b74106ed0 2024-05-08T23:12:00.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.586+0000 7f8b71ffb700 1 -- 172.21.15.16:0/682346476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f8b6801ba20 con 0x7f8b74106ed0 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 msgr2=0x7f8b7419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 --2- 172.21.15.16:0/682346476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b7419c3f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006af0 tx=0x7f8b68011a50 comp rx=0 tx=0).stop 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 shutdown_connections 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 --2- 172.21.15.16:0/682346476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b74106ed0 0x7f8b7419c3f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 >> 172.21.15.16:0/682346476 conn(0x7f8b74100920 msgr2=0x7f8b74101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 shutdown_connections 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:00.587+0000 7f8b7abe2700 1 -- 172.21.15.16:0/682346476 wait complete. 2024-05-08T23:12:00.949 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-05-08T23:12:01.837 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:01 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:01.555+0000 7fa7221f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:12:02.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:01 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:01.837+0000 7fa7221f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:12:02.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:01 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:01.932+0000 7fa7221f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:12:02.114 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:02.055+0000 7fa7221f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:12:02.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:02.233+0000 7fa7221f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:12:02.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:02.377+0000 7fa7221f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:12:02.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:02.472+0000 7fa7221f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:12:03.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:02.906+0000 7fa7221f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:12:03.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:03 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3795745519' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:12:03.646 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:03 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:03.367+0000 7fa7221f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:12:03.646 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:03 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:03.592+0000 7fa7221f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:12:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:12:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:12:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:12:03.746 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:03.747 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:12:03.748 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.749 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:12:03.750 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:11:54.945737+0000", 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.309+0000 7fddcc443700 1 Processor -- start 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.309+0000 7fddcc443700 1 -- start start 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddcc443700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddcc443700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc407a8d0 con 0x7fddc407bea0 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddca1df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddca1df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37976/0 (socket says 172.21.15.16:37976) 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddca1df700 1 -- 172.21.15.16:0/2081200321 learned_addr learned my addr 172.21.15.16:0/2081200321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:03.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddca1df700 1 -- 172.21.15.16:0/2081200321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc407aa10 con 0x7fddc407bea0 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddca1df700 1 --2- 172.21.15.16:0/2081200321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc407a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fddc00098a0 tx=0x7fddc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a562e82feb6a64b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.310+0000 7fddc91dd700 1 -- 172.21.15.16:0/2081200321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddc0004160 con 0x7fddc407bea0 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddc91dd700 1 -- 172.21.15.16:0/2081200321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fddc00042c0 con 0x7fddc407bea0 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddc91dd700 1 -- 172.21.15.16:0/2081200321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddc0011940 con 0x7fddc407bea0 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 -- 172.21.15.16:0/2081200321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 msgr2=0x7fddc407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 --2- 172.21.15.16:0/2081200321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc407a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fddc00098a0 tx=0x7fddc0009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 -- 172.21.15.16:0/2081200321 shutdown_connections 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 --2- 172.21.15.16:0/2081200321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc407a300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 -- 172.21.15.16:0/2081200321 >> 172.21.15.16:0/2081200321 conn(0x7fddc4101a20 msgr2=0x7fddc4103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 -- 172.21.15.16:0/2081200321 shutdown_connections 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.311+0000 7fddcc443700 1 -- 172.21.15.16:0/2081200321 wait complete. 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.312+0000 7fddcc443700 1 Processor -- start 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.312+0000 7fddcc443700 1 -- start start 2024-05-08T23:12:03.752 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddcc443700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddcc443700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc419c920 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddca1df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddca1df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37992/0 (socket says 172.21.15.16:37992) 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddca1df700 1 -- 172.21.15.16:0/3795745519 learned_addr learned my addr 172.21.15.16:0/3795745519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddca1df700 1 -- 172.21.15.16:0/3795745519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009550 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddca1df700 1 --2- 172.21.15.16:0/3795745519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc419c3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fddc0006af0 tx=0x7fddc0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddc0003bf0 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fddc001b8b0 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc419cb20 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.313+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddc0023800 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.314+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc419d680 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.314+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fddc0023960 con 0x7fddc407bea0 2024-05-08T23:12:03.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.314+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fddc002ce00 con 0x7fddc407bea0 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.314+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc41056f0 con 0x7fddc407bea0 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.317+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fddc001bd70 con 0x7fddc407bea0 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.393+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fddc410c7f0 con 0x7fddc407bea0 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.393+0000 7fddbb7fe700 1 -- 172.21.15.16:0/3795745519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fddc0023d50 con 0x7fddc407bea0 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 msgr2=0x7fddc419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 --2- 172.21.15.16:0/3795745519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc419c3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fddc0006af0 tx=0x7fddc0011a50 comp rx=0 tx=0).stop 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 shutdown_connections 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 --2- 172.21.15.16:0/3795745519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc407bea0 0x7fddc419c3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 >> 172.21.15.16:0/3795745519 conn(0x7fddc4101a20 msgr2=0x7fddc41025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 shutdown_connections 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:03.394+0000 7fddcc443700 1 -- 172.21.15.16:0/3795745519 wait complete. 2024-05-08T23:12:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-05-08T23:12:04.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:03 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:03.697+0000 7fa7221f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:12:04.839 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:04 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:04.563+0000 7fa7221f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:12:04.839 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:04 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:04.671+0000 7fa7221f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:12:04.839 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:04 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:04.838+0000 7fa7221f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:12:05.842 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:05 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:05.555+0000 7fa7221f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:12:05.842 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:05 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:05.759+0000 7fa7221f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:12:06.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:05 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:05.988+0000 7fa7221f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:12:06.510 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:06.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:06.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:12:06.520 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-08T23:12:06.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:12:06.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:12:06.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:11:54.945737+0000", 2024-05-08T23:12:06.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.076+0000 7f8b4db8c700 1 Processor -- start 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.076+0000 7f8b4db8c700 1 -- start start 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b4db8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b481092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b4db8c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48079ed0 con 0x7f8b48106ed0 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b481092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b481092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37994/0 (socket says 172.21.15.16:37994) 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b477fe700 1 -- 172.21.15.16:0/1580356763 learned_addr learned my addr 172.21.15.16:0/1580356763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b477fe700 1 -- 172.21.15.16:0/1580356763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48109800 con 0x7f8b48106ed0 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b477fe700 1 --2- 172.21.15.16:0/1580356763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b481092c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098a0 tx=0x7f8b38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564a56b57bdd5013 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.077+0000 7f8b467fc700 1 -- 172.21.15.16:0/1580356763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b38004160 con 0x7f8b48106ed0 2024-05-08T23:12:06.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b467fc700 1 -- 172.21.15.16:0/1580356763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8b380042c0 con 0x7f8b48106ed0 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b467fc700 1 -- 172.21.15.16:0/1580356763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b38011940 con 0x7f8b48106ed0 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 -- 172.21.15.16:0/1580356763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 msgr2=0x7f8b481092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 --2- 172.21.15.16:0/1580356763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b481092c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098a0 tx=0x7f8b38009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 -- 172.21.15.16:0/1580356763 shutdown_connections 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 --2- 172.21.15.16:0/1580356763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b481092c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 -- 172.21.15.16:0/1580356763 >> 172.21.15.16:0/1580356763 conn(0x7f8b48100920 msgr2=0x7f8b48102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 -- 172.21.15.16:0/1580356763 shutdown_connections 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.078+0000 7f8b4db8c700 1 -- 172.21.15.16:0/1580356763 wait complete. 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.079+0000 7f8b4db8c700 1 Processor -- start 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.079+0000 7f8b4db8c700 1 -- start start 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.079+0000 7f8b4db8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b4819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.079+0000 7f8b4db8c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4819a770 con 0x7f8b48106ed0 2024-05-08T23:12:06.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b4819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b4819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38000/0 (socket says 172.21.15.16:38000) 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b477fe700 1 -- 172.21.15.16:0/3389811802 learned_addr learned my addr 172.21.15.16:0/3389811802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b477fe700 1 -- 172.21.15.16:0/3389811802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009550 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b477fe700 1 --2- 172.21.15.16:0/3389811802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b4819a230 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006af0 tx=0x7f8b38011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b38003bf0 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8b3801b8b0 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4819a970 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b38023800 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.080+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4819b480 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.081+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f8b38023c60 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.081+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8b38019710 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.082+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b481045f0 con 0x7f8b48106ed0 2024-05-08T23:12:06.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.086+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f8b3801bd70 con 0x7f8b48106ed0 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.162+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8b4810a5a0 con 0x7f8b48106ed0 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.162+0000 7f8b44ff9700 1 -- 172.21.15.16:0/3389811802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f8b3801ba20 con 0x7f8b48106ed0 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.163+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 msgr2=0x7f8b4819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.163+0000 7f8b4db8c700 1 --2- 172.21.15.16:0/3389811802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b4819a230 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006af0 tx=0x7f8b38011a50 comp rx=0 tx=0).stop 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.163+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 shutdown_connections 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.163+0000 7f8b4db8c700 1 --2- 172.21.15.16:0/3389811802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b48106ed0 0x7f8b4819a230 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.163+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 >> 172.21.15.16:0/3389811802 conn(0x7f8b48100920 msgr2=0x7f8b481014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.164+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 shutdown_connections 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:06.164+0000 7f8b4db8c700 1 -- 172.21.15.16:0/3389811802 wait complete. 2024-05-08T23:12:06.528 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (3/15)... 2024-05-08T23:12:06.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:06 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3389811802' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:12:07.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:07.065+0000 7fa7221f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:12:07.700 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:07.385+0000 7fa7221f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:12:07.701 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:07.483+0000 7fa7221f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:12:07.701 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:07.700+0000 7fa7221f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: Activating manager daemon x 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: mgrmap e2: x(active, starting, since 0.00333964s) 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: Manager daemon x is now available 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:12:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' 2024-05-08T23:12:08.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' 2024-05-08T23:12:08.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:07 smithi016 ceph-mon[42541]: from='mgr.14100 172.21.15.16:0/1279559436' entity='mgr.x' 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.556 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T23:12:09.557 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T23:12:09.558 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:09.559 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T23:11:54.945737+0000", 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.882+0000 7f0a2af4a700 1 Processor -- start 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.882+0000 7f0a2af4a700 1 -- start start 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.882+0000 7f0a2af4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.882+0000 7f0a2af4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24079e70 con 0x7f0a24106f30 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.883+0000 7f0a28ce6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:09.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.883+0000 7f0a28ce6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60926/0 (socket says 172.21.15.16:60926) 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.883+0000 7f0a28ce6700 1 -- 172.21.15.16:0/2017839394 learned_addr learned my addr 172.21.15.16:0/2017839394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.883+0000 7f0a28ce6700 1 -- 172.21.15.16:0/2017839394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a24109860 con 0x7f0a24106f30 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.883+0000 7f0a28ce6700 1 --2- 172.21.15.16:0/2017839394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24109320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0a180098a0 tx=0x7f0a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2665992516e4df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a237fe700 1 -- 172.21.15.16:0/2017839394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a18004160 con 0x7f0a24106f30 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a237fe700 1 -- 172.21.15.16:0/2017839394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0a180042c0 con 0x7f0a24106f30 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a237fe700 1 -- 172.21.15.16:0/2017839394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a18011940 con 0x7f0a24106f30 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 -- 172.21.15.16:0/2017839394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 msgr2=0x7f0a24109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 --2- 172.21.15.16:0/2017839394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24109320 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0a180098a0 tx=0x7f0a18009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 -- 172.21.15.16:0/2017839394 shutdown_connections 2024-05-08T23:12:09.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 --2- 172.21.15.16:0/2017839394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24109320 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 -- 172.21.15.16:0/2017839394 >> 172.21.15.16:0/2017839394 conn(0x7f0a24100920 msgr2=0x7f0a24102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 -- 172.21.15.16:0/2017839394 shutdown_connections 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.884+0000 7f0a2af4a700 1 -- 172.21.15.16:0/2017839394 wait complete. 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.885+0000 7f0a2af4a700 1 Processor -- start 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.885+0000 7f0a2af4a700 1 -- start start 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a2af4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a2af4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24104ee0 con 0x7f0a24106f30 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a28ce6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a28ce6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60934/0 (socket says 172.21.15.16:60934) 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a28ce6700 1 -- 172.21.15.16:0/4096472523 learned_addr learned my addr 172.21.15.16:0/4096472523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a28ce6700 1 -- 172.21.15.16:0/4096472523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18009550 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a28ce6700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24106940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a18006af0 tx=0x7f0a18011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a18003bf0 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a241050e0 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0a1801b8b0 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.886+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a18023800 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.887+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a24105580 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.887+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f0a18023a70 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.888+0000 7f0a21ffb700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f0a14038540 0x7f0a1403aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.888+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a24066f40 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.888+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0a1804c910 con 0x7f0a24106f30 2024-05-08T23:12:09.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.888+0000 7f0a23fff700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f0a14038540 0x7f0a1403aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.888+0000 7f0a23fff700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f0a14038540 0x7f0a1403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a10006fd0 tx=0x7f0a10006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:08.893+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a18023d20 con 0x7f0a24106f30 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.190+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0a2410a5a0 con 0x7f0a24106f30 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.190+0000 7f0a21ffb700 1 -- 172.21.15.16:0/4096472523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0a1802bb00 con 0x7f0a24106f30 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f0a14038540 msgr2=0x7f0a1403aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f0a14038540 0x7f0a1403aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a10006fd0 tx=0x7f0a10006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 msgr2=0x7f0a24106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24106940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a18006af0 tx=0x7f0a18011a50 comp rx=0 tx=0).stop 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 shutdown_connections 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f0a14038540 0x7f0a1403aa00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:09.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 --2- 172.21.15.16:0/4096472523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24106f30 0x7f0a24106940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:09.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 >> 172.21.15.16:0/4096472523 conn(0x7f0a24100920 msgr2=0x7f0a241014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:09.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 shutdown_connections 2024-05-08T23:12:09.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.193+0000 7f0a2af4a700 1 -- 172.21.15.16:0/4096472523 wait complete. 2024-05-08T23:12:09.566 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-05-08T23:12:09.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:09 smithi016 ceph-mon[42541]: mgrmap e3: x(active, since 1.00761s) 2024-05-08T23:12:09.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:09 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4096472523' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T23:12:10.493 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.910+0000 7f88b3194700 1 Processor -- start 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.910+0000 7f88b3194700 1 -- start start 2024-05-08T23:12:10.494 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b3194700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac105170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b3194700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac079e50 con 0x7f88ac104d50 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b0f30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac105170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b0f30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac105170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60950/0 (socket says 172.21.15.16:60950) 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b0f30700 1 -- 172.21.15.16:0/2749945641 learned_addr learned my addr 172.21.15.16:0/2749945641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b0f30700 1 -- 172.21.15.16:0/2749945641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac1056b0 con 0x7f88ac104d50 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.911+0000 7f88b0f30700 1 --2- 172.21.15.16:0/2749945641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac105170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098a0 tx=0x7f889c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35763852f332c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88ab7fe700 1 -- 172.21.15.16:0/2749945641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f889c004160 con 0x7f88ac104d50 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88ab7fe700 1 -- 172.21.15.16:0/2749945641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f889c0042c0 con 0x7f88ac104d50 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88ab7fe700 1 -- 172.21.15.16:0/2749945641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f889c011940 con 0x7f88ac104d50 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 -- 172.21.15.16:0/2749945641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 msgr2=0x7f88ac105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 --2- 172.21.15.16:0/2749945641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac105170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098a0 tx=0x7f889c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:10.495 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 -- 172.21.15.16:0/2749945641 shutdown_connections 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 --2- 172.21.15.16:0/2749945641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac105170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 -- 172.21.15.16:0/2749945641 >> 172.21.15.16:0/2749945641 conn(0x7f88ac1008f0 msgr2=0x7f88ac102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 -- 172.21.15.16:0/2749945641 shutdown_connections 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.912+0000 7f88b3194700 1 -- 172.21.15.16:0/2749945641 wait complete. 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.913+0000 7f88b3194700 1 Processor -- start 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.913+0000 7f88b3194700 1 -- start start 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.913+0000 7f88b3194700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.913+0000 7f88b3194700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19c990 con 0x7f88ac104d50 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88b0f30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88b0f30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60958/0 (socket says 172.21.15.16:60958) 2024-05-08T23:12:10.496 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88b0f30700 1 -- 172.21.15.16:0/3908025552 learned_addr learned my addr 172.21.15.16:0/3908025552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88b0f30700 1 -- 172.21.15.16:0/3908025552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c009550 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88b0f30700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac19c450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f889c003e80 tx=0x7f889c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f889c003ae0 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac19cb90 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f889c003c40 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.914+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f889c019810 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.915+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac19d6f0 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.915+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7f889c019ad0 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.916+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac104600 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.916+0000 7f88a9ffb700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f8894038520 0x7f889403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.916+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f889c04c880 con 0x7f88ac104d50 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.916+0000 7f88abfff700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f8894038520 0x7f889403a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:10.497 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.916+0000 7f88abfff700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f8894038520 0x7f889403a9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006fd0 tx=0x7f88a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:09.921+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f889c018d60 con 0x7f88ac104d50 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.131+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f88ac066f40 con 0x7f88ac104d50 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.132+0000 7f88a9ffb700 1 -- 172.21.15.16:0/3908025552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f889c018350 con 0x7f88ac104d50 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f8894038520 msgr2=0x7f889403a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f8894038520 0x7f889403a9e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006fd0 tx=0x7f88a0006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 msgr2=0x7f88ac19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac19c450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f889c003e80 tx=0x7f889c011a30 comp rx=0 tx=0).stop 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 shutdown_connections 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f8894038520 0x7f889403a9e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 --2- 172.21.15.16:0/3908025552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88ac104d50 0x7f88ac19c450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 >> 172.21.15.16:0/3908025552 conn(0x7f88ac1008f0 msgr2=0x7f88ac193110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 shutdown_connections 2024-05-08T23:12:10.498 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.135+0000 7f88b3194700 1 -- 172.21.15.16:0/3908025552 wait complete. 2024-05-08T23:12:10.499 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-05-08T23:12:10.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3908025552' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T23:12:12.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:11 smithi016 ceph-mon[42541]: mgrmap e4: x(active, since 3s) 2024-05-08T23:12:12.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:11 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/344217886' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T23:12:12.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:11 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: ignoring --setuser ceph since I am not root 2024-05-08T23:12:12.004 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:11 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: ignoring --setgroup ceph since I am not root 2024-05-08T23:12:12.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.856+0000 7fb44846c700 1 Processor -- start 2024-05-08T23:12:12.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.856+0000 7fb44846c700 1 -- start start 2024-05-08T23:12:12.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb44846c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb4401068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb44846c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb440106e90 con 0x7fb4401064a0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb446208700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb4401068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb446208700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb4401068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60974/0 (socket says 172.21.15.16:60974) 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb446208700 1 -- 172.21.15.16:0/3412093057 learned_addr learned my addr 172.21.15.16:0/3412093057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb446208700 1 -- 172.21.15.16:0/3412093057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4401076a0 con 0x7fb4401064a0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.857+0000 7fb446208700 1 --2- 172.21.15.16:0/3412093057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb4401068c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72955481896a6fb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb445206700 1 -- 172.21.15.16:0/3412093057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb43c004160 con 0x7fb4401064a0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb445206700 1 -- 172.21.15.16:0/3412093057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb43c0042c0 con 0x7fb4401064a0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb445206700 1 -- 172.21.15.16:0/3412093057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb43c011940 con 0x7fb4401064a0 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 -- 172.21.15.16:0/3412093057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 msgr2=0x7fb4401068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 --2- 172.21.15.16:0/3412093057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb4401068c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098a0 tx=0x7fb43c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 -- 172.21.15.16:0/3412093057 shutdown_connections 2024-05-08T23:12:12.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 --2- 172.21.15.16:0/3412093057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb4401068c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 -- 172.21.15.16:0/3412093057 >> 172.21.15.16:0/3412093057 conn(0x7fb440101a20 msgr2=0x7fb440103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 -- 172.21.15.16:0/3412093057 shutdown_connections 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.858+0000 7fb44846c700 1 -- 172.21.15.16:0/3412093057 wait complete. 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb44846c700 1 Processor -- start 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb44846c700 1 -- start start 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb44846c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb44007bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb44846c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44007a2c0 con 0x7fb4401064a0 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb446208700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb44007bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb446208700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb44007bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60984/0 (socket says 172.21.15.16:60984) 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.859+0000 7fb446208700 1 -- 172.21.15.16:0/344217886 learned_addr learned my addr 172.21.15.16:0/344217886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb446208700 1 -- 172.21.15.16:0/344217886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c009550 con 0x7fb4401064a0 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb446208700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb44007bd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb43c003f20 tx=0x7fb43c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:12.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb43c0036a0 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb43c01b440 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44007a4c0 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb43c023400 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.860+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44007a960 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.861+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7fb43c0236c0 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.861+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb440105730 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.861+0000 7fb4377fe700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb42c0386b0 0x7fb42c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.861+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb43c04c7c0 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.861+0000 7fb445a07700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb42c0386b0 0x7fb42c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.862+0000 7fb445a07700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb42c0386b0 0x7fb42c03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb430006fd0 tx=0x7fb430006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:10.866+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb43c023970 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.140+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb440066f40 con 0x7fb4401064a0 2024-05-08T23:12:12.100 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.723+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb43c02c770 con 0x7fb4401064a0 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.725+0000 7fb4377fe700 1 -- 172.21.15.16:0/344217886 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb43c018d40 con 0x7fb4401064a0 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb42c0386b0 msgr2=0x7fb42c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb42c0386b0 0x7fb42c03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb430006fd0 tx=0x7fb430006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 msgr2=0x7fb44007bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb44007bd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb43c003f20 tx=0x7fb43c011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 shutdown_connections 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb42c0386b0 0x7fb42c03ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 --2- 172.21.15.16:0/344217886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4401064a0 0x7fb44007bd20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 >> 172.21.15.16:0/344217886 conn(0x7fb440101a20 msgr2=0x7fb440102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 shutdown_connections 2024-05-08T23:12:12.101 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:11.728+0000 7fb44846c700 1 -- 172.21.15.16:0/344217886 wait complete. 2024-05-08T23:12:12.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:12.213+0000 7f0476911200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:12:12.319 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:12.318+0000 7f0476911200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:12:12.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:12.419+0000 7f0476911200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:12:12.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:12.516+0000 7f0476911200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:12:13.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/344217886' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T23:12:13.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:12 smithi016 ceph-mon[42541]: mgrmap e5: x(active, since 4s) 2024-05-08T23:12:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3542714096' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T23:12:13.523 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:13.239+0000 7f0476911200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:12:13.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:13.523+0000 7f0476911200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:12:13.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:13.619+0000 7f0476911200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:12:13.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:13.742+0000 7f0476911200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:12:14.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.435+0000 7f76fa75a700 1 Processor -- start 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.435+0000 7f76fa75a700 1 -- start start 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76fa75a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f41092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76fa75a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4079ed0 con 0x7f76f4106ed0 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f41092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f41092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60998/0 (socket says 172.21.15.16:60998) 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76f3fff700 1 -- 172.21.15.16:0/4045137845 learned_addr learned my addr 172.21.15.16:0/4045137845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:14.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76f3fff700 1 -- 172.21.15.16:0/4045137845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4109800 con 0x7f76f4106ed0 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.436+0000 7f76f3fff700 1 --2- 172.21.15.16:0/4045137845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f41092c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098a0 tx=0x7f76e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bd27ae1dfe4aee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76f2ffd700 1 -- 172.21.15.16:0/4045137845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76e4004160 con 0x7f76f4106ed0 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76f2ffd700 1 -- 172.21.15.16:0/4045137845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f76e40042c0 con 0x7f76f4106ed0 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76f2ffd700 1 -- 172.21.15.16:0/4045137845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76e4011940 con 0x7f76f4106ed0 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 -- 172.21.15.16:0/4045137845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 msgr2=0x7f76f41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 --2- 172.21.15.16:0/4045137845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f41092c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098a0 tx=0x7f76e4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 -- 172.21.15.16:0/4045137845 shutdown_connections 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 --2- 172.21.15.16:0/4045137845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f41092c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 -- 172.21.15.16:0/4045137845 >> 172.21.15.16:0/4045137845 conn(0x7f76f4100920 msgr2=0x7f76f4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 -- 172.21.15.16:0/4045137845 shutdown_connections 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.437+0000 7f76fa75a700 1 -- 172.21.15.16:0/4045137845 wait complete. 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.438+0000 7f76fa75a700 1 Processor -- start 2024-05-08T23:12:14.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.438+0000 7f76fa75a700 1 -- start start 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76fa75a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76fa75a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419c940 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32770/0 (socket says 172.21.15.16:32770) 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f3fff700 1 -- 172.21.15.16:0/3542714096 learned_addr learned my addr 172.21.15.16:0/3542714096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f3fff700 1 -- 172.21.15.16:0/3542714096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4009550 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f3fff700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f419c400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76e4006af0 tx=0x7f76e4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76e4003bf0 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f76e401b8b0 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f419cb40 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76e4023800 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.439+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f419d6a0 con 0x7f76f4106ed0 2024-05-08T23:12:14.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.440+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f76e4023960 con 0x7f76f4106ed0 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.441+0000 7f76f17fa700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 0x7f76dc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.441+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f76e404cac0 con 0x7f76f4106ed0 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.441+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e0005320 con 0x7f76f4106ed0 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.441+0000 7f76f37fe700 1 -- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 msgr2=0x7f76dc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.441+0000 7f76f37fe700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 0x7f76dc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.446+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f76e4023c10 con 0x7f76f4106ed0 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.642+0000 7f76f37fe700 1 -- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 msgr2=0x7f76dc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.642+0000 7f76f37fe700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 0x7f76dc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.720+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f76e0005de0 con 0x7f76f4106ed0 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.720+0000 7f76f17fa700 1 -- 172.21.15.16:0/3542714096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f76e402bb40 con 0x7f76f4106ed0 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 msgr2=0x7f76dc03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 0x7f76dc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 msgr2=0x7f76f419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f419c400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76e4006af0 tx=0x7f76e4011a50 comp rx=0 tx=0).stop 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 shutdown_connections 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7f76dc038700 0x7f76dc03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 --2- 172.21.15.16:0/3542714096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f4106ed0 0x7f76f419c400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 >> 172.21.15.16:0/3542714096 conn(0x7f76f4100920 msgr2=0x7f76f41013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 shutdown_connections 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:12.723+0000 7f76fa75a700 1 -- 172.21.15.16:0/3542714096 wait complete. 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-05-08T23:12:14.036 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-05-08T23:12:14.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:13.921+0000 7f0476911200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:12:14.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:14.070+0000 7f0476911200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:12:14.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:14.166+0000 7f0476911200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:12:14.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:14.595+0000 7f0476911200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:12:15.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:15.039+0000 7f0476911200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:12:15.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:15.260+0000 7f0476911200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:12:15.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:15.365+0000 7f0476911200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:12:16.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:16.231+0000 7f0476911200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:12:16.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:16.339+0000 7f0476911200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:12:16.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:16.508+0000 7f0476911200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:12:17.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:17.227+0000 7f0476911200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:12:17.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:17.431+0000 7f0476911200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:12:18.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:17.650+0000 7f0476911200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:12:19.024 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:18 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:18.703+0000 7f0476911200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:12:19.338 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:19.023+0000 7f0476911200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:12:19.339 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:19.123+0000 7f0476911200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:12:19.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:19.337+0000 7f0476911200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: Active manager daemon x restarted 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: Activating manager daemon x 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: mgrmap e6: x(active, starting, since 0.00357031s) 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: Manager daemon x is now available 2024-05-08T23:12:19.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:19.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:19.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:19 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:20.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: Found migration_current of "None". Setting to last migration. 2024-05-08T23:12:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:12:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:12:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:20 smithi016 ceph-mon[42541]: mgrmap e7: x(active, since 1.00559s) 2024-05-08T23:12:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:21 smithi016 ceph-mon[42541]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T23:12:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:21 smithi016 ceph-mon[42541]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T23:12:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:21 smithi016 ceph-mon[42541]: [08/May/2024:23:12:20] ENGINE Bus STARTING 2024-05-08T23:12:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:21 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.252+0000 7fb1c442d700 1 Processor -- start 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.252+0000 7fb1c442d700 1 -- start start 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c442d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc109340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:22.788 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c442d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc079e50 con 0x7fb1bc106f50 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c21c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c21c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32774/0 (socket says 172.21.15.16:32774) 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c21c9700 1 -- 172.21.15.16:0/731237907 learned_addr learned my addr 172.21.15.16:0/731237907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c21c9700 1 -- 172.21.15.16:0/731237907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc109880 con 0x7fb1bc106f50 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.253+0000 7fb1c21c9700 1 --2- 172.21.15.16:0/731237907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc109340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098a0 tx=0x7fb1b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0086d39525712f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c11c7700 1 -- 172.21.15.16:0/731237907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b8004160 con 0x7fb1bc106f50 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c11c7700 1 -- 172.21.15.16:0/731237907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb1b80042c0 con 0x7fb1bc106f50 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c11c7700 1 -- 172.21.15.16:0/731237907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b8011940 con 0x7fb1bc106f50 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 -- 172.21.15.16:0/731237907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 msgr2=0x7fb1bc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 --2- 172.21.15.16:0/731237907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc109340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098a0 tx=0x7fb1b8009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 -- 172.21.15.16:0/731237907 shutdown_connections 2024-05-08T23:12:22.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 --2- 172.21.15.16:0/731237907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc109340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 -- 172.21.15.16:0/731237907 >> 172.21.15.16:0/731237907 conn(0x7fb1bc100920 msgr2=0x7fb1bc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 -- 172.21.15.16:0/731237907 shutdown_connections 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.254+0000 7fb1c442d700 1 -- 172.21.15.16:0/731237907 wait complete. 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c442d700 1 Processor -- start 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c442d700 1 -- start start 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c442d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc106910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c442d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc104f00 con 0x7fb1bc106f50 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c21c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c21c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32784/0 (socket says 172.21.15.16:32784) 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.255+0000 7fb1c21c9700 1 -- 172.21.15.16:0/1067487395 learned_addr learned my addr 172.21.15.16:0/1067487395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1c21c9700 1 -- 172.21.15.16:0/1067487395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8009550 con 0x7fb1bc106f50 2024-05-08T23:12:22.790 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1c21c9700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc106910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8003f20 tx=0x7fb1b8011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b80037d0 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb1b801b440 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc105100 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b8023590 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.256+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1055a0 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.257+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb1b8023800 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.257+0000 7fb1b37fe700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.257+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 --> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1a803b320 con 0x7fb1a8038750 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.257+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb1b804c630 con 0x7fb1bc106f50 2024-05-08T23:12:22.791 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.257+0000 7fb1c19c8700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 msgr2=0x7fb1a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:22.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:22 smithi016 ceph-mon[42541]: [08/May/2024:23:12:20] ENGINE Serving on http://172.21.15.16:8765 2024-05-08T23:12:22.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:22 smithi016 ceph-mon[42541]: [08/May/2024:23:12:20] ENGINE Serving on https://172.21.15.16:7150 2024-05-08T23:12:22.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:22 smithi016 ceph-mon[42541]: [08/May/2024:23:12:20] ENGINE Bus STARTED 2024-05-08T23:12:22.792 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:22 smithi016 ceph-mon[42541]: mgrmap e8: x(active, since 2s) 2024-05-08T23:12:22.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.257+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:12:22.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.458+0000 7fb1c19c8700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 msgr2=0x7fb1a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:22.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.458+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:12:22.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.859+0000 7fb1c19c8700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 msgr2=0x7fb1a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:22.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:15.859+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:12:22.795 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:16.660+0000 7fb1c19c8700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 msgr2=0x7fb1a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:16.660+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:18.261+0000 7fb1c19c8700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 msgr2=0x7fb1a803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3684081056 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:18.261+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:19.353+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7fb1b8023c90 con 0x7fb1bc106f50 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:19.353+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 msgr2=0x7fb1a803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:19.353+0000 7fb1b37fe700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.354+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fb1b801b5b0 con 0x7fb1bc106f50 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.355+0000 7fb1b37fe700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fb1a803bfe0 0x7fb1a803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.355+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1a803b320 con 0x7fb1a803bfe0 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.355+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fb1a803bfe0 0x7fb1a803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.356+0000 7fb1c19c8700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fb1a803bfe0 0x7fb1a803e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac003a10 tx=0x7fb1ac0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:22.796 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.356+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fb1a803b320 con 0x7fb1a803bfe0 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.368+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb1bc1045f0 con 0x7fb1a803bfe0 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.368+0000 7fb1b37fe700 1 -- 172.21.15.16:0/1067487395 <== mgr.14120 v2:172.21.15.16:6800/1569877251 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb1bc1045f0 con 0x7fb1a803bfe0 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fb1a803bfe0 msgr2=0x7fb1a803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fb1a803bfe0 0x7fb1a803e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac003a10 tx=0x7fb1ac0092b0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 msgr2=0x7fb1bc106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc106910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8003f20 tx=0x7fb1b8011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 shutdown_connections 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fb1a803bfe0 0x7fb1a803e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc106f50 0x7fb1bc106910 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 --2- 172.21.15.16:0/1067487395 >> [v2:172.21.15.16:6800/3684081056,v1:172.21.15.16:6801/3684081056] conn(0x7fb1a8038750 0x7fb1a803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 >> 172.21.15.16:0/1067487395 conn(0x7fb1bc100920 msgr2=0x7fb1bc101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 shutdown_connections 2024-05-08T23:12:22.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:20.369+0000 7fb1c442d700 1 -- 172.21.15.16:0/1067487395 wait complete. 2024-05-08T23:12:22.798 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-05-08T23:12:22.798 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-05-08T23:12:23.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.157+0000 7f2aa1eaa700 1 Processor -- start 2024-05-08T23:12:23.763 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.157+0000 7f2aa1eaa700 1 -- start start 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.157+0000 7f2aa1eaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.157+0000 7f2aa1eaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c079e90 con 0x7f2a9c106f70 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.157+0000 7f2a9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c109360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35350/0 (socket says 172.21.15.16:35350) 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9b7fe700 1 -- 172.21.15.16:0/1330559636 learned_addr learned my addr 172.21.15.16:0/1330559636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9b7fe700 1 -- 172.21.15.16:0/1330559636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c1098a0 con 0x7f2a9c106f70 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9b7fe700 1 --2- 172.21.15.16:0/1330559636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c109360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4592118b5b348123 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9a7fc700 1 -- 172.21.15.16:0/1330559636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c004160 con 0x7f2a9c106f70 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9a7fc700 1 -- 172.21.15.16:0/1330559636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2a8c0042c0 con 0x7f2a9c106f70 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.158+0000 7f2a9a7fc700 1 -- 172.21.15.16:0/1330559636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c011940 con 0x7f2a9c106f70 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/1330559636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 msgr2=0x7f2a9c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 --2- 172.21.15.16:0/1330559636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c109360 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:23.764 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/1330559636 shutdown_connections 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 --2- 172.21.15.16:0/1330559636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c109360 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/1330559636 >> 172.21.15.16:0/1330559636 conn(0x7f2a9c100960 msgr2=0x7f2a9c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/1330559636 shutdown_connections 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/1330559636 wait complete. 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.159+0000 7f2aa1eaa700 1 Processor -- start 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2aa1eaa700 1 -- start start 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2aa1eaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2aa1eaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c19c940 con 0x7f2a9c106f70 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2a9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2a9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35352/0 (socket says 172.21.15.16:35352) 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2a9b7fe700 1 -- 172.21.15.16:0/4087591058 learned_addr learned my addr 172.21.15.16:0/4087591058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2a9b7fe700 1 -- 172.21.15.16:0/4087591058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c009550 con 0x7f2a9c106f70 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.160+0000 7f2a9b7fe700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c19c400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c003f20 tx=0x7f2a8c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:23.765 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.161+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c0037d0 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.161+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2a8c01b440 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.161+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c19cb40 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.161+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a8c023590 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.161+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c19d6a0 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.162+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f2a8c023850 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.162+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c104650 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.162+0000 7f2a98ff9700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f2a88038700 0x7f2a8803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.162+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2a8c04c770 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.162+0000 7f2a9affd700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f2a88038700 0x7f2a8803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.162+0000 7f2a9affd700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f2a88038700 0x7f2a8803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006fd0 tx=0x7f2a84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.167+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a8c023b00 con 0x7f2a9c106f70 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.398+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a9c101aa0 con 0x7f2a88038700 2024-05-08T23:12:23.766 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.407+0000 7f2a98ff9700 1 -- 172.21.15.16:0/4087591058 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2a9c101aa0 con 0x7f2a88038700 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f2a88038700 msgr2=0x7f2a8803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f2a88038700 0x7f2a8803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2a84006fd0 tx=0x7f2a84006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 msgr2=0x7f2a9c19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c19c400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c003f20 tx=0x7f2a8c011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 shutdown_connections 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f2a88038700 0x7f2a8803abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 --2- 172.21.15.16:0/4087591058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a9c106f70 0x7f2a9c19c400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.409+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 >> 172.21.15.16:0/4087591058 conn(0x7f2a9c100960 msgr2=0x7f2a9c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.410+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 shutdown_connections 2024-05-08T23:12:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:23.410+0000 7f2aa1eaa700 1 -- 172.21.15.16:0/4087591058 wait complete. 2024-05-08T23:12:24.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:24 smithi016 ceph-mon[42541]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:24.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:24 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:24.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:24 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:24.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T23:12:24.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.181+0000 7f97c60bb700 1 Processor -- start 2024-05-08T23:12:24.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.181+0000 7f97c60bb700 1 -- start start 2024-05-08T23:12:24.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97c60bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c0105780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:24.755 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97c60bb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0105d50 con 0x7f97c0105360 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c0105780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c0105780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35362/0 (socket says 172.21.15.16:35362) 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97bf7fe700 1 -- 172.21.15.16:0/658541939 learned_addr learned my addr 172.21.15.16:0/658541939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97bf7fe700 1 -- 172.21.15.16:0/658541939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0106560 con 0x7f97c0105360 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.182+0000 7f97bf7fe700 1 --2- 172.21.15.16:0/658541939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c0105780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098a0 tx=0x7f97a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1709f0c03f12726e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97be7fc700 1 -- 172.21.15.16:0/658541939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a8004160 con 0x7f97c0105360 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97be7fc700 1 -- 172.21.15.16:0/658541939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f97a80042c0 con 0x7f97c0105360 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97be7fc700 1 -- 172.21.15.16:0/658541939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a8011940 con 0x7f97c0105360 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 -- 172.21.15.16:0/658541939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 msgr2=0x7f97c0105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 --2- 172.21.15.16:0/658541939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c0105780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098a0 tx=0x7f97a8009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 -- 172.21.15.16:0/658541939 shutdown_connections 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 --2- 172.21.15.16:0/658541939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c0105780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 -- 172.21.15.16:0/658541939 >> 172.21.15.16:0/658541939 conn(0x7f97c0100920 msgr2=0x7f97c0102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:24.756 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 -- 172.21.15.16:0/658541939 shutdown_connections 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.183+0000 7f97c60bb700 1 -- 172.21.15.16:0/658541939 wait complete. 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.184+0000 7f97c60bb700 1 Processor -- start 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.184+0000 7f97c60bb700 1 -- start start 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.184+0000 7f97c60bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.184+0000 7f97c60bb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0079c60 con 0x7f97c0105360 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35376/0 (socket says 172.21.15.16:35376) 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bf7fe700 1 -- 172.21.15.16:0/3190787628 learned_addr learned my addr 172.21.15.16:0/3190787628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bf7fe700 1 -- 172.21.15.16:0/3190787628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009550 con 0x7f97c0105360 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bf7fe700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c019ce50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f97a8003f20 tx=0x7f97a8011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a80036a0 con 0x7f97c0105360 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f97a8003c70 con 0x7f97c0105360 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c0079e60 con 0x7f97c0105360 2024-05-08T23:12:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97a801be80 con 0x7f97c0105360 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.185+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c019d560 con 0x7f97c0105360 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.186+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f97a80234e0 con 0x7f97c0105360 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.186+0000 7f97bcff9700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f97ac038750 0x7f97ac03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.186+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f97a804c7b0 con 0x7f97c0105360 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.186+0000 7f97beffd700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f97ac038750 0x7f97ac03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.186+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c01045f0 con 0x7f97c0105360 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.187+0000 7f97beffd700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f97ac038750 0x7f97ac03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.192+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97a8023cb0 con 0x7f97c0105360 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.406+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f97c0101b90 con 0x7f97ac038750 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.406+0000 7f97bcff9700 1 -- 172.21.15.16:0/3190787628 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f97c0101b90 con 0x7f97ac038750 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.408+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f97ac038750 msgr2=0x7f97ac03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.408+0000 7f97c60bb700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f97ac038750 0x7f97ac03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:24.758 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.408+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 msgr2=0x7f97c019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.408+0000 7f97c60bb700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c019ce50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f97a8003f20 tx=0x7f97a8011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.409+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 shutdown_connections 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.409+0000 7f97c60bb700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f97ac038750 0x7f97ac03ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.409+0000 7f97c60bb700 1 --2- 172.21.15.16:0/3190787628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97c0105360 0x7f97c019ce50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.409+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 >> 172.21.15.16:0/3190787628 conn(0x7f97c0100920 msgr2=0x7f97c0101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.409+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 shutdown_connections 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:24.409+0000 7f97c60bb700 1 -- 172.21.15.16:0/3190787628 wait complete. 2024-05-08T23:12:24.759 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-05-08T23:12:25.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:25 smithi016 ceph-mon[42541]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: Generating public/private rsa key pair. 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: Your identification has been saved in /tmp/tmpgy5u4up0/key. 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: Your public key has been saved in /tmp/tmpgy5u4up0/key.pub. 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: The key fingerprint is: 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: SHA256:mhoRQfmj+KZtZtQKjeGvRphueU4rYPGv9t3hph00RAs ceph-378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: The key's randomart image is: 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: +---[RSA 3072]----+ 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: | .o. E . | 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: | .. o . | 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: | .. o | 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: | .. .o . | 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: | +o=.o .So | 2024-05-08T23:12:25.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: |+.*.+..o. . | 2024-05-08T23:12:25.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: |+..*o.o o | 2024-05-08T23:12:25.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: | =o+X+. +.o | 2024-05-08T23:12:25.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: |..B@=. oo+ | 2024-05-08T23:12:25.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: +----[SHA256]-----+ 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a3b26a700 1 Processor -- start 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a3b26a700 1 -- start start 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a3b26a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a341093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a3b26a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34079df0 con 0x7f0a34106fb0 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a39006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a341093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a39006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a341093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35382/0 (socket says 172.21.15.16:35382) 2024-05-08T23:12:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.111+0000 7f0a39006700 1 -- 172.21.15.16:0/93705061 learned_addr learned my addr 172.21.15.16:0/93705061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a39006700 1 -- 172.21.15.16:0/93705061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34079f30 con 0x7f0a34106fb0 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a39006700 1 --2- 172.21.15.16:0/93705061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a341093a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a300098a0 tx=0x7f0a30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a55ddb1ed7909b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a2bfff700 1 -- 172.21.15.16:0/93705061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a30004160 con 0x7f0a34106fb0 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a2bfff700 1 -- 172.21.15.16:0/93705061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0a300042c0 con 0x7f0a34106fb0 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a2bfff700 1 -- 172.21.15.16:0/93705061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a30011940 con 0x7f0a34106fb0 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a3b26a700 1 -- 172.21.15.16:0/93705061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 msgr2=0x7f0a341093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.112+0000 7f0a3b26a700 1 --2- 172.21.15.16:0/93705061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a341093a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a300098a0 tx=0x7f0a30009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 -- 172.21.15.16:0/93705061 shutdown_connections 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 --2- 172.21.15.16:0/93705061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a341093a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 -- 172.21.15.16:0/93705061 >> 172.21.15.16:0/93705061 conn(0x7f0a34100920 msgr2=0x7f0a34102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 -- 172.21.15.16:0/93705061 shutdown_connections 2024-05-08T23:12:25.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 -- 172.21.15.16:0/93705061 wait complete. 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 Processor -- start 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.113+0000 7f0a3b26a700 1 -- start start 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a3b26a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a34106970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a3b26a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34104f60 con 0x7f0a34106fb0 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a39006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a34106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a39006700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a34106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35398/0 (socket says 172.21.15.16:35398) 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a39006700 1 -- 172.21.15.16:0/3038634228 learned_addr learned my addr 172.21.15.16:0/3038634228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a39006700 1 -- 172.21.15.16:0/3038634228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30009550 con 0x7f0a34106fb0 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a39006700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a34106970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a30003f20 tx=0x7f0a30011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a300037d0 con 0x7f0a34106fb0 2024-05-08T23:12:25.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0a3001b440 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a30023590 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a34105160 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.114+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a34105600 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.115+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f0a3001b5b0 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.116+0000 7f0a2a7fc700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f0a20038700 0x7f0a2003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.115+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a34066f40 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.116+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a3004c620 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.116+0000 7f0a38805700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f0a20038700 0x7f0a2003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.116+0000 7f0a38805700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f0a20038700 0x7f0a2003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006fd0 tx=0x7f0a24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.121+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0a3001bb00 con 0x7f0a34106fb0 2024-05-08T23:12:25.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.334+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a341059c0 con 0x7f0a20038700 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.578+0000 7f0a2a7fc700 1 -- 172.21.15.16:0/3038634228 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0a341059c0 con 0x7f0a20038700 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f0a20038700 msgr2=0x7f0a2003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f0a20038700 0x7f0a2003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006fd0 tx=0x7f0a24006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 msgr2=0x7f0a34106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a34106970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a30003f20 tx=0x7f0a30011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 shutdown_connections 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f0a20038700 0x7f0a2003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 --2- 172.21.15.16:0/3038634228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a34106fb0 0x7f0a34106970 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 >> 172.21.15.16:0/3038634228 conn(0x7f0a34100920 msgr2=0x7f0a341014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 shutdown_connections 2024-05-08T23:12:25.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:25.581+0000 7f0a3b26a700 1 -- 172.21.15.16:0/3038634228 wait complete. 2024-05-08T23:12:26.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:26 smithi016 ceph-mon[42541]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:26.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:26 smithi016 ceph-mon[42541]: Generating ssh key... 2024-05-08T23:12:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:26 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:26 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb909f700 1 Processor -- start 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb909f700 1 -- start start 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb909f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb909f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4079e50 con 0x7f4eb4106fd0 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35402/0 (socket says 172.21.15.16:35402) 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.289+0000 7f4eb2d9d700 1 -- 172.21.15.16:0/447372424 learned_addr learned my addr 172.21.15.16:0/447372424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:26.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb2d9d700 1 -- 172.21.15.16:0/447372424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb4109900 con 0x7f4eb4106fd0 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb2d9d700 1 --2- 172.21.15.16:0/447372424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41093c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40098a0 tx=0x7f4ea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe36c42b8250dbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb1d9b700 1 -- 172.21.15.16:0/447372424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea4004160 con 0x7f4eb4106fd0 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb1d9b700 1 -- 172.21.15.16:0/447372424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4ea40042c0 con 0x7f4eb4106fd0 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb1d9b700 1 -- 172.21.15.16:0/447372424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea4011940 con 0x7f4eb4106fd0 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb909f700 1 -- 172.21.15.16:0/447372424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 msgr2=0x7f4eb41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.290+0000 7f4eb909f700 1 --2- 172.21.15.16:0/447372424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41093c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40098a0 tx=0x7f4ea4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.291+0000 7f4eb909f700 1 -- 172.21.15.16:0/447372424 shutdown_connections 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.291+0000 7f4eb909f700 1 --2- 172.21.15.16:0/447372424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41093c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.291+0000 7f4eb909f700 1 -- 172.21.15.16:0/447372424 >> 172.21.15.16:0/447372424 conn(0x7f4eb41009c0 msgr2=0x7f4eb4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.291+0000 7f4eb909f700 1 -- 172.21.15.16:0/447372424 shutdown_connections 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.291+0000 7f4eb909f700 1 -- 172.21.15.16:0/447372424 wait complete. 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.291+0000 7f4eb909f700 1 Processor -- start 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb909f700 1 -- start start 2024-05-08T23:12:26.873 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb909f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb909f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4104f80 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35414/0 (socket says 172.21.15.16:35414) 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb2d9d700 1 -- 172.21.15.16:0/2646262949 learned_addr learned my addr 172.21.15.16:0/2646262949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.292+0000 7f4eb2d9d700 1 -- 172.21.15.16:0/2646262949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4009550 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.293+0000 7f4eb2d9d700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41069e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003f20 tx=0x7f4ea4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.293+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea40037d0 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.293+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4ea401b440 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.293+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb4105180 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.293+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea4023590 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.293+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb4105620 con 0x7f4eb4106fd0 2024-05-08T23:12:26.874 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.294+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f4ea4023800 con 0x7f4eb4106fd0 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.294+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb4066f40 con 0x7f4eb4106fd0 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.294+0000 7f4eabfff700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f4ea0038750 0x7f4ea003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.294+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ea404c650 con 0x7f4eb4106fd0 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.294+0000 7f4eb259c700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f4ea0038750 0x7f4ea003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.295+0000 7f4eb259c700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f4ea0038750 0x7f4ea003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c006fd0 tx=0x7f4e9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.299+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4ea4023ab0 con 0x7f4eb4106fd0 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.512+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eb4105a20 con 0x7f4ea0038750 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.512+0000 7f4eabfff700 1 -- 172.21.15.16:0/2646262949 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4eb4105a20 con 0x7f4ea0038750 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f4ea0038750 msgr2=0x7f4ea003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f4ea0038750 0x7f4ea003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c006fd0 tx=0x7f4e9c006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 msgr2=0x7f4eb41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41069e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4003f20 tx=0x7f4ea4011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 shutdown_connections 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f4ea0038750 0x7f4ea003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:26.875 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 --2- 172.21.15.16:0/2646262949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4eb4106fd0 0x7f4eb41069e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:26.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 >> 172.21.15.16:0/2646262949 conn(0x7f4eb41009c0 msgr2=0x7f4eb4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:26.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 shutdown_connections 2024-05-08T23:12:26.876 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:26.515+0000 7f4eb909f700 1 -- 172.21.15.16:0/2646262949 wait complete. 2024-05-08T23:12:26.876 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:12:26.876 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T23:12:26.876 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-05-08T23:12:27.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:27 smithi016 ceph-mon[42541]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:28 smithi016 ceph-mon[42541]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:29.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:29 smithi016 ceph-mon[42541]: Deploying cephadm binary to smithi016 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.233+0000 7ff256a7b700 1 Processor -- start 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.233+0000 7ff256a7b700 1 -- start start 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.233+0000 7ff256a7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff2501051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.233+0000 7ff256a7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff250079e50 con 0x7ff250104da0 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.233+0000 7ff254817700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff2501051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.233+0000 7ff254817700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff2501051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50746/0 (socket says 172.21.15.16:50746) 2024-05-08T23:12:30.095 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.234+0000 7ff254817700 1 -- 172.21.15.16:0/1787635594 learned_addr learned my addr 172.21.15.16:0/1787635594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.234+0000 7ff254817700 1 -- 172.21.15.16:0/1787635594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250105700 con 0x7ff250104da0 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.234+0000 7ff254817700 1 --2- 172.21.15.16:0/1787635594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff2501051c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff2440098a0 tx=0x7ff244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c7fb57e1707daf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.234+0000 7ff24f7fe700 1 -- 172.21.15.16:0/1787635594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff244004160 con 0x7ff250104da0 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.234+0000 7ff24f7fe700 1 -- 172.21.15.16:0/1787635594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff2440042c0 con 0x7ff250104da0 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.234+0000 7ff24f7fe700 1 -- 172.21.15.16:0/1787635594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff244011940 con 0x7ff250104da0 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 -- 172.21.15.16:0/1787635594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 msgr2=0x7ff2501051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 --2- 172.21.15.16:0/1787635594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff2501051c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff2440098a0 tx=0x7ff244009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 -- 172.21.15.16:0/1787635594 shutdown_connections 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 --2- 172.21.15.16:0/1787635594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff2501051c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 -- 172.21.15.16:0/1787635594 >> 172.21.15.16:0/1787635594 conn(0x7ff250100960 msgr2=0x7ff250102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 -- 172.21.15.16:0/1787635594 shutdown_connections 2024-05-08T23:12:30.096 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.235+0000 7ff256a7b700 1 -- 172.21.15.16:0/1787635594 wait complete. 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff256a7b700 1 Processor -- start 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff256a7b700 1 -- start start 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff256a7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff25019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff256a7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25019c8b0 con 0x7ff250104da0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff254817700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff25019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff254817700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff25019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50754/0 (socket says 172.21.15.16:50754) 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff254817700 1 -- 172.21.15.16:0/3136333229 learned_addr learned my addr 172.21.15.16:0/3136333229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.236+0000 7ff254817700 1 -- 172.21.15.16:0/3136333229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff244009550 con 0x7ff250104da0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.237+0000 7ff254817700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff25019c370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff244006af0 tx=0x7ff244011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.237+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff244003bf0 con 0x7ff250104da0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.237+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff24401b8b0 con 0x7ff250104da0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.237+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25019cab0 con 0x7ff250104da0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.237+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff244023830 con 0x7ff250104da0 2024-05-08T23:12:30.097 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.237+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25019d610 con 0x7ff250104da0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.238+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7ff244023aa0 con 0x7ff250104da0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.238+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff250104650 con 0x7ff250104da0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.238+0000 7ff24dffb700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7ff240038700 0x7ff24003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.238+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff24404ce10 con 0x7ff250104da0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.238+0000 7ff24ffff700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7ff240038700 0x7ff24003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.239+0000 7ff24ffff700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7ff240038700 0x7ff24003abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006fd0 tx=0x7ff23c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.243+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff24402bb40 con 0x7ff250104da0 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:27.462+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) v1 -- 0x7ff25019e110 con 0x7ff240038700 2024-05-08T23:12:30.098 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.781+0000 7ff24dffb700 1 -- 172.21.15.16:0/3136333229 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ff25019e110 con 0x7ff240038700 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7ff240038700 msgr2=0x7ff24003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7ff240038700 0x7ff24003abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff23c006fd0 tx=0x7ff23c006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 msgr2=0x7ff25019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff25019c370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff244006af0 tx=0x7ff244011a50 comp rx=0 tx=0).stop 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 shutdown_connections 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7ff240038700 0x7ff24003abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 --2- 172.21.15.16:0/3136333229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250104da0 0x7ff25019c370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 >> 172.21.15.16:0/3136333229 conn(0x7ff250100960 msgr2=0x7ff250193160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 shutdown_connections 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:29.784+0000 7ff256a7b700 1 -- 172.21.15.16:0/3136333229 wait complete. 2024-05-08T23:12:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-05-08T23:12:30.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:30 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:30.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:30 smithi016 ceph-mon[42541]: Added host smithi016 2024-05-08T23:12:30.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:30 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:30.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:30 smithi016 ceph-mon[42541]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:30.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:30 smithi016 ceph-mon[42541]: Saving service mon spec with placement count:5 2024-05-08T23:12:30.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:30 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:31.045 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.448+0000 7f535e462700 1 Processor -- start 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.448+0000 7f535e462700 1 -- start start 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.449+0000 7f535e462700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f53581068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.449+0000 7f535e462700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358106e90 con 0x7f53581064a0 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.449+0000 7f5357fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f53581068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.449+0000 7f5357fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f53581068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50758/0 (socket says 172.21.15.16:50758) 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.449+0000 7f5357fff700 1 -- 172.21.15.16:0/745634962 learned_addr learned my addr 172.21.15.16:0/745634962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.449+0000 7f5357fff700 1 -- 172.21.15.16:0/745634962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53581076a0 con 0x7f53581064a0 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.450+0000 7f5357fff700 1 --2- 172.21.15.16:0/745634962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f53581068c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53480098a0 tx=0x7f5348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e719e4f68daa11f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.450+0000 7f5356ffd700 1 -- 172.21.15.16:0/745634962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5348004160 con 0x7f53581064a0 2024-05-08T23:12:31.046 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.450+0000 7f5356ffd700 1 -- 172.21.15.16:0/745634962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f53480042c0 con 0x7f53581064a0 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.450+0000 7f5356ffd700 1 -- 172.21.15.16:0/745634962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5348011940 con 0x7f53581064a0 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 -- 172.21.15.16:0/745634962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 msgr2=0x7f53581068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 --2- 172.21.15.16:0/745634962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f53581068c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53480098a0 tx=0x7f5348009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 -- 172.21.15.16:0/745634962 shutdown_connections 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 --2- 172.21.15.16:0/745634962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f53581068c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 -- 172.21.15.16:0/745634962 >> 172.21.15.16:0/745634962 conn(0x7f5358101a40 msgr2=0x7f5358103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 -- 172.21.15.16:0/745634962 shutdown_connections 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 -- 172.21.15.16:0/745634962 wait complete. 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 Processor -- start 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.451+0000 7f535e462700 1 -- start start 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f535e462700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f535819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f535e462700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535819a6f0 con 0x7f53581064a0 2024-05-08T23:12:31.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f5357fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f535819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f5357fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f535819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50766/0 (socket says 172.21.15.16:50766) 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f5357fff700 1 -- 172.21.15.16:0/409483722 learned_addr learned my addr 172.21.15.16:0/409483722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f5357fff700 1 -- 172.21.15.16:0/409483722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348009550 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f5357fff700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f535819a1b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5348003f20 tx=0x7f5348011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53480036a0 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f534801b440 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5348023400 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.452+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535819a8f0 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.453+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535819b450 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.453+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f5348023670 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.454+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535819ad90 con 0x7f53581064a0 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.454+0000 7f53557fa700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5340038750 0x7f534003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.454+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f534804c8c0 con 0x7f53581064a0 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.454+0000 7f53577fe700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5340038750 0x7f534003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.454+0000 7f53577fe700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5340038750 0x7f534003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f534c006fd0 tx=0x7f534c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.459+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f535819ad90 con 0x7f53581064a0 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.690+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f535819ad90 con 0x7f5340038750 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.696+0000 7f53557fa700 1 -- 172.21.15.16:0/409483722 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f535819ad90 con 0x7f5340038750 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5340038750 msgr2=0x7f534003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5340038750 0x7f534003ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f534c006fd0 tx=0x7f534c006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 msgr2=0x7f535819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f535819a1b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5348003f20 tx=0x7f5348011f00 comp rx=0 tx=0).stop 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 shutdown_connections 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5340038750 0x7f534003ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 --2- 172.21.15.16:0/409483722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53581064a0 0x7f535819a1b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 >> 172.21.15.16:0/409483722 conn(0x7f5358101a40 msgr2=0x7f53581024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 shutdown_connections 2024-05-08T23:12:31.050 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:30.698+0000 7f535e462700 1 -- 172.21.15.16:0/409483722 wait complete. 2024-05-08T23:12:31.050 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-05-08T23:12:31.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T23:12:31.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd47b2700 1 Processor -- start 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd47b2700 1 -- start start 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd47b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd47b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc079df0 con 0x7fafcc107010 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd254e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd254e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50782/0 (socket says 172.21.15.16:50782) 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd254e700 1 -- 172.21.15.16:0/2373038987 learned_addr learned my addr 172.21.15.16:0/2373038987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd254e700 1 -- 172.21.15.16:0/2373038987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc079f30 con 0x7fafcc107010 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd254e700 1 --2- 172.21.15.16:0/2373038987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc109400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fafc80098a0 tx=0x7fafc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e164f4c9d0759814 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd154c700 1 -- 172.21.15.16:0/2373038987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafc8004160 con 0x7fafcc107010 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd154c700 1 -- 172.21.15.16:0/2373038987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc80042c0 con 0x7fafcc107010 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.404+0000 7fafd154c700 1 -- 172.21.15.16:0/2373038987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafc80118e0 con 0x7fafcc107010 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 -- 172.21.15.16:0/2373038987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 msgr2=0x7fafcc109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 --2- 172.21.15.16:0/2373038987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc109400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fafc80098a0 tx=0x7fafc8009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 -- 172.21.15.16:0/2373038987 shutdown_connections 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 --2- 172.21.15.16:0/2373038987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc109400 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 -- 172.21.15.16:0/2373038987 >> 172.21.15.16:0/2373038987 conn(0x7fafcc100960 msgr2=0x7fafcc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 -- 172.21.15.16:0/2373038987 shutdown_connections 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 -- 172.21.15.16:0/2373038987 wait complete. 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.405+0000 7fafd47b2700 1 Processor -- start 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd47b2700 1 -- start start 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd47b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc106a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd47b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc104fc0 con 0x7fafcc107010 2024-05-08T23:12:31.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd254e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd254e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50788/0 (socket says 172.21.15.16:50788) 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd254e700 1 -- 172.21.15.16:0/662858823 learned_addr learned my addr 172.21.15.16:0/662858823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd254e700 1 -- 172.21.15.16:0/662858823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8009550 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.406+0000 7fafd254e700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc106a20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fafc8003f70 tx=0x7fafc80119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.407+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafc80036a0 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.407+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafc801b440 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.407+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc1051c0 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.407+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafc8023400 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.407+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc105660 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.408+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc066f40 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.408+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fafc801b5b0 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.408+0000 7fafc37fe700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fafb8038750 0x7fafb803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.408+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fafc804c620 con 0x7fafcc107010 2024-05-08T23:12:31.967 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.408+0000 7fafd1d4d700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fafb8038750 0x7fafb803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.408+0000 7fafd1d4d700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fafb8038750 0x7fafb803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006fd0 tx=0x7fafbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.413+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fafc801ba90 con 0x7fafcc107010 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.644+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fafcc1059e0 con 0x7fafb8038750 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.650+0000 7fafc37fe700 1 -- 172.21.15.16:0/662858823 <== mgr.14120 v2:172.21.15.16:6800/1569877251 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fafcc1059e0 con 0x7fafb8038750 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fafb8038750 msgr2=0x7fafb803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fafb8038750 0x7fafb803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006fd0 tx=0x7fafbc006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 msgr2=0x7fafcc106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc106a20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fafc8003f70 tx=0x7fafc80119a0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 shutdown_connections 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fafb8038750 0x7fafb803ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 --2- 172.21.15.16:0/662858823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc107010 0x7fafcc106a20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 >> 172.21.15.16:0/662858823 conn(0x7fafcc100960 msgr2=0x7fafcc1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 shutdown_connections 2024-05-08T23:12:31.968 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:31.652+0000 7fafd47b2700 1 -- 172.21.15.16:0/662858823 wait complete. 2024-05-08T23:12:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:32 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:32 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:32 smithi016 ceph-mon[42541]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:32 smithi016 ceph-mon[42541]: Saving service mgr spec with placement count:2 2024-05-08T23:12:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:32 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.348+0000 7f948a302700 1 Processor -- start 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.348+0000 7f948a302700 1 -- start start 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.348+0000 7f948a302700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f9484105800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.348+0000 7f948a302700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9484105dd0 con 0x7f94841053e0 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.348+0000 7f9483fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f9484105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9483fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f9484105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50800/0 (socket says 172.21.15.16:50800) 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9483fff700 1 -- 172.21.15.16:0/1313766433 learned_addr learned my addr 172.21.15.16:0/1313766433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:32.946 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9483fff700 1 -- 172.21.15.16:0/1313766433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94841065e0 con 0x7f94841053e0 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9483fff700 1 --2- 172.21.15.16:0/1313766433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f9484105800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f94740098a0 tx=0x7f9474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aefec2acf360bc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9482ffd700 1 -- 172.21.15.16:0/1313766433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9474004160 con 0x7f94841053e0 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9482ffd700 1 -- 172.21.15.16:0/1313766433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94740042c0 con 0x7f94841053e0 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.349+0000 7f9482ffd700 1 -- 172.21.15.16:0/1313766433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94740118e0 con 0x7f94841053e0 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 -- 172.21.15.16:0/1313766433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 msgr2=0x7f9484105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 --2- 172.21.15.16:0/1313766433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f9484105800 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f94740098a0 tx=0x7f9474009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 -- 172.21.15.16:0/1313766433 shutdown_connections 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 --2- 172.21.15.16:0/1313766433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f9484105800 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 -- 172.21.15.16:0/1313766433 >> 172.21.15.16:0/1313766433 conn(0x7f94841009c0 msgr2=0x7f9484102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 -- 172.21.15.16:0/1313766433 shutdown_connections 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.350+0000 7f948a302700 1 -- 172.21.15.16:0/1313766433 wait complete. 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f948a302700 1 Processor -- start 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f948a302700 1 -- start start 2024-05-08T23:12:32.947 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f948a302700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f948419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f948a302700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419a890 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f9483fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f948419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f9483fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f948419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50806/0 (socket says 172.21.15.16:50806) 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f9483fff700 1 -- 172.21.15.16:0/826295647 learned_addr learned my addr 172.21.15.16:0/826295647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.351+0000 7f9483fff700 1 -- 172.21.15.16:0/826295647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009550 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.352+0000 7f9483fff700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f948419a350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9474003f70 tx=0x7f94740119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.352+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94740037d0 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.352+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f947401b440 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.352+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948419aa90 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.352+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94740193d0 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.352+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948419b5f0 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.353+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f94740195c0 con 0x7f94841053e0 2024-05-08T23:12:32.948 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.353+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948419af30 con 0x7f94841053e0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.353+0000 7f94817fa700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f946c038750 0x7f946c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.353+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f947404c4a0 con 0x7f94841053e0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.353+0000 7f94837fe700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f946c038750 0x7f946c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.354+0000 7f94837fe700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f946c038750 0x7f946c03ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9478006fd0 tx=0x7f9478006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.358+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f948419af30 con 0x7f94841053e0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.572+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f948419af30 con 0x7f94841053e0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.575+0000 7f94817fa700 1 -- 172.21.15.16:0/826295647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f948419af30 con 0x7f94841053e0 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f946c038750 msgr2=0x7f946c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f946c038750 0x7f946c03ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9478006fd0 tx=0x7f9478006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 msgr2=0x7f948419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f948419a350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9474003f70 tx=0x7f94740119a0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 shutdown_connections 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f946c038750 0x7f946c03ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.949 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 --2- 172.21.15.16:0/826295647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94841053e0 0x7f948419a350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:32.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 >> 172.21.15.16:0/826295647 conn(0x7f94841009c0 msgr2=0x7f94841018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:32.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 shutdown_connections 2024-05-08T23:12:32.950 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:32.578+0000 7f948a302700 1 -- 172.21.15.16:0/826295647 wait complete. 2024-05-08T23:12:33.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:33 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/826295647' entity='client.admin' 2024-05-08T23:12:33.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:33 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:33.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:33 smithi016 ceph-mon[42541]: from='mgr.14120 172.21.15.16:0/1661830402' entity='mgr.x' 2024-05-08T23:12:33.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:33 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/171730667' entity='client.admin' 2024-05-08T23:12:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.319+0000 7facc00f7700 1 Processor -- start 2024-05-08T23:12:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.319+0000 7facc00f7700 1 -- start start 2024-05-08T23:12:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.319+0000 7facc00f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb81051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.319+0000 7facc00f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb8079e50 con 0x7facb8104da0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.320+0000 7facbde93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb81051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.320+0000 7facbde93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb81051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50812/0 (socket says 172.21.15.16:50812) 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.320+0000 7facbde93700 1 -- 172.21.15.16:0/2697056901 learned_addr learned my addr 172.21.15.16:0/2697056901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.320+0000 7facbde93700 1 -- 172.21.15.16:0/2697056901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8105700 con 0x7facb8104da0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.321+0000 7facbde93700 1 --2- 172.21.15.16:0/2697056901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb81051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facb40098a0 tx=0x7facb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bbce9fdb091249 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.321+0000 7facbce91700 1 -- 172.21.15.16:0/2697056901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facb4004160 con 0x7facb8104da0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.321+0000 7facbce91700 1 -- 172.21.15.16:0/2697056901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facb40042c0 con 0x7facb8104da0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.321+0000 7facbce91700 1 -- 172.21.15.16:0/2697056901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facb40118e0 con 0x7facb8104da0 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 -- 172.21.15.16:0/2697056901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 msgr2=0x7facb81051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 --2- 172.21.15.16:0/2697056901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb81051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facb40098a0 tx=0x7facb4009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 -- 172.21.15.16:0/2697056901 shutdown_connections 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 --2- 172.21.15.16:0/2697056901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb81051c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 -- 172.21.15.16:0/2697056901 >> 172.21.15.16:0/2697056901 conn(0x7facb8100960 msgr2=0x7facb8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 -- 172.21.15.16:0/2697056901 shutdown_connections 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.322+0000 7facc00f7700 1 -- 172.21.15.16:0/2697056901 wait complete. 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.323+0000 7facc00f7700 1 Processor -- start 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.323+0000 7facc00f7700 1 -- start start 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.323+0000 7facc00f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.323+0000 7facc00f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb819c9e0 con 0x7facb8104da0 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facbde93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facbde93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50816/0 (socket says 172.21.15.16:50816) 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facbde93700 1 -- 172.21.15.16:0/171730667 learned_addr learned my addr 172.21.15.16:0/171730667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facbde93700 1 -- 172.21.15.16:0/171730667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb4009550 con 0x7facb8104da0 2024-05-08T23:12:33.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facbde93700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb819c4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facb4003ed0 tx=0x7facb4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facb40036a0 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facb401b440 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb819cbe0 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facb40193d0 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.324+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb819d740 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.325+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7facb4019640 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.326+0000 7facaeffd700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7faca4038700 0x7faca403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.326+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb8104650 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.326+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7facb404c650 con 0x7facb8104da0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.326+0000 7facbd692700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7faca4038700 0x7faca403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.326+0000 7facbd692700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7faca4038700 0x7faca403abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faca8006fd0 tx=0x7faca8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:33.871 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.331+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7facb4018940 con 0x7facb8104da0 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.547+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7facb8066f40 con 0x7facb8104da0 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.550+0000 7facaeffd700 1 -- 172.21.15.16:0/171730667 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7facb40223a0 con 0x7facb8104da0 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7faca4038700 msgr2=0x7faca403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7faca4038700 0x7faca403abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faca8006fd0 tx=0x7faca8006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 msgr2=0x7facb819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb819c4a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facb4003ed0 tx=0x7facb4011e80 comp rx=0 tx=0).stop 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 shutdown_connections 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7faca4038700 0x7faca403abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 --2- 172.21.15.16:0/171730667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facb8104da0 0x7facb819c4a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 >> 172.21.15.16:0/171730667 conn(0x7facb8100960 msgr2=0x7facb8193160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 shutdown_connections 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:33.554+0000 7facc00f7700 1 -- 172.21.15.16:0/171730667 wait complete. 2024-05-08T23:12:33.872 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-05-08T23:12:35.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:34 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4269210545' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T23:12:35.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: ignoring --setuser ceph since I am not root 2024-05-08T23:12:35.110 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: ignoring --setgroup ceph since I am not root 2024-05-08T23:12:35.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.475+0000 7fe2fb0cb700 1 Processor -- start 2024-05-08T23:12:35.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.475+0000 7fe2fb0cb700 1 -- start start 2024-05-08T23:12:35.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.475+0000 7fe2fb0cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f4105150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:35.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.475+0000 7fe2fb0cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4079e50 con 0x7fe2f4104d30 2024-05-08T23:12:35.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f8e67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f4105150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:35.204 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f8e67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f4105150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50820/0 (socket says 172.21.15.16:50820) 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f8e67700 1 -- 172.21.15.16:0/273424795 learned_addr learned my addr 172.21.15.16:0/273424795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f8e67700 1 -- 172.21.15.16:0/273424795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4105690 con 0x7fe2f4104d30 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f8e67700 1 --2- 172.21.15.16:0/273424795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f4105150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80098a0 tx=0x7fe2e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73568a389ff5700 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f37fe700 1 -- 172.21.15.16:0/273424795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2e8004160 con 0x7fe2f4104d30 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f37fe700 1 -- 172.21.15.16:0/273424795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2e80042c0 con 0x7fe2f4104d30 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.476+0000 7fe2f37fe700 1 -- 172.21.15.16:0/273424795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2e80118e0 con 0x7fe2f4104d30 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/273424795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 msgr2=0x7fe2f4105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 --2- 172.21.15.16:0/273424795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f4105150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80098a0 tx=0x7fe2e8009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/273424795 shutdown_connections 2024-05-08T23:12:35.205 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 --2- 172.21.15.16:0/273424795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f4105150 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/273424795 >> 172.21.15.16:0/273424795 conn(0x7fe2f4100970 msgr2=0x7fe2f4102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/273424795 shutdown_connections 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.477+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/273424795 wait complete. 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2fb0cb700 1 Processor -- start 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2fb0cb700 1 -- start start 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2fb0cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2fb0cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f419c9a0 con 0x7fe2f4104d30 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2f8e67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2f8e67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50828/0 (socket says 172.21.15.16:50828) 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2f8e67700 1 -- 172.21.15.16:0/4269210545 learned_addr learned my addr 172.21.15.16:0/4269210545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:35.206 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.478+0000 7fe2f8e67700 1 -- 172.21.15.16:0/4269210545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8009550 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.479+0000 7fe2f8e67700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f419c460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8003e30 tx=0x7fe2e8011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.479+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2e80036a0 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.479+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2e801b440 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.479+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f419cba0 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.479+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2e80193d0 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.479+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f419d700 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.480+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fe2e8019690 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.480+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f41045e0 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.480+0000 7fe2f1ffb700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fe2e4038700 0x7fe2e403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.480+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe2e804c760 con 0x7fe2f4104d30 2024-05-08T23:12:35.207 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.480+0000 7fe2f3fff700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fe2e4038700 0x7fe2e403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.481+0000 7fe2f3fff700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fe2e4038700 0x7fe2e403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006fd0 tx=0x7fe2e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.486+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe2e8018530 con 0x7fe2f4104d30 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.758+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fe2f4066f40 con 0x7fe2f4104d30 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.811+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fe2e801a750 con 0x7fe2f4104d30 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.813+0000 7fe2f1ffb700 1 -- 172.21.15.16:0/4269210545 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fe2e8018920 con 0x7fe2f4104d30 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fe2e4038700 msgr2=0x7fe2e403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fe2e4038700 0x7fe2e403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0006fd0 tx=0x7fe2e0006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 msgr2=0x7fe2f419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f419c460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8003e30 tx=0x7fe2e8011e80 comp rx=0 tx=0).stop 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 shutdown_connections 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7fe2e4038700 0x7fe2e403abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 --2- 172.21.15.16:0/4269210545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2f4104d30 0x7fe2f419c460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:35.208 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 >> 172.21.15.16:0/4269210545 conn(0x7fe2f4100970 msgr2=0x7fe2f4193190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:35.209 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 shutdown_connections 2024-05-08T23:12:35.209 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:34.816+0000 7fe2fb0cb700 1 -- 172.21.15.16:0/4269210545 wait complete. 2024-05-08T23:12:35.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:35.324+0000 7fe56f50e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:12:35.813 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:35.431+0000 7fe56f50e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:12:35.814 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:35.532+0000 7fe56f50e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:12:35.814 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:35.631+0000 7fe56f50e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:12:36.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:35 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4269210545' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T23:12:36.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:35 smithi016 ceph-mon[42541]: mgrmap e9: x(active, since 15s) 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.542+0000 7f5822e65700 1 Processor -- start 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.542+0000 7f5822e65700 1 -- start start 2024-05-08T23:12:36.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.543+0000 7f5822e65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c1092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.543+0000 7f5822e65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c079ed0 con 0x7f581c106ed0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.543+0000 7f5820c01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c1092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.543+0000 7f5820c01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c1092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50862/0 (socket says 172.21.15.16:50862) 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.543+0000 7f5820c01700 1 -- 172.21.15.16:0/89235114 learned_addr learned my addr 172.21.15.16:0/89235114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.543+0000 7f5820c01700 1 -- 172.21.15.16:0/89235114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c109800 con 0x7f581c106ed0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.544+0000 7f5820c01700 1 --2- 172.21.15.16:0/89235114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c1092c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58100098a0 tx=0x7f5810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba8e17b2a37347a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.544+0000 7f581b7fe700 1 -- 172.21.15.16:0/89235114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5810004160 con 0x7f581c106ed0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.544+0000 7f581b7fe700 1 -- 172.21.15.16:0/89235114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58100042c0 con 0x7f581c106ed0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.544+0000 7f581b7fe700 1 -- 172.21.15.16:0/89235114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58100118e0 con 0x7f581c106ed0 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 -- 172.21.15.16:0/89235114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 msgr2=0x7f581c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 --2- 172.21.15.16:0/89235114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c1092c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58100098a0 tx=0x7f5810009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 -- 172.21.15.16:0/89235114 shutdown_connections 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 --2- 172.21.15.16:0/89235114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c1092c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:36.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 -- 172.21.15.16:0/89235114 >> 172.21.15.16:0/89235114 conn(0x7f581c100920 msgr2=0x7f581c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 -- 172.21.15.16:0/89235114 shutdown_connections 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.545+0000 7f5822e65700 1 -- 172.21.15.16:0/89235114 wait complete. 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5822e65700 1 Processor -- start 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5822e65700 1 -- start start 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5822e65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5822e65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c19c9e0 con 0x7f581c106ed0 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5820c01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5820c01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50876/0 (socket says 172.21.15.16:50876) 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5820c01700 1 -- 172.21.15.16:0/515322849 learned_addr learned my addr 172.21.15.16:0/515322849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.546+0000 7f5820c01700 1 -- 172.21.15.16:0/515322849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810009550 con 0x7f581c106ed0 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.547+0000 7f5820c01700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c19c4a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5810003e30 tx=0x7f5810011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.547+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58100036a0 con 0x7f581c106ed0 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.547+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f581001b440 con 0x7f581c106ed0 2024-05-08T23:12:36.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.547+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c19cbe0 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.547+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58100193d0 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.547+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c19d740 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.548+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f5810019640 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.548+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c1045f0 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.548+0000 7f5819ffb700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 0x7f580c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.548+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f581004d4d0 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.548+0000 7f581bfff700 1 -- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 msgr2=0x7f580c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.548+0000 7f581bfff700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 0x7f580c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.553+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5810018db0 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.749+0000 7f581bfff700 1 -- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 msgr2=0x7f580c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.749+0000 7f581bfff700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 0x7f580c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.826+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f581c066f40 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.826+0000 7f5819ffb700 1 -- 172.21.15.16:0/515322849 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5810018350 con 0x7f581c106ed0 2024-05-08T23:12:36.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 msgr2=0x7f580c03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 0x7f580c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 msgr2=0x7f581c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c19c4a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5810003e30 tx=0x7f5810011e80 comp rx=0 tx=0).stop 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 shutdown_connections 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f580c038750 0x7f580c03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 --2- 172.21.15.16:0/515322849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581c106ed0 0x7f581c19c4a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 >> 172.21.15.16:0/515322849 conn(0x7f581c100920 msgr2=0x7f581c1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 shutdown_connections 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:35.829+0000 7f5822e65700 1 -- 172.21.15.16:0/515322849 wait complete. 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-05-08T23:12:36.191 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-05-08T23:12:36.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:36.370+0000 7fe56f50e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:12:37.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:36 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/515322849' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T23:12:37.062 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:36.661+0000 7fe56f50e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:12:37.062 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:36.759+0000 7fe56f50e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:12:37.062 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:36.882+0000 7fe56f50e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:12:37.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:37.062+0000 7fe56f50e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:12:37.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:37.207+0000 7fe56f50e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:12:37.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:37.304+0000 7fe56f50e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:12:38.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:37.724+0000 7fe56f50e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:12:38.489 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:38.165+0000 7fe56f50e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:12:38.489 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:38.384+0000 7fe56f50e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:12:38.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:38.489+0000 7fe56f50e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:12:39.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:39.361+0000 7fe56f50e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:12:39.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:39.471+0000 7fe56f50e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:12:40.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:39.641+0000 7fe56f50e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:12:40.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:40.362+0000 7fe56f50e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:12:40.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:40.568+0000 7fe56f50e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:12:41.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:40.789+0000 7fe56f50e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:12:42.113 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:41 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:41.847+0000 7fe56f50e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:12:42.486 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:42 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:42.168+0000 7fe56f50e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:12:42.486 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:42 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:42.268+0000 7fe56f50e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:12:42.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:12:42 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:12:42.486+0000 7fe56f50e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: Active manager daemon x restarted 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: Activating manager daemon x 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: mgrmap e10: x(active, starting, since 0.00370909s) 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:12:42.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:12:42.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:12:42.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: Manager daemon x is now available 2024-05-08T23:12:42.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:43.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:12:43.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:12:43.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:43 smithi016 ceph-mon[42541]: [08/May/2024:23:12:42] ENGINE Bus STARTING 2024-05-08T23:12:43.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:43.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:43 smithi016 ceph-mon[42541]: mgrmap e11: x(active, since 1.00649s) 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.561+0000 7f5e6fe7c700 1 Processor -- start 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.561+0000 7f5e6fe7c700 1 -- start start 2024-05-08T23:12:43.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6fe7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e68109300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6fe7c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68079e90 con 0x7f5e68106f10 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6dc18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e68109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6dc18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e68109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50006/0 (socket says 172.21.15.16:50006) 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6dc18700 1 -- 172.21.15.16:0/518499795 learned_addr learned my addr 172.21.15.16:0/518499795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6dc18700 1 -- 172.21.15.16:0/518499795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68109840 con 0x7f5e68106f10 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6dc18700 1 --2- 172.21.15.16:0/518499795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e68109300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e640098a0 tx=0x7f5e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9efb8cecaf30e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.562+0000 7f5e6cc16700 1 -- 172.21.15.16:0/518499795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e64004160 con 0x7f5e68106f10 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6cc16700 1 -- 172.21.15.16:0/518499795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e640042c0 con 0x7f5e68106f10 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6cc16700 1 -- 172.21.15.16:0/518499795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e640118e0 con 0x7f5e68106f10 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/518499795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 msgr2=0x7f5e68109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 --2- 172.21.15.16:0/518499795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e68109300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e640098a0 tx=0x7f5e64009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/518499795 shutdown_connections 2024-05-08T23:12:43.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 --2- 172.21.15.16:0/518499795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e68109300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/518499795 >> 172.21.15.16:0/518499795 conn(0x7f5e68100920 msgr2=0x7f5e68102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/518499795 shutdown_connections 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.563+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/518499795 wait complete. 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.564+0000 7f5e6fe7c700 1 Processor -- start 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.564+0000 7f5e6fe7c700 1 -- start start 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.564+0000 7f5e6fe7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e6819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.564+0000 7f5e6fe7c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6819a7e0 con 0x7f5e68106f10 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.564+0000 7f5e6dc18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e6819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e6dc18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e6819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50022/0 (socket says 172.21.15.16:50022) 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e6dc18700 1 -- 172.21.15.16:0/2368632505 learned_addr learned my addr 172.21.15.16:0/2368632505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e6dc18700 1 -- 172.21.15.16:0/2368632505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64009550 con 0x7f5e68106f10 2024-05-08T23:12:43.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e6dc18700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e6819a2a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e640046e0 tx=0x7f5e640119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e640037d0 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e6401b440 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/2368632505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6819a9e0 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e64023400 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.565+0000 7f5e6fe7c700 1 -- 172.21.15.16:0/2368632505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6819b540 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.566+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f5e64023630 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.567+0000 7f5e5effd700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.567+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 --> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e5403b370 con 0x7f5e540387a0 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.567+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5e6404c7b0 con 0x7f5e68106f10 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.567+0000 7f5e6d417700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 msgr2=0x7f5e5403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.567+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.768+0000 7f5e6d417700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 msgr2=0x7f5e5403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:36.768+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:12:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:37.169+0000 7f5e6d417700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 msgr2=0x7f5e5403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:37.169+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:37.970+0000 7f5e6d417700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 msgr2=0x7f5e5403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:37.970+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:39.571+0000 7f5e6d417700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 msgr2=0x7f5e5403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1569877251 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:39.571+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:42.501+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7f5e64023b10 con 0x7f5e68106f10 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:42.501+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 msgr2=0x7f5e5403ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:42.501+0000 7f5e5effd700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.503+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7f5e64016d40 con 0x7f5e68106f10 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.503+0000 7f5e5effd700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5e5403c270 0x7f5e5403e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.503+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e5403b370 con 0x7f5e5403c270 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.504+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5e5403c270 0x7f5e5403e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:43.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.505+0000 7f5e6d417700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5e5403c270 0x7f5e5403e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5e58003a10 tx=0x7f5e58009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.505+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f5e5403b370 con 0x7f5e5403c270 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.513+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5e50002600 con 0x7f5e5403c270 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.513+0000 7f5e5effd700 1 -- 172.21.15.16:0/2368632505 <== mgr.14152 v2:172.21.15.16:6800/781396085 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5e50002600 con 0x7f5e5403c270 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5e5403c270 msgr2=0x7f5e5403e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5e5403c270 0x7f5e5403e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5e58003a10 tx=0x7f5e58009270 comp rx=0 tx=0).stop 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 msgr2=0x7f5e6819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e6819a2a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e640046e0 tx=0x7f5e640119a0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 shutdown_connections 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5e5403c270 0x7f5e5403e660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e68106f10 0x7f5e6819a2a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 --2- 172.21.15.16:0/2368632505 >> [v2:172.21.15.16:6800/1569877251,v1:172.21.15.16:6801/1569877251] conn(0x7f5e540387a0 0x7f5e5403ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 >> 172.21.15.16:0/2368632505 conn(0x7f5e68100920 msgr2=0x7f5e68101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 shutdown_connections 2024-05-08T23:12:43.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:43.514+0000 7f5e5cff9700 1 -- 172.21.15.16:0/2368632505 wait complete. 2024-05-08T23:12:43.870 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-05-08T23:12:43.870 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-05-08T23:12:44.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:44 smithi016 ceph-mon[42541]: [08/May/2024:23:12:42] ENGINE Serving on http://172.21.15.16:8765 2024-05-08T23:12:44.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:44 smithi016 ceph-mon[42541]: [08/May/2024:23:12:43] ENGINE Serving on https://172.21.15.16:7150 2024-05-08T23:12:44.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:44 smithi016 ceph-mon[42541]: [08/May/2024:23:12:43] ENGINE Bus STARTED 2024-05-08T23:12:44.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:44 smithi016 ceph-mon[42541]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T23:12:44.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:44 smithi016 ceph-mon[42541]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T23:12:44.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.265+0000 7fda71bcd700 1 Processor -- start 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.265+0000 7fda71bcd700 1 -- start start 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda71bcd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda71bcd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c079e90 con 0x7fda6c104d00 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50110/0 (socket says 172.21.15.16:50110) 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda6b7fe700 1 -- 172.21.15.16:0/3068933958 learned_addr learned my addr 172.21.15.16:0/3068933958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda6b7fe700 1 -- 172.21.15.16:0/3068933958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c105660 con 0x7fda6c104d00 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.266+0000 7fda6b7fe700 1 --2- 172.21.15.16:0/3068933958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c105120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009e80 tx=0x7fda5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dc503cc6c077bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda6a7fc700 1 -- 172.21.15.16:0/3068933958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda5c00e070 con 0x7fda6c104d00 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda6a7fc700 1 -- 172.21.15.16:0/3068933958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda5c00b8c0 con 0x7fda6c104d00 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda6a7fc700 1 -- 172.21.15.16:0/3068933958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda5c004360 con 0x7fda6c104d00 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda71bcd700 1 -- 172.21.15.16:0/3068933958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 msgr2=0x7fda6c105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda71bcd700 1 --2- 172.21.15.16:0/3068933958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c105120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009e80 tx=0x7fda5c00b330 comp rx=0 tx=0).stop 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda71bcd700 1 -- 172.21.15.16:0/3068933958 shutdown_connections 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda71bcd700 1 --2- 172.21.15.16:0/3068933958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c105120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.267+0000 7fda71bcd700 1 -- 172.21.15.16:0/3068933958 >> 172.21.15.16:0/3068933958 conn(0x7fda6c100920 msgr2=0x7fda6c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.268+0000 7fda71bcd700 1 -- 172.21.15.16:0/3068933958 shutdown_connections 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.268+0000 7fda71bcd700 1 -- 172.21.15.16:0/3068933958 wait complete. 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.268+0000 7fda71bcd700 1 Processor -- start 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda71bcd700 1 -- start start 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda71bcd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:45.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda71bcd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c19c9c0 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50120/0 (socket says 172.21.15.16:50120) 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda6b7fe700 1 -- 172.21.15.16:0/1853391358 learned_addr learned my addr 172.21.15.16:0/1853391358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda6b7fe700 1 -- 172.21.15.16:0/1853391358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009b30 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.269+0000 7fda6b7fe700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c19c480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b7b0 tx=0x7fda5c003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.270+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda5c00e040 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.270+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda5c00bd70 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.270+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c19cbc0 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.270+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda5c01c590 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.270+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c19d720 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.271+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fda5c01c800 con 0x7fda6c104d00 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.271+0000 7fda68ff9700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda54038610 0x7fda5403aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:45.188 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.271+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fda5c04c480 con 0x7fda6c104d00 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.271+0000 7fda6affd700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda54038610 0x7fda5403aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.271+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c1045f0 con 0x7fda6c104d00 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.272+0000 7fda6affd700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda54038610 0x7fda5403aad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda60006fd0 tx=0x7fda60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.277+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fda5c017020 con 0x7fda6c104d00 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.508+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fda6c1067c0 con 0x7fda54038610 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.616+0000 7fda68ff9700 1 -- 172.21.15.16:0/1853391358 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fda6c1067c0 con 0x7fda54038610 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda54038610 msgr2=0x7fda5403aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda54038610 0x7fda5403aad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fda60006fd0 tx=0x7fda60006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 msgr2=0x7fda6c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c19c480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b7b0 tx=0x7fda5c003c50 comp rx=0 tx=0).stop 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 shutdown_connections 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda54038610 0x7fda5403aad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:45.189 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 --2- 172.21.15.16:0/1853391358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda6c104d00 0x7fda6c19c480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:45.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.618+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 >> 172.21.15.16:0/1853391358 conn(0x7fda6c100920 msgr2=0x7fda6c193160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:45.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.619+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 shutdown_connections 2024-05-08T23:12:45.190 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:44.619+0000 7fda71bcd700 1 -- 172.21.15.16:0/1853391358 wait complete. 2024-05-08T23:12:45.190 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-05-08T23:12:45.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:45 smithi016 ceph-mon[42541]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:45.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:45.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:45.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:45 smithi016 ceph-mon[42541]: mgrmap e12: x(active, since 2s) 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$plKHR.PfMClUJVQ6ZqukjOPTseU5CaBaws6dSo7uoUJlQOhIMohWK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715209966, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.543+0000 7fdfb401c700 1 Processor -- start 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.543+0000 7fdfb401c700 1 -- start start 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb401c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac106890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb401c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac106e60 con 0x7fdfac106470 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb1db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb1db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50132/0 (socket says 172.21.15.16:50132) 2024-05-08T23:12:46.388 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb1db8700 1 -- 172.21.15.16:0/3508727290 learned_addr learned my addr 172.21.15.16:0/3508727290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb1db8700 1 -- 172.21.15.16:0/3508727290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac107670 con 0x7fdfac106470 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb1db8700 1 --2- 172.21.15.16:0/3508727290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac106890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdfa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1849dfe8d32a087 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.544+0000 7fdfb0db6700 1 -- 172.21.15.16:0/3508727290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa0004160 con 0x7fdfac106470 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb0db6700 1 -- 172.21.15.16:0/3508727290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfa00042c0 con 0x7fdfac106470 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb0db6700 1 -- 172.21.15.16:0/3508727290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa00118e0 con 0x7fdfac106470 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 -- 172.21.15.16:0/3508727290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 msgr2=0x7fdfac106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 --2- 172.21.15.16:0/3508727290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac106890 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdfa0009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 -- 172.21.15.16:0/3508727290 shutdown_connections 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 --2- 172.21.15.16:0/3508727290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac106890 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 -- 172.21.15.16:0/3508727290 >> 172.21.15.16:0/3508727290 conn(0x7fdfac1019f0 msgr2=0x7fdfac103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 -- 172.21.15.16:0/3508727290 shutdown_connections 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.545+0000 7fdfb401c700 1 -- 172.21.15.16:0/3508727290 wait complete. 2024-05-08T23:12:46.389 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb401c700 1 Processor -- start 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb401c700 1 -- start start 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb401c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb401c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac19a850 con 0x7fdfac106470 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb1db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb1db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50138/0 (socket says 172.21.15.16:50138) 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.546+0000 7fdfb1db8700 1 -- 172.21.15.16:0/355877847 learned_addr learned my addr 172.21.15.16:0/355877847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdfb1db8700 1 -- 172.21.15.16:0/355877847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0009550 con 0x7fdfac106470 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdfb1db8700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac19a310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0003e30 tx=0x7fdfa0011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa00036a0 con 0x7fdfac106470 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfa001b440 con 0x7fdfac106470 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac19aa50 con 0x7fdfac106470 2024-05-08T23:12:46.390 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfa00193d0 con 0x7fdfac106470 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.547+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac19b5b0 con 0x7fdfac106470 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.548+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fdfa0019690 con 0x7fdfac106470 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.548+0000 7fdf9effd700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fdf98038700 0x7fdf9803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.548+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac19aef0 con 0x7fdfac106470 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.548+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdfa004c880 con 0x7fdfac106470 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.548+0000 7fdfb15b7700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fdf98038700 0x7fdf9803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.549+0000 7fdfb15b7700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fdf98038700 0x7fdf9803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006fd0 tx=0x7fdfa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.553+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdfac19aef0 con 0x7fdfac106470 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:45.783+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdfac19aef0 con 0x7fdf98038700 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.036+0000 7fdf9effd700 1 -- 172.21.15.16:0/355877847 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fdfac19aef0 con 0x7fdf98038700 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fdf98038700 msgr2=0x7fdf9803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fdf98038700 0x7fdf9803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006fd0 tx=0x7fdfa8006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 msgr2=0x7fdfac19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac19a310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0003e30 tx=0x7fdfa0011e80 comp rx=0 tx=0).stop 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 shutdown_connections 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fdf98038700 0x7fdf9803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 --2- 172.21.15.16:0/355877847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfac106470 0x7fdfac19a310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 >> 172.21.15.16:0/355877847 conn(0x7fdfac1019f0 msgr2=0x7fdfac102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 shutdown_connections 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.038+0000 7fdfb401c700 1 -- 172.21.15.16:0/355877847 wait complete. 2024-05-08T23:12:46.392 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-05-08T23:12:46.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:46 smithi016 ceph-mon[42541]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:46.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:46 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.754+0000 7fd12ffe3700 1 Processor -- start 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.754+0000 7fd12ffe3700 1 -- start start 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.754+0000 7fd12ffe3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd128105190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.754+0000 7fd12ffe3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd128079e90 con 0x7fd128104d70 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12dd7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd128105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12dd7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd128105190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45976/0 (socket says 172.21.15.16:45976) 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12dd7f700 1 -- 172.21.15.16:0/4293453174 learned_addr learned my addr 172.21.15.16:0/4293453174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:47.319 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12dd7f700 1 -- 172.21.15.16:0/4293453174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1281056d0 con 0x7fd128104d70 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12dd7f700 1 --2- 172.21.15.16:0/4293453174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd128105190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0098a0 tx=0x7fd11c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f4f346c96d312 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12cd7d700 1 -- 172.21.15.16:0/4293453174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd11c004160 con 0x7fd128104d70 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12cd7d700 1 -- 172.21.15.16:0/4293453174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd11c0042c0 con 0x7fd128104d70 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.755+0000 7fd12cd7d700 1 -- 172.21.15.16:0/4293453174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd11c0118e0 con 0x7fd128104d70 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 -- 172.21.15.16:0/4293453174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 msgr2=0x7fd128105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 --2- 172.21.15.16:0/4293453174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd128105190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0098a0 tx=0x7fd11c009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 -- 172.21.15.16:0/4293453174 shutdown_connections 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 --2- 172.21.15.16:0/4293453174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd128105190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 -- 172.21.15.16:0/4293453174 >> 172.21.15.16:0/4293453174 conn(0x7fd128100970 msgr2=0x7fd128102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 -- 172.21.15.16:0/4293453174 shutdown_connections 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.756+0000 7fd12ffe3700 1 -- 172.21.15.16:0/4293453174 wait complete. 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12ffe3700 1 Processor -- start 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12ffe3700 1 -- start start 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12ffe3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd12819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:47.320 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12ffe3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12819c9a0 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12dd7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd12819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12dd7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd12819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45992/0 (socket says 172.21.15.16:45992) 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12dd7f700 1 -- 172.21.15.16:0/728900063 learned_addr learned my addr 172.21.15.16:0/728900063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.757+0000 7fd12dd7f700 1 -- 172.21.15.16:0/728900063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009550 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.758+0000 7fd12dd7f700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd12819c460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd11c003f70 tx=0x7fd11c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.758+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd11c0036a0 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.758+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd11c01b440 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.758+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12819cba0 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.758+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd11c023400 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.758+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12819d6b0 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.759+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fd11c023630 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.759+0000 7fd11affd700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd1140387a0 0x7fd11403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.759+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd11c0039d0 con 0x7fd128104d70 2024-05-08T23:12:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.759+0000 7fd12d57e700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd1140387a0 0x7fd11403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.759+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c005320 con 0x7fd128104d70 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.760+0000 7fd12d57e700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd1140387a0 0x7fd11403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd124006fd0 tx=0x7fd124006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.764+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd11c023930 con 0x7fd128104d70 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.976+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd10c005b50 con 0x7fd128104d70 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.976+0000 7fd11affd700 1 -- 172.21.15.16:0/728900063 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd11c04a020 con 0x7fd128104d70 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd1140387a0 msgr2=0x7fd11403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd1140387a0 0x7fd11403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd124006fd0 tx=0x7fd124006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 msgr2=0x7fd12819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd12819c460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd11c003f70 tx=0x7fd11c0119a0 comp rx=0 tx=0).stop 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 shutdown_connections 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd1140387a0 0x7fd11403ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 --2- 172.21.15.16:0/728900063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128104d70 0x7fd12819c460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.978+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 >> 172.21.15.16:0/728900063 conn(0x7fd128100970 msgr2=0x7fd128193190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:47.322 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.979+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 shutdown_connections 2024-05-08T23:12:47.323 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:46.979+0000 7fd12ffe3700 1 -- 172.21.15.16:0/728900063 wait complete. 2024-05-08T23:12:47.330 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T23:12:47.330 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-05-08T23:12:47.339 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T23:12:47.340 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-05-08T23:12:47.340 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-05-08T23:12:47.340 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016:8443/ 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout: Password: h0i0f17pr9 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:47.342 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config directory 2024-05-08T23:12:47.343 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling autotune for osd_memory_target 2024-05-08T23:12:47.961 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/728900063' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T23:12:48.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.711+0000 7f884577a700 1 Processor -- start 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.711+0000 7f884577a700 1 -- start start 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.711+0000 7f884577a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884007a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.711+0000 7f884577a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884007a910 con 0x7f884007bee0 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884007a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884007a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45998/0 (socket says 172.21.15.16:45998) 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883effd700 1 -- 172.21.15.16:0/1687664435 learned_addr learned my addr 172.21.15.16:0/1687664435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883effd700 1 -- 172.21.15.16:0/1687664435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884007aa50 con 0x7f884007bee0 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883effd700 1 --2- 172.21.15.16:0/1687664435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884007a340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88300098a0 tx=0x7f8830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fdf233e659559c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883dffb700 1 -- 172.21.15.16:0/1687664435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8830004160 con 0x7f884007bee0 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883dffb700 1 -- 172.21.15.16:0/1687664435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88300042c0 con 0x7f884007bee0 2024-05-08T23:12:48.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.712+0000 7f883dffb700 1 -- 172.21.15.16:0/1687664435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88300118e0 con 0x7f884007bee0 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 -- 172.21.15.16:0/1687664435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 msgr2=0x7f884007a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 --2- 172.21.15.16:0/1687664435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884007a340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f88300098a0 tx=0x7f8830009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 -- 172.21.15.16:0/1687664435 shutdown_connections 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 --2- 172.21.15.16:0/1687664435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884007a340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 -- 172.21.15.16:0/1687664435 >> 172.21.15.16:0/1687664435 conn(0x7f8840101a20 msgr2=0x7f8840103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 -- 172.21.15.16:0/1687664435 shutdown_connections 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.713+0000 7f884577a700 1 -- 172.21.15.16:0/1687664435 wait complete. 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f884577a700 1 Processor -- start 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f884577a700 1 -- start start 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f884577a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f884577a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019c960 con 0x7f884007bee0 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f883effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:48.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f883effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46014/0 (socket says 172.21.15.16:46014) 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.714+0000 7f883effd700 1 -- 172.21.15.16:0/2976513389 learned_addr learned my addr 172.21.15.16:0/2976513389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f883effd700 1 -- 172.21.15.16:0/2976513389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009550 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f883effd700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884019c420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8830006af0 tx=0x7f8830011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88300037d0 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f883001b440 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884019cb60 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8830023400 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.715+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884019d6c0 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.716+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f88300236c0 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.716+0000 7f8827fff700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8828038700 0x7f882803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.716+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f883001b5b0 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.716+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8840105730 con 0x7f884007bee0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.716+0000 7f883e7fc700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8828038700 0x7f882803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:48.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.717+0000 7f883e7fc700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8828038700 0x7f882803abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8834006fd0 tx=0x7f8834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.722+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88300239c0 con 0x7f884007bee0 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.935+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f884019d900 con 0x7f884007bee0 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.935+0000 7f8827fff700 1 -- 172.21.15.16:0/2976513389 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f883004a020 con 0x7f884007bee0 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8828038700 msgr2=0x7f882803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8828038700 0x7f882803abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8834006fd0 tx=0x7f8834006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 msgr2=0x7f884019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884019c420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8830006af0 tx=0x7f8830011ed0 comp rx=0 tx=0).stop 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 shutdown_connections 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8828038700 0x7f882803abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 --2- 172.21.15.16:0/2976513389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884007bee0 0x7f884019c420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 >> 172.21.15.16:0/2976513389 conn(0x7f8840101a20 msgr2=0x7f88401025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 shutdown_connections 2024-05-08T23:12:48.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:47.938+0000 7f884577a700 1 -- 172.21.15.16:0/2976513389 wait complete. 2024-05-08T23:12:49.326 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.632+0000 7f918de81700 1 Processor -- start 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.632+0000 7f918de81700 1 -- start start 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.632+0000 7f918de81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f91881051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.632+0000 7f918de81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188079e50 con 0x7f9188104da0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f91881051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f91881051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46020/0 (socket says 172.21.15.16:46020) 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91877fe700 1 -- 172.21.15.16:0/3161767991 learned_addr learned my addr 172.21.15.16:0/3161767991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91877fe700 1 -- 172.21.15.16:0/3161767991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188105700 con 0x7f9188104da0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91877fe700 1 --2- 172.21.15.16:0/3161767991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f91881051c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f91780098a0 tx=0x7f9178009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651457c120cad0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91867fc700 1 -- 172.21.15.16:0/3161767991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9178004160 con 0x7f9188104da0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91867fc700 1 -- 172.21.15.16:0/3161767991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91780042c0 con 0x7f9188104da0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.633+0000 7f91867fc700 1 -- 172.21.15.16:0/3161767991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91780118e0 con 0x7f9188104da0 2024-05-08T23:12:49.327 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 -- 172.21.15.16:0/3161767991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 msgr2=0x7f91881051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 --2- 172.21.15.16:0/3161767991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f91881051c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f91780098a0 tx=0x7f9178009bb0 comp rx=0 tx=0).stop 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 -- 172.21.15.16:0/3161767991 shutdown_connections 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 --2- 172.21.15.16:0/3161767991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f91881051c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 -- 172.21.15.16:0/3161767991 >> 172.21.15.16:0/3161767991 conn(0x7f9188100960 msgr2=0x7f9188102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 -- 172.21.15.16:0/3161767991 shutdown_connections 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.634+0000 7f918de81700 1 -- 172.21.15.16:0/3161767991 wait complete. 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f918de81700 1 Processor -- start 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f918de81700 1 -- start start 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f918de81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f918819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f918de81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819c9e0 con 0x7f9188104da0 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f91877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f918819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f91877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f918819c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46034/0 (socket says 172.21.15.16:46034) 2024-05-08T23:12:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.635+0000 7f91877fe700 1 -- 172.21.15.16:0/1026999809 learned_addr learned my addr 172.21.15.16:0/1026999809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f91877fe700 1 -- 172.21.15.16:0/1026999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009550 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f91877fe700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f918819c4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91780046e0 tx=0x7f91780119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91780037d0 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918819cbe0 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f917801b440 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9178023400 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.636+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918819d740 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.637+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f9178023680 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.637+0000 7f9184ff9700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f91740387f0 0x7f917403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.637+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f917804c8b0 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.637+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9188104650 con 0x7f9188104da0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.637+0000 7f9186ffd700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f91740387f0 0x7f917403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.638+0000 7f9186ffd700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f91740387f0 0x7f917403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9170006fd0 tx=0x7f9170006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:49.329 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.642+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f917802bb80 con 0x7f9188104da0 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.973+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f9188066f40 con 0x7f9188104da0 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.975+0000 7f9184ff9700 1 -- 172.21.15.16:0/1026999809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f917802b7c0 con 0x7f9188104da0 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f91740387f0 msgr2=0x7f917403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f91740387f0 0x7f917403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9170006fd0 tx=0x7f9170006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 msgr2=0x7f918819c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f918819c4a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91780046e0 tx=0x7f91780119a0 comp rx=0 tx=0).stop 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 shutdown_connections 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f91740387f0 0x7f917403acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 --2- 172.21.15.16:0/1026999809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9188104da0 0x7f918819c4a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 >> 172.21.15.16:0/1026999809 conn(0x7f9188100960 msgr2=0x7f9188193160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 shutdown_connections 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-05-08T23:12:48.978+0000 7f918de81700 1 -- 172.21.15.16:0/1026999809 wait complete. 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T23:12:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:49.331 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-05-08T23:12:49.372 INFO:tasks.cephadm:Fetching config... 2024-05-08T23:12:49.373 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:12:49.373 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T23:12:49.393 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T23:12:49.393 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:12:49.393 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T23:12:49.454 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T23:12:49.454 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:12:49.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T23:12:49.532 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T23:12:49.532 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:12:49.532 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T23:12:49.592 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T23:12:49.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T23:12:49.677 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:12:49.691 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T23:12:49.737 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:12:49.750 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T23:12:49.796 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoRX7Vz/EUo848k2cT6fFyto18XCnm1BBDr8h2/TsoUQRTUjid1oEADullblTZBSUXmq1zSUQEdFpu8+Dv2PoVo+Fre4IgBiHJ3wHQTIxZbeP0zh+bnAYBbaa8Wx08PvyVKDkPyEFb1cI0DEiCAwNy8ILLNCcVScTOedMRlO/pIAAp0UsTcb35uCFnh1cJtOJil7z7jUBGz5MDVHt3k6c5CM/0WGZ36wgqYMdUDBlyV0FJNifX6DPTMu3zsyAj89YXuKjnq4bxclQfLe6EZXUwcPpw3pTMSPeJLVcyfwQginuOM8InBMh+rVu5SCUXU/S8FpKWAZj3JhInmf9A1JhiLc4auzkN64t6vYZnTC97FE3FAJBwRDJpI8iEs2sa3N3BErSzJ/Wsqo2QWEfelCfXWrGIJJKuFE4wTyANIcATxyh0WrRHp3IDkkSGlJPPby9Kcl7A8PgWzgHSGCTzNSrgW0XDU5CN1XEkgsa96LTvvLuGa5xsMMfm+7+b5bhu790= ceph-378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:12:49.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T23:12:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:12:50.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:49 smithi016 ceph-mon[42541]: mgrmap e13: x(active, since 6s) 2024-05-08T23:12:50.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:49 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1026999809' entity='client.admin' 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 -- 172.21.15.16:0/1423121333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 msgr2=0x7fc8481037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 --2- 172.21.15.16:0/1423121333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc8481037b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc838009910 tx=0x7fc838009c20 comp rx=0 tx=0).stop 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 -- 172.21.15.16:0/1423121333 shutdown_connections 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 --2- 172.21.15.16:0/1423121333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc8481037b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 -- 172.21.15.16:0/1423121333 >> 172.21.15.16:0/1423121333 conn(0x7fc8480fadd0 msgr2=0x7fc8480fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 -- 172.21.15.16:0/1423121333 shutdown_connections 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.258+0000 7fc84e33b700 1 -- 172.21.15.16:0/1423121333 wait complete. 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc84e33b700 1 Processor -- start 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc84e33b700 1 -- start start 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc84e33b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc848195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc84e33b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8481961b0 con 0x7fc8481013c0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc847fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc848195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc847fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc848195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46052/0 (socket says 172.21.15.16:46052) 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc847fff700 1 -- 172.21.15.16:0/3887929476 learned_addr learned my addr 172.21.15.16:0/3887929476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.259+0000 7fc847fff700 1 -- 172.21.15.16:0/3887929476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380095f0 con 0x7fc8481013c0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.260+0000 7fc847fff700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc848195c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc83800b5c0 tx=0x7fc838005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.260+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc83800b960 con 0x7fc8481013c0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.260+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc83800f460 con 0x7fc8481013c0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.260+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8481963b0 con 0x7fc8481013c0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.260+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc838017440 con 0x7fc8481013c0 2024-05-08T23:12:51.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.260+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc848196850 con 0x7fc8481013c0 2024-05-08T23:12:51.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.261+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84818f500 con 0x7fc8481013c0 2024-05-08T23:12:51.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.261+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fc83800bac0 con 0x7fc8481013c0 2024-05-08T23:12:51.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.262+0000 7fc8457fa700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc8300387f0 0x7fc83003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:51.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.262+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc83804cfd0 con 0x7fc8481013c0 2024-05-08T23:12:51.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.262+0000 7fc8477fe700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc8300387f0 0x7fc83003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:51.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.262+0000 7fc8477fe700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc8300387f0 0x7fc83003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc83c006fd0 tx=0x7fc83c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:51.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.266+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc838015b80 con 0x7fc8481013c0 2024-05-08T23:12:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.485+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc8480625d0 con 0x7fc8481013c0 2024-05-08T23:12:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.489+0000 7fc8457fa700 1 -- 172.21.15.16:0/3887929476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc838027070 con 0x7fc8481013c0 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc8300387f0 msgr2=0x7fc83003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc8300387f0 0x7fc83003acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc83c006fd0 tx=0x7fc83c006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 msgr2=0x7fc848195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc848195c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc83800b5c0 tx=0x7fc838005810 comp rx=0 tx=0).stop 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 shutdown_connections 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc8300387f0 0x7fc83003acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 --2- 172.21.15.16:0/3887929476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8481013c0 0x7fc848195c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 >> 172.21.15.16:0/3887929476 conn(0x7fc8480fadd0 msgr2=0x7fc8480fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 shutdown_connections 2024-05-08T23:12:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:51.493+0000 7fc84e33b700 1 -- 172.21.15.16:0/3887929476 wait complete. 2024-05-08T23:12:51.875 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T23:12:51.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T23:12:52.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:12:52.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:52 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3887929476' entity='client.admin' 2024-05-08T23:12:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.288+0000 7fd00460f700 1 -- 172.21.15.16:0/352540621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 msgr2=0x7fcffc100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 --2- 172.21.15.16:0/352540621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc100d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009910 tx=0x7fcff8009c20 comp rx=0 tx=0).stop 2024-05-08T23:12:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 -- 172.21.15.16:0/352540621 shutdown_connections 2024-05-08T23:12:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 --2- 172.21.15.16:0/352540621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc100d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 -- 172.21.15.16:0/352540621 >> 172.21.15.16:0/352540621 conn(0x7fcffc0fbef0 msgr2=0x7fcffc0fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 -- 172.21.15.16:0/352540621 shutdown_connections 2024-05-08T23:12:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 -- 172.21.15.16:0/352540621 wait complete. 2024-05-08T23:12:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 Processor -- start 2024-05-08T23:12:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.289+0000 7fd00460f700 1 -- start start 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd00460f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd00460f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc196210 con 0x7fcffc100950 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd0023ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd0023ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46074/0 (socket says 172.21.15.16:46074) 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd0023ab700 1 -- 172.21.15.16:0/854844947 learned_addr learned my addr 172.21.15.16:0/854844947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd0023ab700 1 -- 172.21.15.16:0/854844947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80095f0 con 0x7fcffc100950 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd0023ab700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc195cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcff80098e0 tx=0x7fcff8005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff800ba90 con 0x7fcffc100950 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcff800bbf0 con 0x7fcffc100950 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff800fe70 con 0x7fcffc100950 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc196410 con 0x7fcffc100950 2024-05-08T23:12:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.290+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc1968b0 con 0x7fcffc100950 2024-05-08T23:12:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.291+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fcff800f460 con 0x7fcffc100950 2024-05-08T23:12:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.291+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc18f520 con 0x7fcffc100950 2024-05-08T23:12:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.292+0000 7fcff37fe700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcfe80387f0 0x7fcfe803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.292+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcff804c8c0 con 0x7fcffc100950 2024-05-08T23:12:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.292+0000 7fd001baa700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcfe80387f0 0x7fcfe803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.292+0000 7fd001baa700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcfe80387f0 0x7fcfe803acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006fd0 tx=0x7fcfec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.297+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcff802ad30 con 0x7fcffc100950 2024-05-08T23:12:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.514+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fcffc0613e0 con 0x7fcfe80387f0 2024-05-08T23:12:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.518+0000 7fcff37fe700 1 -- 172.21.15.16:0/854844947 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcffc0613e0 con 0x7fcfe80387f0 2024-05-08T23:12:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcfe80387f0 msgr2=0x7fcfe803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcfe80387f0 0x7fcfe803acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006fd0 tx=0x7fcfec006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 msgr2=0x7fcffc195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc195cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcff80098e0 tx=0x7fcff8005810 comp rx=0 tx=0).stop 2024-05-08T23:12:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 shutdown_connections 2024-05-08T23:12:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcfe80387f0 0x7fcfe803acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 --2- 172.21.15.16:0/854844947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcffc100950 0x7fcffc195cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 >> 172.21.15.16:0/854844947 conn(0x7fcffc0fbef0 msgr2=0x7fcffc0fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 shutdown_connections 2024-05-08T23:12:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:53.520+0000 7fd00460f700 1 -- 172.21.15.16:0/854844947 wait complete. 2024-05-08T23:12:53.883 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-05-08T23:12:53.883 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:12:53.883 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-05-08T23:12:53.905 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:12:53.905 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:12:53.965 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-05-08T23:12:53.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch host add smithi155 2024-05-08T23:12:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:54.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:12:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:12:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-08T23:12:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:54 smithi016 ceph-mon[42541]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2718097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 msgr2=0x7fd8c8068500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 --2- 172.21.15.16:0/2718097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c8068500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009910 tx=0x7fd8b8009c20 comp rx=0 tx=0).stop 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2718097495 shutdown_connections 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 --2- 172.21.15.16:0/2718097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c8068500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2718097495 >> 172.21.15.16:0/2718097495 conn(0x7fd8c8075350 msgr2=0x7fd8c8075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2718097495 shutdown_connections 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.221+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2718097495 wait complete. 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8ccbfd700 1 Processor -- start 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8ccbfd700 1 -- start start 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8ccbfd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8ccbfd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8198800 con 0x7fd8c8068120 2024-05-08T23:12:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8c659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c81982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46092/0 (socket says 172.21.15.16:46092) 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8c659c700 1 -- 172.21.15.16:0/2432901164 learned_addr learned my addr 172.21.15.16:0/2432901164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.222+0000 7fd8c659c700 1 -- 172.21.15.16:0/2432901164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b80095f0 con 0x7fd8c8068120 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.223+0000 7fd8c659c700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c81982c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80098e0 tx=0x7fd8b8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.223+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8b801d070 con 0x7fd8c8068120 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.223+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8b8009e60 con 0x7fd8c8068120 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.223+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c8198a00 con 0x7fd8c8068120 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.223+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8b8022b30 con 0x7fd8c8068120 2024-05-08T23:12:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.223+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8198e20 con 0x7fd8c8068120 2024-05-08T23:12:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.224+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd8b800f460 con 0x7fd8c8068120 2024-05-08T23:12:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.224+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c80ffdf0 con 0x7fd8c8068120 2024-05-08T23:12:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.224+0000 7fd8b77fe700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8b00387f0 0x7fd8b003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.224+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd8b804c970 con 0x7fd8c8068120 2024-05-08T23:12:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.224+0000 7fd8c5d9b700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8b00387f0 0x7fd8b003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.225+0000 7fd8c5d9b700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8b00387f0 0x7fd8b003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc006fd0 tx=0x7fd8bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:55.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.229+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd8b8020960 con 0x7fd8c8068120 2024-05-08T23:12:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:55.443+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8c810cca0 con 0x7fd8b00387f0 2024-05-08T23:12:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:55 smithi016 ceph-mon[42541]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:12:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:55 smithi016 ceph-mon[42541]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:12:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:57.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:56 smithi016 ceph-mon[42541]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:12:57.765 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-05-08T23:12:57.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.765+0000 7fd8b77fe700 1 -- 172.21.15.16:0/2432901164 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd8c810cca0 con 0x7fd8b00387f0 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8b00387f0 msgr2=0x7fd8b003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8b00387f0 0x7fd8b003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc006fd0 tx=0x7fd8bc006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 msgr2=0x7fd8c81982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c81982c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80098e0 tx=0x7fd8b8005710 comp rx=0 tx=0).stop 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 shutdown_connections 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8b00387f0 0x7fd8b003acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 --2- 172.21.15.16:0/2432901164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8c8068120 0x7fd8c81982c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 >> 172.21.15.16:0/2432901164 conn(0x7fd8c8075350 msgr2=0x7fd8c80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.768+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 shutdown_connections 2024-05-08T23:12:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:57.769+0000 7fd8ccbfd700 1 -- 172.21.15.16:0/2432901164 wait complete. 2024-05-08T23:12:57.935 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:57 smithi016 ceph-mon[42541]: Deploying cephadm binary to smithi155 2024-05-08T23:12:58.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T23:12:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:12:59.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:59.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:58 smithi016 ceph-mon[42541]: Added host smithi155 2024-05-08T23:12:59.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:12:59.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:59.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:12:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 -- 172.21.15.16:0/3799913488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 msgr2=0x7f829c104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 --2- 172.21.15.16:0/3799913488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c104b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f828c009910 tx=0x7f828c009c20 comp rx=0 tx=0).stop 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 -- 172.21.15.16:0/3799913488 shutdown_connections 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 --2- 172.21.15.16:0/3799913488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c104b90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 -- 172.21.15.16:0/3799913488 >> 172.21.15.16:0/3799913488 conn(0x7f829c0faa50 msgr2=0x7f829c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 -- 172.21.15.16:0/3799913488 shutdown_connections 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.561+0000 7f82a1404700 1 -- 172.21.15.16:0/3799913488 wait complete. 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.562+0000 7f82a1404700 1 Processor -- start 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.562+0000 7f82a1404700 1 -- start start 2024-05-08T23:12:59.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.562+0000 7f82a1404700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:59.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.562+0000 7f82a1404700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c1964e0 con 0x7f829c1047b0 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.562+0000 7f829affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f829affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50082/0 (socket says 172.21.15.16:50082) 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f829affd700 1 -- 172.21.15.16:0/1173729578 learned_addr learned my addr 172.21.15.16:0/1173729578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f829affd700 1 -- 172.21.15.16:0/1173729578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0095f0 con 0x7f829c1047b0 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f829affd700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c195fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f828c005660 tx=0x7f828c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f828c01d070 con 0x7f829c1047b0 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f828c00b750 con 0x7f829c1047b0 2024-05-08T23:12:59.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c196740 con 0x7f829c1047b0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f828c0037d0 con 0x7f829c1047b0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.563+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c196c40 con 0x7f829c1047b0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.564+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f828c003a90 con 0x7f829c1047b0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.564+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c100f90 con 0x7f829c1047b0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.564+0000 7f8283fff700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f82840387a0 0x7f828403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.564+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f828c04c340 con 0x7f829c1047b0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.564+0000 7f829a7fc700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f82840387a0 0x7f828403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:12:59.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.565+0000 7f829a7fc700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f82840387a0 0x7f828403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8290006fd0 tx=0x7f8290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:12:59.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.570+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f828c00f640 con 0x7f829c1047b0 2024-05-08T23:12:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.784+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f829c105210 con 0x7f82840387a0 2024-05-08T23:12:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.786+0000 7f8283fff700 1 -- 172.21.15.16:0/1173729578 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f829c105210 con 0x7f82840387a0 2024-05-08T23:12:59.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:12:59.786 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f82840387a0 msgr2=0x7f828403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f82840387a0 0x7f828403ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8290006fd0 tx=0x7f8290006e40 comp rx=0 tx=0).stop 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 msgr2=0x7f829c195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c195fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f828c005660 tx=0x7f828c005740 comp rx=0 tx=0).stop 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 shutdown_connections 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f82840387a0 0x7f828403ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 --2- 172.21.15.16:0/1173729578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f829c1047b0 0x7f829c195fa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:12:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 >> 172.21.15.16:0/1173729578 conn(0x7f829c0faa50 msgr2=0x7f829c107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:12:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 shutdown_connections 2024-05-08T23:12:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:12:59.788+0000 7f82a1404700 1 -- 172.21.15.16:0/1173729578 wait complete. 2024-05-08T23:13:00.173 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-05-08T23:13:00.173 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:13:00.173 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-05-08T23:13:00.194 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:13:00.195 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:13:00.254 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-05-08T23:13:00.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch host add smithi189 2024-05-08T23:13:00.508 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:13:00.609 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:00 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:00.610 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:00 smithi016 ceph-mon[42541]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1668209322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 msgr2=0x7fd9a4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 --2- 172.21.15.16:0/1668209322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a4101e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd994009910 tx=0x7fd994009c20 comp rx=0 tx=0).stop 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1668209322 shutdown_connections 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 --2- 172.21.15.16:0/1668209322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a4101e70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1668209322 >> 172.21.15.16:0/1668209322 conn(0x7fd9a40fb2e0 msgr2=0x7fd9a40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1668209322 shutdown_connections 2024-05-08T23:13:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.463+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1668209322 wait complete. 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9aa1ac700 1 Processor -- start 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9aa1ac700 1 -- start start 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9aa1ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a40ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9aa1ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a41015b0 con 0x7fd9a4101a90 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a40ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a40ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50098/0 (socket says 172.21.15.16:50098) 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.464+0000 7fd9a37fe700 1 -- 172.21.15.16:0/1417539842 learned_addr learned my addr 172.21.15.16:0/1417539842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9a37fe700 1 -- 172.21.15.16:0/1417539842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940095f0 con 0x7fd9a4101a90 2024-05-08T23:13:01.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9a37fe700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a40ff6f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9940055f0 tx=0x7fd9940056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd99401d070 con 0x7fd9a4101a90 2024-05-08T23:13:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd99400b750 con 0x7fd9a4101a90 2024-05-08T23:13:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a40ffc30 con 0x7fd9a4101a90 2024-05-08T23:13:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9940037b0 con 0x7fd9a4101a90 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.465+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a4100050 con 0x7fd9a4101a90 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.466+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd994003a20 con 0x7fd9a4101a90 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.466+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a4069260 con 0x7fd9a4101a90 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.466+0000 7fd9a0ff9700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd98c0387f0 0x7fd98c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.466+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd99404c470 con 0x7fd9a4101a90 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.466+0000 7fd9a2ffd700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd98c0387f0 0x7fd98c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.467+0000 7fd9a2ffd700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd98c0387f0 0x7fd98c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd998006fd0 tx=0x7fd998006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:01.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.471+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd99400f980 con 0x7fd9a4101a90 2024-05-08T23:13:01.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:01.689+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9a4100290 con 0x7fd98c0387f0 2024-05-08T23:13:02.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: Updating smithi155:/etc/ceph/ceph.conf 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:13:02.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:02 smithi016 ceph-mon[42541]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: Deploying cephadm binary to smithi189 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:03.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:03 smithi016 ceph-mon[42541]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.002+0000 7fd9a0ff9700 1 -- 172.21.15.16:0/1417539842 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd9a4100290 con 0x7fd98c0387f0 2024-05-08T23:13:04.003 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.005+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd98c0387f0 msgr2=0x7fd98c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.005+0000 7fd9aa1ac700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd98c0387f0 0x7fd98c03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd998006fd0 tx=0x7fd998006e40 comp rx=0 tx=0).stop 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.005+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 msgr2=0x7fd9a40ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.005+0000 7fd9aa1ac700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a40ff6f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9940055f0 tx=0x7fd9940056d0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.006+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 shutdown_connections 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.006+0000 7fd9aa1ac700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd98c0387f0 0x7fd98c03acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.006+0000 7fd9aa1ac700 1 --2- 172.21.15.16:0/1417539842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a4101a90 0x7fd9a40ff6f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.006+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 >> 172.21.15.16:0/1417539842 conn(0x7fd9a40fb2e0 msgr2=0x7fd9a4104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.006+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 shutdown_connections 2024-05-08T23:13:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:04.006+0000 7fd9aa1ac700 1 -- 172.21.15.16:0/1417539842 wait complete. 2024-05-08T23:13:04.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T23:13:04.624 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:13:05.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:05.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:05 smithi016 ceph-mon[42541]: Added host smithi189 2024-05-08T23:13:05.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:05.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:05.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:05 smithi016 ceph-mon[42541]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:05.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3487126343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 msgr2=0x7fd4a4105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 --2- 172.21.15.16:0/3487126343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4105c00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd494009910 tx=0x7fd494009c20 comp rx=0 tx=0).stop 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3487126343 shutdown_connections 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 --2- 172.21.15.16:0/3487126343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4105c00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3487126343 >> 172.21.15.16:0/3487126343 conn(0x7fd4a4075350 msgr2=0x7fd4a4075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3487126343 shutdown_connections 2024-05-08T23:13:05.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.626+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3487126343 wait complete. 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a8ac0700 1 Processor -- start 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a8ac0700 1 -- start start 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a8ac0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a8ac0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4196660 con 0x7fd4a4105820 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50126/0 (socket says 172.21.15.16:50126) 2024-05-08T23:13:05.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.627+0000 7fd4a259c700 1 -- 172.21.15.16:0/3967729017 learned_addr learned my addr 172.21.15.16:0/3967729017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4a259c700 1 -- 172.21.15.16:0/3967729017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940095f0 con 0x7fd4a4105820 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4a259c700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4196120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd4940055f0 tx=0x7fd4940056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd49401d070 con 0x7fd4a4105820 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd49400b750 con 0x7fd4a4105820 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a4196860 con 0x7fd4a4105820 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4940037b0 con 0x7fd4a4105820 2024-05-08T23:13:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.628+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a4196c80 con 0x7fd4a4105820 2024-05-08T23:13:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.629+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fd494003a70 con 0x7fd4a4105820 2024-05-08T23:13:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.629+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a4100ef0 con 0x7fd4a4105820 2024-05-08T23:13:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.629+0000 7fd4937fe700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd48c0387a0 0x7fd48c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.629+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd49404c4e0 con 0x7fd4a4105820 2024-05-08T23:13:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.629+0000 7fd4a1d9b700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd48c0387a0 0x7fd48c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.630+0000 7fd4a1d9b700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd48c0387a0 0x7fd48c03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd498006fd0 tx=0x7fd498006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.634+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd494029770 con 0x7fd4a4105820 2024-05-08T23:13:05.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.852+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4a41099e0 con 0x7fd48c0387a0 2024-05-08T23:13:05.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.854+0000 7fd4937fe700 1 -- 172.21.15.16:0/3967729017 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fd4a41099e0 con 0x7fd48c0387a0 2024-05-08T23:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-05-08T23:13:05.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd48c0387a0 msgr2=0x7fd48c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd48c0387a0 0x7fd48c03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd498006fd0 tx=0x7fd498006e40 comp rx=0 tx=0).stop 2024-05-08T23:13:05.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 msgr2=0x7fd4a4196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:05.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4196120 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd4940055f0 tx=0x7fd4940056d0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 shutdown_connections 2024-05-08T23:13:05.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd48c0387a0 0x7fd48c03ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 --2- 172.21.15.16:0/3967729017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a4105820 0x7fd4a4196120 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:05.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 >> 172.21.15.16:0/3967729017 conn(0x7fd4a4075350 msgr2=0x7fd4a40680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:05.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 shutdown_connections 2024-05-08T23:13:05.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:05.856+0000 7fd4a8ac0700 1 -- 172.21.15.16:0/3967729017 wait complete. 2024-05-08T23:13:06.222 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T23:13:06.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T23:13:06.475 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:13:06.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:06 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:06.593 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:06 smithi016 ceph-mon[42541]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:13:07.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 -- 172.21.15.16:0/386997998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 msgr2=0x7feff4068500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:07.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 --2- 172.21.15.16:0/386997998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff4068500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feff0009910 tx=0x7feff0009c20 comp rx=0 tx=0).stop 2024-05-08T23:13:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 -- 172.21.15.16:0/386997998 shutdown_connections 2024-05-08T23:13:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 --2- 172.21.15.16:0/386997998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff4068500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 -- 172.21.15.16:0/386997998 >> 172.21.15.16:0/386997998 conn(0x7feff4075350 msgr2=0x7feff4075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 -- 172.21.15.16:0/386997998 shutdown_connections 2024-05-08T23:13:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.445+0000 7feffb7be700 1 -- 172.21.15.16:0/386997998 wait complete. 2024-05-08T23:13:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.446+0000 7feffb7be700 1 Processor -- start 2024-05-08T23:13:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.446+0000 7feffb7be700 1 -- start start 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.446+0000 7feffb7be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.446+0000 7feffb7be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4198800 con 0x7feff4068120 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feff955a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feff955a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff41982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37990/0 (socket says 172.21.15.16:37990) 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feff955a700 1 -- 172.21.15.16:0/3607996858 learned_addr learned my addr 172.21.15.16:0/3607996858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feff955a700 1 -- 172.21.15.16:0/3607996858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00095f0 con 0x7feff4068120 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feff955a700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff41982c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feff0005660 tx=0x7feff0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff001d070 con 0x7feff4068120 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feff000b750 con 0x7feff4068120 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff4198a00 con 0x7feff4068120 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feff00037d0 con 0x7feff4068120 2024-05-08T23:13:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.447+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff4198e20 con 0x7feff4068120 2024-05-08T23:13:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.448+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff40ffdf0 con 0x7feff4068120 2024-05-08T23:13:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.449+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7feff0004390 con 0x7feff4068120 2024-05-08T23:13:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.449+0000 7fefea7fc700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefe00387f0 0x7fefe003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.449+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feff004c360 con 0x7feff4068120 2024-05-08T23:13:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.449+0000 7feff8d59700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefe00387f0 0x7fefe003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.449+0000 7feff8d59700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefe00387f0 0x7fefe003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fefe4006fd0 tx=0x7fefe4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.454+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feff0029360 con 0x7feff4068120 2024-05-08T23:13:07.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:07 smithi016 ceph-mon[42541]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:07.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:07.672+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7feff4199720 con 0x7feff4068120 2024-05-08T23:13:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.387+0000 7fefea7fc700 1 -- 172.21.15.16:0/3607996858 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7feff0026020 con 0x7feff4068120 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefe00387f0 msgr2=0x7fefe003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefe00387f0 0x7fefe003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fefe4006fd0 tx=0x7fefe4006e40 comp rx=0 tx=0).stop 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 msgr2=0x7feff41982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff41982c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feff0005660 tx=0x7feff0005740 comp rx=0 tx=0).stop 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 shutdown_connections 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefe00387f0 0x7fefe003acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 --2- 172.21.15.16:0/3607996858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4068120 0x7feff41982c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 >> 172.21.15.16:0/3607996858 conn(0x7feff4075350 msgr2=0x7feff40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 shutdown_connections 2024-05-08T23:13:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:08.390+0000 7feffb7be700 1 -- 172.21.15.16:0/3607996858 wait complete. 2024-05-08T23:13:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3607996858' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:08.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:08 smithi016 ceph-mon[42541]: Updating smithi189:/etc/ceph/ceph.conf 2024-05-08T23:13:08.763 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-05-08T23:13:08.763 INFO:tasks.cephadm:Adding mon.b on smithi155 2024-05-08T23:13:08.763 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-05-08T23:13:08.763 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi189:172.21.15.189=c' 2024-05-08T23:13:09.007 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:13:09.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3607996858' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 -- 172.21.15.189:0/165025119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 msgr2=0x7fbc60106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 --2- 172.21.15.189:0/165025119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60106c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b0a0 tx=0x7fbc5800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 -- 172.21.15.189:0/165025119 shutdown_connections 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 --2- 172.21.15.189:0/165025119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60106c20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 -- 172.21.15.189:0/165025119 >> 172.21.15.189:0/165025119 conn(0x7fbc600fc370 msgr2=0x7fbc600fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 -- 172.21.15.189:0/165025119 shutdown_connections 2024-05-08T23:13:09.936 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.935+0000 7fbc65d56700 1 -- 172.21.15.189:0/165025119 wait complete. 2024-05-08T23:13:09.937 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc65d56700 1 Processor -- start 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc65d56700 1 -- start start 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc65d56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc65d56700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc601964b0 con 0x7fbc60106840 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.189:48672/0 (socket says 172.21.15.189:48672) 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.936+0000 7fbc5f7fe700 1 -- 172.21.15.189:0/2983712732 learned_addr learned my addr 172.21.15.189:0/2983712732 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc5f7fe700 1 -- 172.21.15.189:0/2983712732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009d00 con 0x7fbc60106840 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc5f7fe700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60195f70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc58015040 tx=0x7fbc580078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc5800e040 con 0x7fbc60106840 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc58003c10 con 0x7fbc60106840 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc601966b0 con 0x7fbc60106840 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc5801d610 con 0x7fbc60106840 2024-05-08T23:13:09.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.937+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc60196b30 con 0x7fbc60106840 2024-05-08T23:13:09.939 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.938+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc60101f80 con 0x7fbc60106840 2024-05-08T23:13:09.939 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.938+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fbc58004040 con 0x7fbc60106840 2024-05-08T23:13:09.939 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.938+0000 7fbc5cff9700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbc4c0387a0 0x7fbc4c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:09.940 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.938+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc5804cc60 con 0x7fbc60106840 2024-05-08T23:13:09.940 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.939+0000 7fbc5effd700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbc4c0387a0 0x7fbc4c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:09.940 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.939+0000 7fbc5effd700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbc4c0387a0 0x7fbc4c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbc48006fd0 tx=0x7fbc48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:09.944 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:09.944+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbc58027460 con 0x7fbc60106840 2024-05-08T23:13:10.145 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.145+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6010aab0 con 0x7fbc4c0387a0 2024-05-08T23:13:10.152 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.152+0000 7fbc5cff9700 1 -- 172.21.15.189:0/2983712732 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbc6010aab0 con 0x7fbc4c0387a0 2024-05-08T23:13:10.153 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbc4c0387a0 msgr2=0x7fbc4c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbc4c0387a0 0x7fbc4c03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbc48006fd0 tx=0x7fbc48006e40 comp rx=0 tx=0).stop 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 msgr2=0x7fbc60195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60195f70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc58015040 tx=0x7fbc580078d0 comp rx=0 tx=0).stop 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 shutdown_connections 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbc4c0387a0 0x7fbc4c03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 --2- 172.21.15.189:0/2983712732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc60106840 0x7fbc60195f70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 >> 172.21.15.189:0/2983712732 conn(0x7fbc600fc370 msgr2=0x7fbc60108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 shutdown_connections 2024-05-08T23:13:10.155 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:10.154+0000 7fbc65d56700 1 -- 172.21.15.189:0/2983712732 wait complete. 2024-05-08T23:13:10.551 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T23:13:10.554 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T23:13:10.557 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T23:13:10.557 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T23:13:10.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:13:10.592 INFO:journalctl@ceph.mon.b.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:13:11.046 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:13:11.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi189:172.21.15.189=c;count:3 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: Deploying daemon mon.c on smithi189 2024-05-08T23:13:11.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:11 smithi016 ceph-mon[42541]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.740+0000 7f2710416700 1 -- 172.21.15.189:0/1631567715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 msgr2=0x7f2708101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.740+0000 7f2710416700 1 --2- 172.21.15.189:0/1631567715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708101e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2704009910 tx=0x7f2704009c20 comp rx=0 tx=0).stop 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 -- 172.21.15.189:0/1631567715 shutdown_connections 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 --2- 172.21.15.189:0/1631567715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708101e70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 -- 172.21.15.189:0/1631567715 >> 172.21.15.189:0/1631567715 conn(0x7f27080fb2e0 msgr2=0x7f27080fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 -- 172.21.15.189:0/1631567715 shutdown_connections 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 -- 172.21.15.189:0/1631567715 wait complete. 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 Processor -- start 2024-05-08T23:13:12.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.741+0000 7f2710416700 1 -- start start 2024-05-08T23:13:12.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.742+0000 7f2710416700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.742+0000 7f2710416700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27081966d0 con 0x7f2708101a90 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.742+0000 7f270e1b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.742+0000 7f270e1b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.189:48684/0 (socket says 172.21.15.189:48684) 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.742+0000 7f270e1b2700 1 -- 172.21.15.189:0/949095894 learned_addr learned my addr 172.21.15.189:0/949095894 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.742+0000 7f270e1b2700 1 -- 172.21.15.189:0/949095894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040095f0 con 0x7f2708101a90 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.743+0000 7f270e1b2700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708196190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f27040055f0 tx=0x7f27040056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.743+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f270401d070 con 0x7f2708101a90 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.743+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27081968d0 con 0x7f2708101a90 2024-05-08T23:13:12.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.743+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2708196cf0 con 0x7f2708101a90 2024-05-08T23:13:12.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.743+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f270400b750 con 0x7f2708101a90 2024-05-08T23:13:12.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.743+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27040037f0 con 0x7f2708101a90 2024-05-08T23:13:12.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.744+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec0052f0 con 0x7f2708101a90 2024-05-08T23:13:12.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.744+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f2704004390 con 0x7f2708101a90 2024-05-08T23:13:12.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.744+0000 7f26ff7fe700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f26f40387a0 0x7f26f403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:12.746 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.744+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f270404c5a0 con 0x7f2708101a90 2024-05-08T23:13:12.746 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.744+0000 7f270d9b1700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f26f40387a0 0x7f26f403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:12.746 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.745+0000 7f270d9b1700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f26f40387a0 0x7f26f403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f26f8006fd0 tx=0x7f26f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:12.750 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:12.750+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f270400bba0 con 0x7f2708101a90 2024-05-08T23:13:13.072 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.071+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f26ec005160 con 0x7f2708101a90 2024-05-08T23:13:13.072 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.072+0000 7f26ff7fe700 1 -- 172.21.15.189:0/949095894 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f2704026070 con 0x7f2708101a90 2024-05-08T23:13:13.073 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:13:13.073 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","modified":"2024-05-08T23:11:52.906042Z","created":"2024-05-08T23:11:52.906042Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T23:13:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f26f40387a0 msgr2=0x7f26f403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f26f40387a0 0x7f26f403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f26f8006fd0 tx=0x7f26f8006e40 comp rx=0 tx=0).stop 2024-05-08T23:13:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 msgr2=0x7f2708196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708196190 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f27040055f0 tx=0x7f27040056d0 comp rx=0 tx=0).stop 2024-05-08T23:13:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 shutdown_connections 2024-05-08T23:13:13.076 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f26f40387a0 0x7f26f403ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:13.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 --2- 172.21.15.189:0/949095894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2708101a90 0x7f2708196190 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:13.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 >> 172.21.15.189:0/949095894 conn(0x7f27080fb2e0 msgr2=0x7f2708104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:13.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.075+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 shutdown_connections 2024-05-08T23:13:13.077 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:13.076+0000 7f2710416700 1 -- 172.21.15.189:0/949095894 wait complete. 2024-05-08T23:13:13.077 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-05-08T23:13:13.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:13 smithi016 ceph-mon[42541]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:13.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:13 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/949095894' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:13:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 systemd[1]: Started Ceph mon.c for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:13:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T23:13:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kZSTD supported: 0 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kXpressCompression supported: 0 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kZlibCompression supported: 1 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kSnappyCompression supported: 1 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kLZ4Compression supported: 1 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: kBZip2Compression supported: 0 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.merge_operator: 2024-05-08T23:13:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_filter: None 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560bde459620) 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks: 1 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_top_level_index_and_filter: 1 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_type: 0 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_index_type: 0 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_shortening: 1 2024-05-08T23:13:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: checksum: 4 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: no_block_cache: 0 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache: 0x560bdd7711f0 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_name: BinnedLRUCache 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_options: 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: capacity : 536870912 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_shard_bits : 4 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: strict_capacity_limit : 0 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: high_pri_pool_ratio: 0.000 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_compressed: (nil) 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: persistent_cache: (nil) 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size: 4096 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size_deviation: 10 2024-05-08T23:13:14.172 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_restart_interval: 16 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_block_restart_interval: 1 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: metadata_block_size: 4096 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: partition_filters: 0 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: use_delta_encoding: 1 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: filter_policy: bloomfilter 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: whole_key_filtering: 1 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: verify_compression: 0 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: read_amp_bytes_per_bit: 0 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: format_version: 5 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_index_compression: 1 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_align: 0 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: max_auto_readahead_size: 262144 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: prepopulate_block_cache: 0 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: initial_auto_readahead_size: 8192 2024-05-08T23:13:14.173 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression: NoCompression 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.num_levels: 7 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T23:13:14.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T23:13:14.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T23:13:14.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:13:14.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T23:13:14.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T23:13:14.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T23:13:14.177 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T23:13:14.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.table_properties_collectors: 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.inplace_update_support: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.bloom_locality: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.max_successive_merges: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T23:13:14.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.ttl: 2592000 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.enable_blob_files: false 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.min_blob_size: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T23:13:14.180 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 68a365f9-6e0b-46fa-b873-0ad6dfd7f0a9 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209993843871, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209993844356, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715209993, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "68a365f9-6e0b-46fa-b873-0ad6dfd7f0a9", "db_session_id": "7SQ64RC89PBO3LE0J9OK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715209993844453, "job": 1, "event": "recovery_finished"} 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560bdd80e000 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: DB pointer 0x560bdd7fc000 2024-05-08T23:13:14.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** DB Stats ** 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:13:14.182 INFO:journalctl@ceph.mon.c.smithi189.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T23:13:14.183 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache BinnedLRUCache@0x560bdd7711f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: using public_addr v2:172.21.15.189:0/0 -> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-05-08T23:13:14.184 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: starting mon.c rank -1 at public addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] at bind addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(???) e0 preinit fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout: e1 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout: legacy client fscid: -1 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout: No filesystems configured 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T23:13:14.185 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/949095894' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:13:14.186 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:13 smithi189 ceph-mon[46919]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T23:13:14.490 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T23:13:14.491 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T23:13:14.717 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:13:18.953 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.951+0000 7f4974cb9700 1 -- 172.21.15.189:0/2522587407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49580035c0 msgr2=0x7f4958005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:18.953 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.951+0000 7f4974cb9700 1 --2- 172.21.15.189:0/2522587407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49580035c0 0x7f4958005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f49700ff7f0 tx=0x7f4960013e90 comp rx=0 tx=0).stop 2024-05-08T23:13:18.953 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.952+0000 7f4974cb9700 1 -- 172.21.15.189:0/2522587407 shutdown_connections 2024-05-08T23:13:18.953 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.952+0000 7f4974cb9700 1 --2- 172.21.15.189:0/2522587407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49580035c0 0x7f4958005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:18.954 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.952+0000 7f4974cb9700 1 -- 172.21.15.189:0/2522587407 >> 172.21.15.189:0/2522587407 conn(0x7f4970076570 msgr2=0x7f49700fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:18.954 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.952+0000 7f4974cb9700 1 -- 172.21.15.189:0/2522587407 shutdown_connections 2024-05-08T23:13:18.954 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.952+0000 7f4974cb9700 1 -- 172.21.15.189:0/2522587407 wait complete. 2024-05-08T23:13:18.959 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.953+0000 7f4974cb9700 1 Processor -- start 2024-05-08T23:13:18.959 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.953+0000 7f4974cb9700 1 -- start start 2024-05-08T23:13:18.959 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.953+0000 7f4974cb9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:18.959 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.953+0000 7f4974cb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49701966b0 0x7f497019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.953+0000 7f4974cb9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970196c40 con 0x7f49701966b0 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.953+0000 7f4974cb9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970196db0 con 0x7f49580035c0 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:37486/0 (socket says 172.21.15.189:37486) 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 -- 172.21.15.189:0/878207285 learned_addr learned my addr 172.21.15.189:0/878207285 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496dd9b700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49701966b0 0x7f497019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 -- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 msgr2=0x7f4970196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 -- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 msgr2=0x7f4970196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496e59c700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:13:18.960 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496dd9b700 1 -- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 msgr2=0x7f4970196170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496dd9b700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496dd9b700 1 -- 172.21.15.189:0/878207285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600138e0 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f496dd9b700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49701966b0 0x7f497019aa60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f496400b4d0 tx=0x7f496400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4964011630 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4964011c20 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497019b000 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.954+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4964010c60 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.955+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497019b550 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.955+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f496401a400 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.956+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497019b880 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.956+0000 7f495f7fe700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f49500385e0 0x7f495003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.956+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f496404bc60 con 0x7f49701966b0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.956+0000 7f496e59c700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f49500385e0 0x7f495003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:18.961 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.956+0000 7f496e59c700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f49500385e0 0x7f495003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f496000f470 tx=0x7f496000f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:18.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:18.961+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4964010dc0 con 0x7f49701966b0 2024-05-08T23:13:19.235 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.234+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4970101000 con 0x7f49701966b0 2024-05-08T23:13:19.235 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.235+0000 7f495f7fe700 1 -- 172.21.15.189:0/878207285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f49640172c0 con 0x7f49701966b0 2024-05-08T23:13:19.236 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:13:19.236 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","modified":"2024-05-08T23:13:13.877464Z","created":"2024-05-08T23:11:52.906042Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T23:13:19.237 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f49500385e0 msgr2=0x7f495003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f49500385e0 0x7f495003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f496000f470 tx=0x7f496000f3c0 comp rx=0 tx=0).stop 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49701966b0 msgr2=0x7f497019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49701966b0 0x7f497019aa60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f496400b4d0 tx=0x7f496400b890 comp rx=0 tx=0).stop 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 shutdown_connections 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f49500385e0 0x7f495003aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49580035c0 0x7f4970196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 --2- 172.21.15.189:0/878207285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49701966b0 0x7f497019aa60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 >> 172.21.15.189:0/878207285 conn(0x7f4970076570 msgr2=0x7f49700fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 shutdown_connections 2024-05-08T23:13:19.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:19.237+0000 7f4974cb9700 1 -- 172.21.15.189:0/878207285 wait complete. 2024-05-08T23:13:19.239 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: mon.a calling monitor election 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: mon.c calling monitor election 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:19.258 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: monmap e2: 2 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: fsmap 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: mgrmap e13: x(active, since 36s) 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: overall HEALTH_OK 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:19.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:19.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:19.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:13:19.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: mon.a calling monitor election 2024-05-08T23:13:19.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:19.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: mon.c calling monitor election 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: monmap e2: 2 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: fsmap 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: mgrmap e13: x(active, since 36s) 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: overall HEALTH_OK 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:19.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:19.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:19.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:20.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:19 smithi016 ceph-mon[42541]: Deploying daemon mon.b on smithi155 2024-05-08T23:13:20.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:19 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/878207285' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:13:20.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:20.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:19 smithi189 ceph-mon[46919]: Deploying daemon mon.b on smithi155 2024-05-08T23:13:20.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:19 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/878207285' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:13:20.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:20.612 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T23:13:20.612 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T23:13:20.841 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:13:21.256 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:20 smithi189 ceph-mon[46919]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:20 smithi016 ceph-mon[42541]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 systemd[1]: Started Ceph mon.b for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:13:21.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.wal_dir: 2024-05-08T23:13:21.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.write_buffer_manager: 0x55e5bdeaa320 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T23:13:21.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.unordered_write: 0 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.row_cache: None 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.wal_filter: None 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T23:13:21.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.two_write_queues: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.wal_compression: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.atomic_flush: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.log_readahead_size: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T23:13:21.969 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_background_jobs: 2 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_background_compactions: -1 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_subcompactions: 1 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_open_files: -1 2024-05-08T23:13:21.970 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_background_flushes: -1 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Compression algorithms supported: 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kZSTD supported: 0 2024-05-08T23:13:21.971 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kXpressCompression supported: 0 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kZlibCompression supported: 1 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kSnappyCompression supported: 1 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kLZ4Compression supported: 1 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: kBZip2Compression supported: 0 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.merge_operator: 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_filter: None 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T23:13:21.972 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5bec15620) 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: cache_index_and_filter_blocks: 1 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: pin_top_level_index_and_filter: 1 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_type: 0 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: data_block_index_type: 0 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_shortening: 1 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: checksum: 4 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: no_block_cache: 0 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache: 0x55e5bdf2d1f0 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_name: BinnedLRUCache 2024-05-08T23:13:21.973 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_options: 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: capacity : 536870912 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: num_shard_bits : 4 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: strict_capacity_limit : 0 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: high_pri_pool_ratio: 0.000 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_cache_compressed: (nil) 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: persistent_cache: (nil) 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_size: 4096 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_size_deviation: 10 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_restart_interval: 16 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: index_block_restart_interval: 1 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: metadata_block_size: 4096 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: partition_filters: 0 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: use_delta_encoding: 1 2024-05-08T23:13:21.974 INFO:journalctl@ceph.mon.b.smithi155.stdout: filter_policy: bloomfilter 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: whole_key_filtering: 1 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: verify_compression: 0 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: read_amp_bytes_per_bit: 0 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: format_version: 5 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: enable_index_compression: 1 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: block_align: 0 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: max_auto_readahead_size: 262144 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: prepopulate_block_cache: 0 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: initial_auto_readahead_size: 8192 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression: NoCompression 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T23:13:21.975 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.num_levels: 7 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:13:21.976 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T23:13:21.977 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T23:13:21.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T23:13:21.979 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.table_properties_collectors: 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.inplace_update_support: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.bloom_locality: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.max_successive_merges: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.ttl: 2592000 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T23:13:21.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.enable_blob_files: false 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.min_blob_size: 0 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T23:13:21.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8dd117cd-1e48-4c72-ae75-638ede2c1e2c 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715210001488510, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715210001489104, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715210001, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8dd117cd-1e48-4c72-ae75-638ede2c1e2c", "db_session_id": "W8IDRWGZV2TP29J4Z38M", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715210001489205, "job": 1, "event": "recovery_finished"} 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T23:13:21.982 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e5bdfca000 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: DB pointer 0x55e5bdfb8000 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** DB Stats ** 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2024-05-08T23:13:21.983 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** Compaction Stats [default] ** 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** Compaction Stats [default] ** 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T23:13:21.984 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Block cache BinnedLRUCache@0x55e5bdf2d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2024-05-08T23:13:21.985 INFO:journalctl@ceph.mon.b.smithi155.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: using public_addr v2:172.21.15.155:0/0 -> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: starting mon.b rank -1 at public addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] at bind addrs [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(???) e0 preinit fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout: e1 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout: legacy client fscid: -1 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout: 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout: No filesystems configured 2024-05-08T23:13:21.986 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/949095894' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:13:21.987 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.a calling monitor election 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.c calling monitor election 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: monmap e2: 2 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:21.988 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: fsmap 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mgrmap e13: x(active, since 36s) 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: overall HEALTH_OK 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: Deploying daemon mon.b on smithi155 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/878207285' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:21.989 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:21.990 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:21 smithi155 ceph-mon[47105]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: mon.a calling monitor election 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: mon.c calling monitor election 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: fsmap 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: mgrmap e13: x(active, since 44s) 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: mon.a calling monitor election 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: mon.c calling monitor election 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:13:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: fsmap 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: mgrmap e13: x(active, since 44s) 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:26.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 -- 172.21.15.189:0/3326848755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 msgr2=0x7f02a0008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 --2- 172.21.15.189:0/3326848755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02a0008710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02b0101aa0 tx=0x7f02a400fe90 comp rx=0 tx=0).stop 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 -- 172.21.15.189:0/3326848755 shutdown_connections 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 --2- 172.21.15.189:0/3326848755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02a0008710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 -- 172.21.15.189:0/3326848755 >> 172.21.15.189:0/3326848755 conn(0x7f02b00fc4f0 msgr2=0x7f02b00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 -- 172.21.15.189:0/3326848755 shutdown_connections 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.810+0000 7f02b6d78700 1 -- 172.21.15.189:0/3326848755 wait complete. 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 Processor -- start 2024-05-08T23:13:27.811 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 -- start start 2024-05-08T23:13:27.812 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02b01963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:27.812 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 0x7f02b019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0196fd0 con 0x7f02a00062a0 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01a2f40 con 0x7f02b01968e0 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.811+0000 7f02b6d78700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01a3080 con 0x7f02b019b1d0 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b5315700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02b01963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02b01963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.189:40574/0 (socket says 172.21.15.189:40574) 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 -- 172.21.15.189:0/2992516046 learned_addr learned my addr 172.21.15.189:0/2992516046 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b5315700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52530/0 (socket says 172.21.15.189:52530) 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02affff700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 0x7f02b019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b5315700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 msgr2=0x7f02b019b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b5315700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 msgr2=0x7f02b019b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:13:27.813 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b5315700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02affff700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 msgr2=0x7f02b019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02affff700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 msgr2=0x7f02b019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02affff700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 0x7f02b019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b5315700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02affff700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 0x7f02b019ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 msgr2=0x7f02b019b680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 msgr2=0x7f02b019ac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 0x7f02b019ac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 -- 172.21.15.189:0/2992516046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a400f8e0 con 0x7f02a00062a0 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b4b14700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02b01963a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02a4010010 tx=0x7f02a401dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4011440 con 0x7f02a00062a0 2024-05-08T23:13:27.814 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b01a3300 con 0x7f02a00062a0 2024-05-08T23:13:27.815 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02a4003680 con 0x7f02a00062a0 2024-05-08T23:13:27.815 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.812+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4011870 con 0x7f02a00062a0 2024-05-08T23:13:27.815 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.813+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b01a3850 con 0x7f02a00062a0 2024-05-08T23:13:27.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.813+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b0102120 con 0x7f02a00062a0 2024-05-08T23:13:27.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.814+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f02a40119d0 con 0x7f02a00062a0 2024-05-08T23:13:27.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.814+0000 7f02adffb700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0294038c20 0x7f029403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:27.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.814+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02a4000e40 con 0x7f02a00062a0 2024-05-08T23:13:27.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.814+0000 7f02affff700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0294038c20 0x7f029403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:27.816 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.815+0000 7f02affff700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0294038c20 0x7f029403b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f029c009850 tx=0x7f029c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:27.818 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:27.819+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f02a402f360 con 0x7f02a00062a0 2024-05-08T23:13:27.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:27.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.106+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f02b01085a0 con 0x7f02a00062a0 2024-05-08T23:13:28.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.106+0000 7f02adffb700 1 -- 172.21.15.189:0/2992516046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f02a4025b30 con 0x7f02a00062a0 2024-05-08T23:13:28.106 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-05-08T23:13:28.106 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","modified":"2024-05-08T23:13:21.519491Z","created":"2024-05-08T23:11:52.906042Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:3300","nonce":0},{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0294038c20 msgr2=0x7f029403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0294038c20 0x7f029403b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f029c009850 tx=0x7f029c008040 comp rx=0 tx=0).stop 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 msgr2=0x7f02b01963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02b01963a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02a4010010 tx=0x7f02a401dae0 comp rx=0 tx=0).stop 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 shutdown_connections 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0294038c20 0x7f029403b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:28.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02b019b1d0 0x7f02b019b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:28.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a00062a0 0x7f02b01963a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:28.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 --2- 172.21.15.189:0/2992516046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02b01968e0 0x7f02b019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:28.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 >> 172.21.15.189:0/2992516046 conn(0x7f02b00fc4f0 msgr2=0x7f02b01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:28.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 shutdown_connections 2024-05-08T23:13:28.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:28.108+0000 7f02b6d78700 1 -- 172.21.15.189:0/2992516046 wait complete. 2024-05-08T23:13:28.109 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-05-08T23:13:28.509 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T23:13:28.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T23:13:28.924 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:13:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:13:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: mon.a calling monitor election 2024-05-08T23:13:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: mon.c calling monitor election 2024-05-08T23:13:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: fsmap 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: mgrmap e13: x(active, since 44s) 2024-05-08T23:13:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: mon.b (rank 2) addr [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] is down (out of quorum) 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:28.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:28.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:28.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:28.968 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.594 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: mon.b calling monitor election 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: Reconfiguring daemon mon.a on smithi016 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: mon.b calling monitor election 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: mon.c calling monitor election 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: mon.a calling monitor election 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: fsmap 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: mgrmap e13: x(active, since 46s) 2024-05-08T23:13:29.595 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: Cluster is now healthy 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: overall HEALTH_OK 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:13:29.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:29.624 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: mon.b calling monitor election 2024-05-08T23:13:29.624 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T23:13:29.624 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: Reconfiguring daemon mon.a on smithi016 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: mon.b calling monitor election 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: mon.c calling monitor election 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: mon.a calling monitor election 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: fsmap 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: mgrmap e13: x(active, since 46s) 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: Cluster is now healthy 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: overall HEALTH_OK 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:29.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:13:29.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: mon.b calling monitor election 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: Reconfiguring daemon mon.a on smithi016 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: mon.b calling monitor election 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: mon.c calling monitor election 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: mon.a calling monitor election 2024-05-08T23:13:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: monmap e3: 3 mons at {a=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],b=[v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: fsmap 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: mgrmap e13: x(active, since 46s) 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: Cluster is now healthy 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: overall HEALTH_OK 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:13:29.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 -- 172.21.15.16:0/3776919538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069250 msgr2=0x7fc5080696d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:30.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 --2- 172.21.15.16:0/3776919538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069250 0x7fc5080696d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80099c0 tx=0x7fc4f8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 -- 172.21.15.16:0/3776919538 shutdown_connections 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 --2- 172.21.15.16:0/3776919538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508069c10 0x7fc50810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 --2- 172.21.15.16:0/3776919538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508106fe0 0x7fc5081073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 --2- 172.21.15.16:0/3776919538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069250 0x7fc5080696d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.032+0000 7fc50f66e700 1 -- 172.21.15.16:0/3776919538 >> 172.21.15.16:0/3776919538 conn(0x7fc508076e70 msgr2=0x7fc508077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 -- 172.21.15.16:0/3776919538 shutdown_connections 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 -- 172.21.15.16:0/3776919538 wait complete. 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 Processor -- start 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 -- start start 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508069250 0x7fc508197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 0x7fc508197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508106fe0 0x7fc50819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50819c200 con 0x7fc508069c10 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50819c370 con 0x7fc508106fe0 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.033+0000 7fc50f66e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50819c530 con 0x7fc508069250 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 0x7fc508197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 0x7fc508197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33428/0 (socket says 172.21.15.16:33428) 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 -- 172.21.15.16:0/2942833135 learned_addr learned my addr 172.21.15.16:0/2942833135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50dc0b700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508106fe0 0x7fc50819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50d40a700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508069250 0x7fc508197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 -- 172.21.15.16:0/2942833135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508069250 msgr2=0x7fc508197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508069250 0x7fc508197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 -- 172.21.15.16:0/2942833135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508106fe0 msgr2=0x7fc50819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508106fe0 0x7fc50819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 -- 172.21.15.16:0/2942833135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80095f0 con 0x7fc508069c10 2024-05-08T23:13:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50dc0b700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508106fe0 0x7fc50819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50d40a700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508069250 0x7fc508197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50cc09700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 0x7fc508197840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009f50 tx=0x7fc4f800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f801d070 con 0x7fc508069c10 2024-05-08T23:13:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4f800fe10 con 0x7fc508069c10 2024-05-08T23:13:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50819c7b0 con 0x7fc508069c10 2024-05-08T23:13:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80173f0 con 0x7fc508069c10 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.034+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5081a4070 con 0x7fc508069c10 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.036+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fc4f800f980 con 0x7fc508069c10 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.036+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5081094e0 con 0x7fc508069c10 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.036+0000 7fc4fe7fc700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc4f40389d0 0x7fc4f403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.036+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc4f804c720 con 0x7fc508069c10 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.036+0000 7fc50d40a700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc4f40389d0 0x7fc4f403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.036+0000 7fc50d40a700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc4f40389d0 0x7fc4f403ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5040099d0 tx=0x7fc504008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.041+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4f8022650 con 0x7fc508069c10 2024-05-08T23:13:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.257+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc50804ec90 con 0x7fc508069c10 2024-05-08T23:13:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.257+0000 7fc4fe7fc700 1 -- 172.21.15.16:0/2942833135 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fc4f8027020 con 0x7fc508069c10 2024-05-08T23:13:30.258 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:13:30.258 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-05-08T23:13:30.258 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:13:30.258 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-05-08T23:13:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc4f40389d0 msgr2=0x7fc4f403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc4f40389d0 0x7fc4f403ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5040099d0 tx=0x7fc504008040 comp rx=0 tx=0).stop 2024-05-08T23:13:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 msgr2=0x7fc508197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 0x7fc508197840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009f50 tx=0x7fc4f800f510 comp rx=0 tx=0).stop 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 shutdown_connections 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc508106fe0 0x7fc50819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc4f40389d0 0x7fc4f403ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc508069250 0x7fc508197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 --2- 172.21.15.16:0/2942833135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc508069c10 0x7fc508197840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 >> 172.21.15.16:0/2942833135 conn(0x7fc508076e70 msgr2=0x7fc5080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 shutdown_connections 2024-05-08T23:13:30.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:30.260+0000 7fc50f66e700 1 -- 172.21.15.16:0/2942833135 wait complete. 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: Reconfiguring mon.b (monmap changed)... 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: Reconfiguring daemon mon.b on smithi155 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:30 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2942833135' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: Reconfiguring mon.b (monmap changed)... 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: Reconfiguring daemon mon.b on smithi155 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.569 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:30 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2942833135' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.628 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T23:13:30.628 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:13:30.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T23:13:30.666 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:13:30.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: Reconfiguring mon.b (monmap changed)... 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: Reconfiguring daemon mon.b on smithi155 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T23:13:30.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:30 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2942833135' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:30.741 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:13:30.742 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T23:13:30.776 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:13:30.776 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:13:30.845 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:13:30.845 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T23:13:30.875 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:13:30.876 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:13:30.945 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-05-08T23:13:30.945 INFO:tasks.cephadm:Adding mgr.y on smithi155 2024-05-08T23:13:30.945 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-05-08T23:13:30.945 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi155=y;smithi189=z' 2024-05-08T23:13:31.199 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: Reconfiguring mon.c (monmap changed)... 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: Reconfiguring daemon mon.c on smithi189 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:31.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.627 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: Reconfiguring mon.c (monmap changed)... 2024-05-08T23:13:31.627 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: Reconfiguring daemon mon.c on smithi189 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:31.628 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: Reconfiguring mon.c (monmap changed)... 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: Reconfiguring daemon mon.c on smithi189 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:31.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:32.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 -- 172.21.15.189:0/1639100452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444102200 msgr2=0x7f4444102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 --2- 172.21.15.189:0/1639100452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444102200 0x7f4444102660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4434009990 tx=0x7f4434009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 -- 172.21.15.189:0/1639100452 shutdown_connections 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 --2- 172.21.15.189:0/1639100452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4444102ba0 0x7f4444109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 --2- 172.21.15.189:0/1639100452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4444101000 0x7f4444101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 --2- 172.21.15.189:0/1639100452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444102200 0x7f4444102660 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 -- 172.21.15.189:0/1639100452 >> 172.21.15.189:0/1639100452 conn(0x7f44440fc5c0 msgr2=0x7f44440fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 -- 172.21.15.189:0/1639100452 shutdown_connections 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.392+0000 7f4448806700 1 -- 172.21.15.189:0/1639100452 wait complete. 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 Processor -- start 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 -- start start 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 0x7f4444196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4444102200 0x7f4444197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4444102ba0 0x7f444419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444419e9b0 con 0x7f4444102ba0 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444419eaf0 con 0x7f4444102200 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4448806700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444419ec80 con 0x7f4444101000 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 0x7f4444196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 0x7f4444196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:52540/0 (socket says 172.21.15.189:52540) 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 -- 172.21.15.189:0/3408982921 learned_addr learned my addr 172.21.15.189:0/3408982921 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4441d9b700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4444102200 0x7f4444197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f4442d9d700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4444102ba0 0x7f444419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 -- 172.21.15.189:0/3408982921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4444102200 msgr2=0x7f4444197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4444102200 0x7f4444197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 -- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4444102ba0 msgr2=0x7f444419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.393+0000 7f444259c700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4444102ba0 0x7f444419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f444259c700 1 -- 172.21.15.189:0/3408982921 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340095f0 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f444259c700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 0x7f4444196ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f443800e9a0 tx=0x7f443800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438009790 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f44380044b0 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444419ef60 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438003c60 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.394+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444419f370 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.395+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4438018430 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.395+0000 7f44337fe700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f442c0389d0 0x7f442c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.395+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44441085d0 con 0x7f4444101000 2024-05-08T23:13:32.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.395+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4438014070 con 0x7f4444101000 2024-05-08T23:13:32.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.395+0000 7f4441d9b700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f442c0389d0 0x7f442c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:32.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.396+0000 7f4441d9b700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f442c0389d0 0x7f442c03ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f443400b2e0 tx=0x7f4434005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:32.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.400+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4438003dc0 con 0x7f4444101000 2024-05-08T23:13:32.606 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.606+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi155=y;smithi189=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f444402ceb0 con 0x7f442c0389d0 2024-05-08T23:13:32.613 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.613+0000 7f44337fe700 1 -- 172.21.15.189:0/3408982921 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f444402ceb0 con 0x7f442c0389d0 2024-05-08T23:13:32.613 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f442c0389d0 msgr2=0x7f442c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f442c0389d0 0x7f442c03ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f443400b2e0 tx=0x7f4434005fd0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 msgr2=0x7f4444196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 0x7f4444196ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f443800e9a0 tx=0x7f443800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 shutdown_connections 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4444102ba0 0x7f444419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f442c0389d0 0x7f442c03ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4444101000 0x7f4444196ec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 --2- 172.21.15.189:0/3408982921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4444102200 0x7f4444197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:32.615 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 >> 172.21.15.189:0/3408982921 conn(0x7f44440fc5c0 msgr2=0x7f4444105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:32.616 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 shutdown_connections 2024-05-08T23:13:32.616 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:13:32.615+0000 7f4448806700 1 -- 172.21.15.189:0/3408982921 wait complete. 2024-05-08T23:13:32.992 DEBUG:teuthology.orchestra.run.smithi155:mgr.y> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T23:13:32.996 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T23:13:32.998 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T23:13:32.998 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:13:32.998 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:13:33.021 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:13:33.021 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-05-08T23:13:33.028 INFO:journalctl@ceph.mgr.y.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:13:33.031 INFO:journalctl@ceph.mgr.z.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:13:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:13:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:33.084 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 184374 Links: 1 2024-05-08T23:13:33.084 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:33.084 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:33.084 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-08 23:12:51.956262996 +0000 2024-05-08T23:13:33.085 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-08 23:05:54.073612442 +0000 2024-05-08T23:13:33.085 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-08 23:05:54.073612442 +0000 2024-05-08T23:13:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-05-08T23:13:33.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:13:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-08T23:13:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-08T23:13:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000157822 s, 3.2 MB/s 2024-05-08T23:13:33.160 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:13:33.224 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 182629 Links: 1 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-08 23:12:52.023261983 +0000 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-08 23:05:54.431606929 +0000 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-08 23:05:54.431606929 +0000 2024-05-08T23:13:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-05-08T23:13:33.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:13:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-08T23:13:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-08T23:13:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000124354 s, 4.1 MB/s 2024-05-08T23:13:33.361 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:13:33.425 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 173939 Links: 1 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-08 23:12:52.095260894 +0000 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-08 23:05:54.787601447 +0000 2024-05-08T23:13:33.489 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-08 23:05:54.787601447 +0000 2024-05-08T23:13:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-05-08T23:13:33.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:13:33.563 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-08T23:13:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-08T23:13:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000137878 s, 3.7 MB/s 2024-05-08T23:13:33.565 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:13:33.629 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-05-08T23:13:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:13:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:33.691 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 177862 Links: 1 2024-05-08T23:13:33.691 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:33.691 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:33.691 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-08 23:12:52.172259729 +0000 2024-05-08T23:13:33.692 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-08 23:05:55.156595765 +0000 2024-05-08T23:13:33.692 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-08 23:05:55.156595765 +0000 2024-05-08T23:13:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-05-08T23:13:33.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:13:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-05-08T23:13:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-05-08T23:13:33.790 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000135489 s, 3.8 MB/s 2024-05-08T23:13:33.792 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:13:33.855 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-05-08T23:13:33.856 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:13:33.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:33.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi155=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:33.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: Saving service mgr spec with placement smithi016=x;smithi155=y;smithi189=z;count:3 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:33.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:33 smithi016 ceph-mon[42541]: Deploying daemon mgr.z on smithi189 2024-05-08T23:13:33.877 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:13:33.877 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi155=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: Saving service mgr spec with placement smithi016=x;smithi155=y;smithi189=z;count:3 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:33.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:13:33.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:33.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:33.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:33 smithi189 ceph-mon[46919]: Deploying daemon mgr.z on smithi189 2024-05-08T23:13:33.937 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:13:33.937 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:33.937 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 180691 Links: 1 2024-05-08T23:13:33.938 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:33.938 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:33.938 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-08 23:13:21.104217428 +0000 2024-05-08T23:13:33.938 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-08 23:05:11.749356400 +0000 2024-05-08T23:13:33.938 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-08 23:05:11.749356400 +0000 2024-05-08T23:13:33.938 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-08T23:13:33.938 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi155=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: Saving service mgr spec with placement smithi016=x;smithi155=y;smithi189=z;count:3 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:13:33.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:33.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:33.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:33 smithi155 ceph-mon[47105]: Deploying daemon mgr.z on smithi189 2024-05-08T23:13:33.994 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-08T23:13:33.994 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-08T23:13:33.994 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000219513 s, 2.3 MB/s 2024-05-08T23:13:33.995 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:13:34.055 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-05-08T23:13:34.116 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:13:34.116 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:34.116 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 173044 Links: 1 2024-05-08T23:13:34.117 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:34.117 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:34.117 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-08 23:13:21.104217428 +0000 2024-05-08T23:13:34.117 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-08 23:05:12.084355598 +0000 2024-05-08T23:13:34.117 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-08 23:05:12.084355598 +0000 2024-05-08T23:13:34.117 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-08T23:13:34.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:13:34.184 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-08T23:13:34.184 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-08T23:13:34.184 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000221438 s, 2.3 MB/s 2024-05-08T23:13:34.186 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:13:34.245 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-05-08T23:13:34.277 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:34 smithi189 systemd[1]: Starting Ceph mgr.z for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:13:34.303 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:13:34.303 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:34.304 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 178965 Links: 1 2024-05-08T23:13:34.304 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:34.304 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:34.304 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-08 23:13:21.105217425 +0000 2024-05-08T23:13:34.304 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-08 23:05:12.436354756 +0000 2024-05-08T23:13:34.304 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-08 23:05:12.436354756 +0000 2024-05-08T23:13:34.305 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-08T23:13:34.305 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:13:34.370 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-08T23:13:34.370 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-08T23:13:34.370 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000162929 s, 3.1 MB/s 2024-05-08T23:13:34.372 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:13:34.430 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-05-08T23:13:34.487 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:13:34.488 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:34.488 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 177957 Links: 1 2024-05-08T23:13:34.488 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:34.488 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:34.488 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-05-08 23:13:21.105217425 +0000 2024-05-08T23:13:34.488 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-05-08 23:05:12.762353976 +0000 2024-05-08T23:13:34.489 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-05-08 23:05:12.762353976 +0000 2024-05-08T23:13:34.489 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-05-08T23:13:34.489 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:13:34.554 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-05-08T23:13:34.554 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-05-08T23:13:34.554 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000211786 s, 2.4 MB/s 2024-05-08T23:13:34.555 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:13:34.614 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-05-08T23:13:34.614 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T23:13:34.634 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T23:13:34.634 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-05-08T23:13:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T23:13:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:34.694 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 181169 Links: 1 2024-05-08T23:13:34.694 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:34.694 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:34.695 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-05-08 23:13:33.988261447 +0000 2024-05-08T23:13:34.695 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-05-08 23:05:02.923040627 +0000 2024-05-08T23:13:34.695 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-05-08 23:05:02.923040627 +0000 2024-05-08T23:13:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-05-08T23:13:34.695 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T23:13:34.763 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:34 smithi189 ceph-mgr[48803]: -- 172.21.15.189:0/1546244144 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce9fdd41a0 con 0x55ce9fdaa800 2024-05-08T23:13:34.763 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:34 smithi189 bash[48766]: cc1458c838708d49673dadb07dc70472f2ea6909e9b46ee35cb7f4bc11896852 2024-05-08T23:13:34.763 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:34 smithi189 systemd[1]: Started Ceph mgr.z for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:13:34.766 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-05-08T23:13:34.766 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-05-08T23:13:34.766 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00019919 s, 2.6 MB/s 2024-05-08T23:13:34.767 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T23:13:34.825 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-05-08T23:13:34.883 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 182507 Links: 1 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-05-08 23:13:33.989261428 +0000 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-05-08 23:05:03.308033307 +0000 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-05-08 23:05:03.308033307 +0000 2024-05-08T23:13:34.884 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-05-08T23:13:34.884 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T23:13:34.947 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:34.859+0000 7fb898dee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:13:34.950 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-05-08T23:13:34.950 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-05-08T23:13:34.950 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000226313 s, 2.3 MB/s 2024-05-08T23:13:34.951 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T23:13:35.011 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-05-08T23:13:35.050 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:34.958+0000 7fb898dee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:13:35.069 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T23:13:35.069 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:35.069 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 177127 Links: 1 2024-05-08T23:13:35.069 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:35.069 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:35.070 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-05-08 23:13:33.989261428 +0000 2024-05-08T23:13:35.070 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-05-08 23:05:03.655026710 +0000 2024-05-08T23:13:35.070 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-05-08 23:05:03.655026710 +0000 2024-05-08T23:13:35.070 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-05-08T23:13:35.070 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T23:13:35.137 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-05-08T23:13:35.137 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-05-08T23:13:35.137 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000192329 s, 2.7 MB/s 2024-05-08T23:13:35.137 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T23:13:35.195 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-05-08T23:13:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T23:13:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T23:13:35.252 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 182612 Links: 1 2024-05-08T23:13:35.252 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:13:35.253 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T23:13:35.253 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-05-08 23:13:33.989261428 +0000 2024-05-08T23:13:35.253 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-05-08 23:05:03.991020322 +0000 2024-05-08T23:13:35.253 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-05-08 23:05:03.991020322 +0000 2024-05-08T23:13:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-05-08T23:13:35.253 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T23:13:35.316 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:35.051+0000 7fb898dee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:13:35.316 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:35.143+0000 7fb898dee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:13:35.318 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-05-08T23:13:35.318 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-05-08T23:13:35.318 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000234104 s, 2.2 MB/s 2024-05-08T23:13:35.320 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T23:13:35.380 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-05-08T23:13:35.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T23:13:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:13:35.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:35.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:35 smithi189 ceph-mon[46919]: Deploying daemon mgr.y on smithi155 2024-05-08T23:13:35.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:35.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:35 smithi016 ceph-mon[42541]: Deploying daemon mgr.y on smithi155 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:35 smithi155 systemd[1]: Starting Ceph mgr.y for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T23:13:35.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:35.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:35.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:35 smithi155 ceph-mon[47105]: Deploying daemon mgr.y on smithi155 2024-05-08T23:13:36.050 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:35.800+0000 7fb898dee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:13:36.409 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:36.050+0000 7fb898dee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:13:36.409 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:36.136+0000 7fb898dee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:13:36.409 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:36.246+0000 7fb898dee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:13:36.464 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 ceph-mgr[48614]: -- 172.21.15.155:0/606023428 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f67ec761a0 con 0x55f67ec4c800 2024-05-08T23:13:36.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 bash[48576]: c7fcb23cddc6253c70111500b3137d15d42a7332d35166797a417b6de8aecef2 2024-05-08T23:13:36.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 systemd[1]: Started Ceph mgr.y for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:13:36.633 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:36.536+0000 7fcdda034200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:13:36.668 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:36.409+0000 7fb898dee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:13:36.669 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:36.549+0000 7fb898dee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:13:36.669 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:36.639+0000 7fb898dee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:13:36.831 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:36.633+0000 7fcdda034200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:13:36.831 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:36.735+0000 7fcdda034200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:13:36.831 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:36.831+0000 7fcdda034200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:13:36.972 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T23:13:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T23:13:36.973 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-08T23:13:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-08T23:13:36.973 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0091031 s, 1.2 GB/s 2024-05-08T23:13:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-08T23:13:37.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:37.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:37 smithi016 ceph-mon[42541]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:37.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-05-08T23:13:37.419 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:37.055+0000 7fb898dee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:13:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:37.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:37 smithi189 ceph-mon[46919]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:37.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:37 smithi155 ceph-mon[47105]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:13:37.754 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:37.502+0000 7fcdda034200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:13:37.762 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:37.475+0000 7fb898dee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:13:37.762 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:37.671+0000 7fb898dee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:13:38.105 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:37.754+0000 7fcdda034200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:13:38.105 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:37.839+0000 7fcdda034200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:13:38.106 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:37.947+0000 7fcdda034200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:13:38.106 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:38.104+0000 7fcdda034200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:13:38.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:37.763+0000 7fb898dee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:38.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:38.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: Reconfiguring daemon mgr.x on smithi016 2024-05-08T23:13:38.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:38.233+0000 7fcdda034200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:38.318+0000 7fcdda034200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: Reconfiguring daemon mgr.x on smithi016 2024-05-08T23:13:38.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.526 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.526 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: Reconfiguring daemon mgr.x on smithi016 2024-05-08T23:13:38.527 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:38.778 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:38.526+0000 7fb898dee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:13:38.778 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:38.626+0000 7fb898dee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:13:38.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 -- 172.21.15.16:0/1756844628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 msgr2=0x7f5db81014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 --2- 172.21.15.16:0/1756844628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81014e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009990 tx=0x7f5da0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 -- 172.21.15.16:0/1756844628 shutdown_connections 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 --2- 172.21.15.16:0/1756844628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 0x7f5db8108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 --2- 172.21.15.16:0/1756844628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 0x7f5db8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 --2- 172.21.15.16:0/1756844628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81014e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 -- 172.21.15.16:0/1756844628 >> 172.21.15.16:0/1756844628 conn(0x7f5db80fb4a0 msgr2=0x7f5db80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 -- 172.21.15.16:0/1756844628 shutdown_connections 2024-05-08T23:13:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.817+0000 7f5dbd480700 1 -- 172.21.15.16:0/1756844628 wait complete. 2024-05-08T23:13:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 Processor -- start 2024-05-08T23:13:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 -- start start 2024-05-08T23:13:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 0x7f5db8103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 0x7f5db8102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db81031b0 con 0x7f5db8101060 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8103320 con 0x7f5db80fff00 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dbd480700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db81a0ef0 con 0x7f5db8101a20 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35452/0 (socket says 172.21.15.16:35452) 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5db6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 0x7f5db8103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.818+0000 7f5dae7fc700 1 -- 172.21.15.16:0/3501472243 learned_addr learned my addr 172.21.15.16:0/3501472243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5db77fe700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 0x7f5db8102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dae7fc700 1 -- 172.21.15.16:0/3501472243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 msgr2=0x7f5db8102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dae7fc700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 0x7f5db8102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dae7fc700 1 -- 172.21.15.16:0/3501472243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 msgr2=0x7f5db8103e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dae7fc700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 0x7f5db8103e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dae7fc700 1 -- 172.21.15.16:0/3501472243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00095f0 con 0x7f5db8101060 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5db77fe700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 0x7f5db8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:13:38.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5db6ffd700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 0x7f5db8103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:13:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dae7fc700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81024d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006010 tx=0x7f5da0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da001d070 con 0x7f5db8101060 2024-05-08T23:13:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5da000bd50 con 0x7f5db8101060 2024-05-08T23:13:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da000fd90 con 0x7f5db8101060 2024-05-08T23:13:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db81a1120 con 0x7f5db8101060 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.819+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db81a1530 con 0x7f5db8101060 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.820+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f5da002b440 con 0x7f5db8101060 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.820+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db804ec60 con 0x7f5db8101060 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.820+0000 7f5db4ff9700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5da40389d0 0x7f5da403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.820+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5da004c6e0 con 0x7f5db8101060 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.821+0000 7f5db6ffd700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5da40389d0 0x7f5da403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.821+0000 7f5db6ffd700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5da40389d0 0x7f5da403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5da8011490 tx=0x7f5da8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:38.825+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5da0029950 con 0x7f5db8101060 2024-05-08T23:13:38.965 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:38.692+0000 7fcdda034200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:13:39.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:39.048+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5db8069060 con 0x7f5da40389d0 2024-05-08T23:13:39.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:38.778+0000 7fb898dee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:13:39.368 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:39.084+0000 7fcdda034200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:13:39.369 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:39.278+0000 7fcdda034200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:13:39.369 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:39.369+0000 7fcdda034200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:13:39.668 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:39.412+0000 7fb898dee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:39.594+0000 7fb898dee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:13:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:39.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:39.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.713 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:39.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:39.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:39.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:13:39.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:13:39.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:39.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:39.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:39.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:39.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:39.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:39.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:13:39.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:13:39.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:40.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:39.788+0000 7fb898dee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:13:40.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:40.128+0000 7fcdda034200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:13:40.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:40.224+0000 7fcdda034200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:13:40.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:40.370+0000 7fcdda034200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:13:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:40 smithi016 ceph-mon[42541]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:40.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:40 smithi189 ceph-mon[46919]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:40.919 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:40.718+0000 7fb898dee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:13:40.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:40 smithi155 ceph-mon[47105]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:41.284 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:41.003+0000 7fb898dee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:13:41.284 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:41.092+0000 7fb898dee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:13:41.284 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:13:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:13:41.284+0000 7fb898dee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:13:41.370 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:41.000+0000 7fcdda034200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:13:41.370 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:41.178+0000 7fcdda034200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:13:41.370 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:41.371+0000 7fcdda034200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4181965279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]: dispatch 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]: dispatch 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]': finished 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: Standby manager daemon z started 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:13:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:41 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:13:41.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:41.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4181965279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]: dispatch 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]: dispatch 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]': finished 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: Standby manager daemon z started 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:13:41.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:41 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:13:41.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4181965279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]: dispatch 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]: dispatch 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7c44106-3485-4f2e-b9ca-fdd6db13f996"}]': finished 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: Standby manager daemon z started 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:13:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:41 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/2369071235' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:13:41.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:41.947+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f5da00173f0 con 0x7f5db8101060 2024-05-08T23:13:42.581 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:42.299+0000 7fcdda034200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:13:42.857 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:42 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2909384972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:13:42.857 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:42 smithi155 ceph-mon[47105]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T23:13:42.857 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:13:42.857 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:42.581+0000 7fcdda034200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:13:42.857 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:42.668+0000 7fcdda034200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:13:42.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:42 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2909384972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:13:42.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:42 smithi016 ceph-mon[42541]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T23:13:42.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:13:42.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:42 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2909384972' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:13:42.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:42 smithi189 ceph-mon[46919]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T23:13:42.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:13:43.215 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:13:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:13:42.857+0000 7fcdda034200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:13:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:43.601+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f5da002bac0 con 0x7f5db8101060 2024-05-08T23:13:43.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:43 smithi016 ceph-mon[42541]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:43.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:43 smithi016 ceph-mon[42541]: Standby manager daemon y started 2024-05-08T23:13:43.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:43 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:13:43.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:43 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:13:43.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:43 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:13:43.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:43 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:13:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:43 smithi189 ceph-mon[46919]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:43 smithi189 ceph-mon[46919]: Standby manager daemon y started 2024-05-08T23:13:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:43 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:13:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:43 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:13:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:43 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:13:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:43 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:13:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:43 smithi155 ceph-mon[47105]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:43 smithi155 ceph-mon[47105]: Standby manager daemon y started 2024-05-08T23:13:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:43 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:13:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:43 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:13:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:43 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:13:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:43 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/129853781' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:13:44.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:44 smithi016 ceph-mon[42541]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-08T23:13:44.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:13:44.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:44 smithi189 ceph-mon[46919]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-08T23:13:44.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:13:44.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:44 smithi155 ceph-mon[47105]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-08T23:13:44.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:13:45.712 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:45 smithi016 ceph-mon[42541]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:45.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:45 smithi189 ceph-mon[46919]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:45.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:45 smithi155 ceph-mon[47105]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:47 smithi016 ceph-mon[42541]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T23:13:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:47 smithi016 ceph-mon[42541]: Deploying daemon osd.0 on smithi016 2024-05-08T23:13:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:47 smithi189 ceph-mon[46919]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T23:13:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:47 smithi189 ceph-mon[46919]: Deploying daemon osd.0 on smithi016 2024-05-08T23:13:47.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:47 smithi155 ceph-mon[47105]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:47.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T23:13:47.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:47 smithi155 ceph-mon[47105]: Deploying daemon osd.0 on smithi016 2024-05-08T23:13:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:49 smithi016 ceph-mon[42541]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:49 smithi189 ceph-mon[46919]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:49 smithi155 ceph-mon[47105]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:51 smithi016 ceph-mon[42541]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:51.815 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:51 smithi189 ceph-mon[46919]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:51 smithi155 ceph-mon[47105]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.792+0000 7f5db4ff9700 1 -- 172.21.15.16:0/3501472243 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5db8069060 con 0x7f5da40389d0 2024-05-08T23:13:52.793 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-05-08T23:13:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.795+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5da40389d0 msgr2=0x7f5da403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.795+0000 7f5dbd480700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5da40389d0 0x7f5da403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5da8011490 tx=0x7f5da8009260 comp rx=0 tx=0).stop 2024-05-08T23:13:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.795+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 msgr2=0x7f5db81024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.795+0000 7f5dbd480700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81024d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5da0006010 tx=0x7f5da0004730 comp rx=0 tx=0).stop 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.795+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 shutdown_connections 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5da40389d0 0x7f5da403ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5db8101a20 0x7f5db8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5db80fff00 0x7f5db8103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 --2- 172.21.15.16:0/3501472243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db8101060 0x7f5db81024d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 >> 172.21.15.16:0/3501472243 conn(0x7f5db80fb4a0 msgr2=0x7f5db8068950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 shutdown_connections 2024-05-08T23:13:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:13:52.796+0000 7f5dbd480700 1 -- 172.21.15.16:0/3501472243 wait complete. 2024-05-08T23:13:52.978 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:52.978 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:52.978 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:52.979 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:52.979 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:53.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:53.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.193 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T23:13:53.196 INFO:tasks.cephadm:Deploying osd.1 on smithi155 with /dev/vg_nvme/lv_4... 2024-05-08T23:13:53.196 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T23:13:53.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:53.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:53.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.231 INFO:journalctl@ceph.osd.0.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:13:53.424 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:13:53.847 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:53 smithi155 ceph-mon[47105]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:53.847 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.847 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.848 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:53.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:53 smithi016 ceph-mon[42541]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:53.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:53.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:54.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:53 smithi189 ceph-mon[46919]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:13:54.705 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T23:13:54.706 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T23:13:54.706 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-05-08T23:13:54.706 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-05-08T23:13:54.706 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125158 s, 838 MB/s 2024-05-08T23:13:54.706 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-05-08T23:13:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:54 smithi016 ceph-mon[42541]: from='osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:13:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:54 smithi016 ceph-mon[42541]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:13:54.864 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:13:54 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0[61610]: 2024-05-08T23:13:54.566+0000 7fb40a2d77c0 -1 osd.0 0 log_to_monitors true 2024-05-08T23:13:55.016 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:54 smithi155 ceph-mon[47105]: from='osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:13:55.017 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:54 smithi155 ceph-mon[47105]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:13:55.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-05-08T23:13:55.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:54 smithi189 ceph-mon[46919]: from='osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:13:55.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:54 smithi189 ceph-mon[46919]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T23:13:55.296 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:13:55.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:55.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:55.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:55.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:55.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:56.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:13:56.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:13:56.243 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 -- 172.21.15.155:0/2798153274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0108f00 msgr2=0x7f65f010b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/2798153274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0108f00 0x7f65f010b370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65e800b0a0 tx=0x7f65e800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 -- 172.21.15.155:0/2798153274 shutdown_connections 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/2798153274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0108f00 0x7f65f010b370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/2798153274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f0074f50 0x7f65f00733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/2798153274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f0073980 0x7f65f0073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 -- 172.21.15.155:0/2798153274 >> 172.21.15.155:0/2798153274 conn(0x7f65f00fc5a0 msgr2=0x7f65f00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 -- 172.21.15.155:0/2798153274 shutdown_connections 2024-05-08T23:13:56.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.244+0000 7f65f4b8d700 1 -- 172.21.15.155:0/2798153274 wait complete. 2024-05-08T23:13:56.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 Processor -- start 2024-05-08T23:13:56.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 -- start start 2024-05-08T23:13:56.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 0x7f65f0199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f0074f50 0x7f65f0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f0108f00 0x7f65f01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f01a0c80 con 0x7f65f0074f50 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f01a0dc0 con 0x7f65f0073980 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65f4b8d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f01a0f50 con 0x7f65f0108f00 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65ee59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 0x7f65f0199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65ee59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 0x7f65f0199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:55290/0 (socket says 172.21.15.155:55290) 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65ee59c700 1 -- 172.21.15.155:0/3903646951 learned_addr learned my addr 172.21.15.155:0/3903646951 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65edd9b700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f0074f50 0x7f65f0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65eed9d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f0108f00 0x7f65f01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.245+0000 7f65ee59c700 1 -- 172.21.15.155:0/3903646951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f0108f00 msgr2=0x7f65f01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65ee59c700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f0108f00 0x7f65f01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65ee59c700 1 -- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f0074f50 msgr2=0x7f65f0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:13:56.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65ee59c700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f0074f50 0x7f65f0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65ee59c700 1 -- 172.21.15.155:0/3903646951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8009d00 con 0x7f65f0073980 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65ee59c700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 0x7f65f0199140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f65e400d950 tx=0x7f65e400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400f890 con 0x7f65f0073980 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65e400fe80 con 0x7f65f0073980 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f01a1230 con 0x7f65f0073980 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e400e3b0 con 0x7f65f0073980 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.246+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f01a1730 con 0x7f65f0073980 2024-05-08T23:13:56.248 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.247+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f0107d10 con 0x7f65f0073980 2024-05-08T23:13:56.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.248+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f65e4010460 con 0x7f65f0073980 2024-05-08T23:13:56.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.248+0000 7f65df7fe700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65d80a1090 0x7f65d80a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:13:56.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.249+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f65e40caa90 con 0x7f65f0073980 2024-05-08T23:13:56.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.249+0000 7f65edd9b700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65d80a1090 0x7f65d80a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:13:56.249 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.249+0000 7f65edd9b700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65d80a1090 0x7f65d80a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005950 tx=0x7f65e000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:13:56.252 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.253+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f65e4095db0 con 0x7f65f0073980 2024-05-08T23:13:56.450 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:13:56.451+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f65f00613e0 con 0x7f65d80a1090 2024-05-08T23:13:56.614 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:13:56 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0[61610]: 2024-05-08T23:13:56.296+0000 7fb40013c700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T23:13:56.614 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:13:56 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0[61610]: 2024-05-08T23:13:56.300+0000 7fb3f7728700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi016 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi016 to 17118M 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.037 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='osd.0 ' entity='osd.0' 2024-05-08T23:13:57.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:13:57.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:13:57.038 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:57.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi016 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi016 to 17118M 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='osd.0 ' entity='osd.0' 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:13:57.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi016 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi016 to 17118M 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='osd.0 ' entity='osd.0' 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:13:57.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:13:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:13:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:13:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] boot 2024-05-08T23:13:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T23:13:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:58.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:13:58.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:13:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] boot 2024-05-08T23:13:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T23:13:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: osd.0 [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] boot 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T23:13:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:13:59.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/1009006540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]: dispatch 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]: dispatch 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]': finished 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: pgmap v38: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:13:59.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:13:59 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/364493969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/1009006540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]: dispatch 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]: dispatch 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]': finished 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: pgmap v38: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:13:59.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:13:59 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/364493969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/1009006540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]: dispatch 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]: dispatch 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f"}]': finished 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: pgmap v38: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:13:59.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:13:59 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/364493969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:01.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:01 smithi016 ceph-mon[42541]: pgmap v39: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:01.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:01 smithi189 ceph-mon[46919]: pgmap v39: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:01.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:01 smithi155 ceph-mon[47105]: pgmap v39: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:03.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:03 smithi155 ceph-mon[47105]: pgmap v40: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:03.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:03 smithi016 ceph-mon[42541]: pgmap v40: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:03.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:03 smithi189 ceph-mon[46919]: pgmap v40: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:04.704 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T23:14:04.704 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:04.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T23:14:04.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:04.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T23:14:04.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:05 smithi155 ceph-mon[47105]: Deploying daemon osd.1 on smithi155 2024-05-08T23:14:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:05 smithi155 ceph-mon[47105]: pgmap v41: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:05 smithi016 ceph-mon[42541]: Deploying daemon osd.1 on smithi155 2024-05-08T23:14:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:05 smithi016 ceph-mon[42541]: pgmap v41: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:05.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:05 smithi189 ceph-mon[46919]: Deploying daemon osd.1 on smithi155 2024-05-08T23:14:05.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:05 smithi189 ceph-mon[46919]: pgmap v41: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:07.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:07 smithi016 ceph-mon[42541]: pgmap v42: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:07.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:07 smithi189 ceph-mon[46919]: pgmap v42: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:07.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:07 smithi155 ceph-mon[47105]: pgmap v42: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:08.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:09.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:09.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:09.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:09.806 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.806+0000 7f65df7fe700 1 -- 172.21.15.155:0/3903646951 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f65f00613e0 con 0x7f65d80a1090 2024-05-08T23:14:09.807 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 1 on host 'smithi155' 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65d80a1090 msgr2=0x7f65d80a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65d80a1090 0x7f65d80a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005950 tx=0x7f65e000a300 comp rx=0 tx=0).stop 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 msgr2=0x7f65f0199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 0x7f65f0199140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f65e400d950 tx=0x7f65e400dc60 comp rx=0 tx=0).stop 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 shutdown_connections 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65d80a1090 0x7f65d80a3550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65f0108f00 0x7f65f01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65f0073980 0x7f65f0199140 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 --2- 172.21.15.155:0/3903646951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f0074f50 0x7f65f0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 >> 172.21.15.155:0/3903646951 conn(0x7f65f00fc5a0 msgr2=0x7f65f0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 shutdown_connections 2024-05-08T23:14:09.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:09.809+0000 7f65f4b8d700 1 -- 172.21.15.155:0/3903646951 wait complete. 2024-05-08T23:14:09.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:09 smithi155 ceph-mon[47105]: pgmap v43: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:09.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:09.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:09.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:09.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:09.832 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:09 smithi016 ceph-mon[42541]: pgmap v43: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:09 smithi189 ceph-mon[46919]: pgmap v43: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:10.188 DEBUG:teuthology.orchestra.run.smithi155:osd.1> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T23:14:10.190 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-05-08T23:14:10.190 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T23:14:10.222 INFO:journalctl@ceph.osd.1.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:14:10.417 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:14:11.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:10 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:10 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:10 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:11.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:10 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:11.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:10 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:11.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:10 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:11.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:10 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:11.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:10 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:11.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:10 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:11.766 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T23:14:11.766 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T23:14:11.767 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-05-08T23:14:11.767 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-05-08T23:14:11.767 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0154366 s, 679 MB/s 2024-05-08T23:14:11.767 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-05-08T23:14:11.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:11 smithi155 ceph-mon[47105]: pgmap v44: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:11.896 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:11 smithi155 ceph-mon[47105]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:14:11.896 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:11 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1[52095]: 2024-05-08T23:14:11.617+0000 7f5120f017c0 -1 osd.1 0 log_to_monitors true 2024-05-08T23:14:12.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:11 smithi189 ceph-mon[46919]: pgmap v44: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:12.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:11 smithi189 ceph-mon[46919]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:14:12.112 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-05-08T23:14:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:11 smithi016 ceph-mon[42541]: pgmap v44: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:11 smithi016 ceph-mon[42541]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T23:14:12.329 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:14:13.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.124 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:13.125 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.125 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:13.125 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:13.125 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:13.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:13.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:13.285 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 -- 172.21.15.189:0/3517591470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 msgr2=0x7f6d80109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.285 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 --2- 172.21.15.189:0/3517591470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d80109450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800b0a0 tx=0x7f6d7800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 -- 172.21.15.189:0/3517591470 shutdown_connections 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 --2- 172.21.15.189:0/3517591470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d80109450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 --2- 172.21.15.189:0/3517591470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d80101000 0x7f6d80101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 --2- 172.21.15.189:0/3517591470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d80102200 0x7f6d80102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 -- 172.21.15.189:0/3517591470 >> 172.21.15.189:0/3517591470 conn(0x7f6d800fc5c0 msgr2=0x7f6d800fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 -- 172.21.15.189:0/3517591470 shutdown_connections 2024-05-08T23:14:13.286 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 -- 172.21.15.189:0/3517591470 wait complete. 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.285+0000 7f6d85c37700 1 Processor -- start 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 -- start start 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d80101000 0x7f6d80196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d80102200 0x7f6d80197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d8019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8019ea30 con 0x7f6d80101000 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8019eb70 con 0x7f6d80102200 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d85c37700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8019ed00 con 0x7f6d80102ba0 2024-05-08T23:14:13.288 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d8019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d8019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:46278/0 (socket says 172.21.15.189:46278) 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 -- 172.21.15.189:0/37429068 learned_addr learned my addr 172.21.15.189:0/37429068 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7effd700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d80102200 0x7f6d80197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7f7fe700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d80101000 0x7f6d80196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 -- 172.21.15.189:0/37429068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d80102200 msgr2=0x7f6d80197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d80102200 0x7f6d80197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 -- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d80101000 msgr2=0x7f6d80196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d80101000 0x7f6d80196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.286+0000 7f6d7ffff700 1 -- 172.21.15.189:0/37429068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009d00 con 0x7f6d80102ba0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.287+0000 7f6d7ffff700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d8019e460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800bd90 tx=0x7f6d7800bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.287+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800e040 con 0x7f6d80102ba0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.287+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8019ef80 con 0x7f6d80102ba0 2024-05-08T23:14:13.289 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.287+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d78004c60 con 0x7f6d80102ba0 2024-05-08T23:14:13.290 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.287+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780052a0 con 0x7f6d80102ba0 2024-05-08T23:14:13.290 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.287+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8019f3a0 con 0x7f6d80102ba0 2024-05-08T23:14:13.290 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.288+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d801084e0 con 0x7f6d80102ba0 2024-05-08T23:14:13.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.289+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f6d7801d470 con 0x7f6d80102ba0 2024-05-08T23:14:13.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.290+0000 7f6d7cff9700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d680a1110 0x7f6d680a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:13.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.290+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f6d780ccdb0 con 0x7f6d80102ba0 2024-05-08T23:14:13.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.290+0000 7f6d7f7fe700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d680a1110 0x7f6d680a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:13.291 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.291+0000 7f6d7f7fe700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d680a1110 0x7f6d680a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d80102060 tx=0x7f6d70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:13.293 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.293+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d78097ea0 con 0x7f6d80102ba0 2024-05-08T23:14:13.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:13.493+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d8002ce10 con 0x7f6d680a1110 2024-05-08T23:14:13.812 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:13 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1[52095]: 2024-05-08T23:14:13.505+0000 7f5116565700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T23:14:13.812 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:13 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1[52095]: 2024-05-08T23:14:13.509+0000 7f510e352700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:14:14.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi155 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi155 to 17114M 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: pgmap v46: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:13 smithi016 ceph-mon[42541]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi155 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi155 to 17114M 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: pgmap v46: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:14.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:13 smithi189 ceph-mon[46919]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi155 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi155 to 17114M 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: pgmap v46: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:13 smithi155 ceph-mon[47105]: from='osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573]' entity='osd.1' 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] boot 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T23:14:14.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] boot 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T23:14:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: osd.1 [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] boot 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T23:14:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:14:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:15 smithi016 ceph-mon[42541]: pgmap v49: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:15 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/2268743111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]: dispatch 2024-05-08T23:14:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:15 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]: dispatch 2024-05-08T23:14:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:15 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]': finished 2024-05-08T23:14:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:15 smithi016 ceph-mon[42541]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T23:14:16.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:15 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:15 smithi189 ceph-mon[46919]: pgmap v49: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:15 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/2268743111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]: dispatch 2024-05-08T23:14:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:15 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]: dispatch 2024-05-08T23:14:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:15 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]': finished 2024-05-08T23:14:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:15 smithi189 ceph-mon[46919]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T23:14:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:15 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:15 smithi155 ceph-mon[47105]: pgmap v49: 0 pgs: ; 0 B data, 34 MiB used, 89 GiB / 89 GiB avail 2024-05-08T23:14:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:15 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/2268743111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]: dispatch 2024-05-08T23:14:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:15 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]: dispatch 2024-05-08T23:14:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:15 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a5d9e16-e1c7-4880-900c-e886b3e9c00d"}]': finished 2024-05-08T23:14:16.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:15 smithi155 ceph-mon[47105]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T23:14:16.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:15 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:17.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:16 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/1401866052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:17.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:16 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/1401866052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:16 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/1401866052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:18.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:17 smithi016 ceph-mon[42541]: pgmap v51: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:18.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:17 smithi189 ceph-mon[46919]: pgmap v51: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:18.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:17 smithi155 ceph-mon[47105]: pgmap v51: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:20.081 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:19 smithi189 ceph-mon[46919]: pgmap v52: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:19 smithi016 ceph-mon[42541]: pgmap v52: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:19 smithi155 ceph-mon[47105]: pgmap v52: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:21 smithi189 ceph-mon[46919]: pgmap v53: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T23:14:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:21 smithi016 ceph-mon[42541]: pgmap v53: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T23:14:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:21 smithi155 ceph-mon[47105]: pgmap v53: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T23:14:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:23.097 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:22 smithi189 ceph-mon[46919]: Deploying daemon osd.2 on smithi189 2024-05-08T23:14:23.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:22 smithi016 ceph-mon[42541]: Deploying daemon osd.2 on smithi189 2024-05-08T23:14:23.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:22 smithi155 ceph-mon[47105]: Deploying daemon osd.2 on smithi189 2024-05-08T23:14:24.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:23 smithi016 ceph-mon[42541]: pgmap v54: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:23 smithi189 ceph-mon[46919]: pgmap v54: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:24.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:23 smithi155 ceph-mon[47105]: pgmap v54: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:25 smithi016 ceph-mon[42541]: pgmap v55: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:25 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:25 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:25 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:25 smithi189 ceph-mon[46919]: pgmap v55: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:25 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:25 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:25 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:25 smithi155 ceph-mon[47105]: pgmap v55: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:25 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:25 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:25 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:27.148 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.149+0000 7f6d7cff9700 1 -- 172.21.15.189:0/37429068 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d8002ce10 con 0x7f6d680a1110 2024-05-08T23:14:27.149 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d680a1110 msgr2=0x7f6d680a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d680a1110 0x7f6d680a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6d80102060 tx=0x7f6d70009260 comp rx=0 tx=0).stop 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 msgr2=0x7f6d8019e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d8019e460 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800bd90 tx=0x7f6d7800bdc0 comp rx=0 tx=0).stop 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 shutdown_connections 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d680a1110 0x7f6d680a35d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d80102ba0 0x7f6d8019e460 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d80101000 0x7f6d80196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 --2- 172.21.15.189:0/37429068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d80102200 0x7f6d80197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 >> 172.21.15.189:0/37429068 conn(0x7f6d800fc5c0 msgr2=0x7f6d80105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:27.151 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 shutdown_connections 2024-05-08T23:14:27.152 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:14:27.151+0000 7f6d85c37700 1 -- 172.21.15.189:0/37429068 wait complete. 2024-05-08T23:14:27.379 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: pgmap v56: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.380 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:27.564 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T23:14:27.566 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-05-08T23:14:27.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T23:14:27.600 INFO:journalctl@ceph.osd.2.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: pgmap v56: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.602 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: pgmap v56: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:27.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:14:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T23:14:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T23:14:29.156 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-08T23:14:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-08T23:14:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00893749 s, 1.2 GB/s 2024-05-08T23:14:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-08T23:14:29.382 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2[52700]: 2024-05-08T23:14:28.929+0000 7ff30028d7c0 -1 osd.2 0 log_to_monitors true 2024-05-08T23:14:29.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-05-08T23:14:29.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:29 smithi189 ceph-mon[46919]: pgmap v57: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:29 smithi189 ceph-mon[46919]: from='osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:14:29.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:29 smithi189 ceph-mon[46919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:14:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:14:29.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:29 smithi016 ceph-mon[42541]: pgmap v57: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:29 smithi016 ceph-mon[42541]: from='osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:14:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:29 smithi016 ceph-mon[42541]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:14:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:29 smithi155 ceph-mon[47105]: pgmap v57: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:29 smithi155 ceph-mon[47105]: from='osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:14:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:29 smithi155 ceph-mon[47105]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T23:14:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 -- 172.21.15.16:0/1503255156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102200 msgr2=0x7f0f28102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 --2- 172.21.15.16:0/1503255156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102200 0x7f0f28102660 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f180099c0 tx=0x7f0f18009cd0 comp rx=0 tx=0).stop 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 -- 172.21.15.16:0/1503255156 shutdown_connections 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 --2- 172.21.15.16:0/1503255156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f28102ba0 0x7f0f28109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 --2- 172.21.15.16:0/1503255156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f28101000 0x7f0f28101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 --2- 172.21.15.16:0/1503255156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102200 0x7f0f28102660 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 -- 172.21.15.16:0/1503255156 >> 172.21.15.16:0/1503255156 conn(0x7f0f280fc5c0 msgr2=0x7f0f280fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 -- 172.21.15.16:0/1503255156 shutdown_connections 2024-05-08T23:14:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.795+0000 7f0f30135700 1 -- 172.21.15.16:0/1503255156 wait complete. 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 Processor -- start 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 -- start start 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f28101000 0x7f0f28196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f28102200 0x7f0f281973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 0x7f0f2819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2819e9c0 con 0x7f0f28102ba0 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2819eb00 con 0x7f0f28102200 2024-05-08T23:14:30.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f30135700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2819ec90 con 0x7f0f28101000 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f2e6d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 0x7f0f2819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f2e6d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 0x7f0f2819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57574/0 (socket says 172.21.15.16:57574) 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f2e6d2700 1 -- 172.21.15.16:0/2103490926 learned_addr learned my addr 172.21.15.16:0/2103490926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f2d6d0700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f28102200 0x7f0f281973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f2ded1700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f28101000 0x7f0f28196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.797+0000 7f0f2e6d2700 1 -- 172.21.15.16:0/2103490926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f28101000 msgr2=0x7f0f28196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f2e6d2700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f28101000 0x7f0f28196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f2e6d2700 1 -- 172.21.15.16:0/2103490926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f28102200 msgr2=0x7f0f281973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f2e6d2700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f28102200 0x7f0f281973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f2e6d2700 1 -- 172.21.15.16:0/2103490926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180095f0 con 0x7f0f28102ba0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f2e6d2700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 0x7f0f2819e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000ef60 tx=0x7f0f2000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20018070 con 0x7f0f28102ba0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f2000f040 con 0x7f0f28102ba0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2819ef10 con 0x7f0f28102ba0 2024-05-08T23:14:30.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000cc60 con 0x7f0f28102ba0 2024-05-08T23:14:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.798+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2819f380 con 0x7f0f28102ba0 2024-05-08T23:14:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.799+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28108490 con 0x7f0f28102ba0 2024-05-08T23:14:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.800+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f0f20004ad0 con 0x7f0f28102ba0 2024-05-08T23:14:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.801+0000 7f0f1effd700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0f140a1110 0x7f0f140a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:30.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.801+0000 7f0f2ded1700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0f140a1110 0x7f0f140a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:30.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.801+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f0f200cab80 con 0x7f0f28102ba0 2024-05-08T23:14:30.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.801+0000 7f0f2ded1700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0f140a1110 0x7f0f140a35d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f28102060 tx=0x7f0f24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:30.805+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0f20095af0 con 0x7f0f28102ba0 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi189 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi189 to 17114M 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:30.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:30.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.918 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2[52700]: 2024-05-08T23:14:30.593+0000 7ff2f58f1700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T23:14:30.919 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2[52700]: 2024-05-08T23:14:30.598+0000 7ff2f16e6700 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi189 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:30.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi189 to 17114M 2024-05-08T23:14:30.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:30.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:30.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi189 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi189 to 17114M 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:30.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:31.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:31.030+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f2802ce40 con 0x7f0f140a1110 2024-05-08T23:14:31.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:31.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:33 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:14:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:33 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:14:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:33 smithi016 ceph-mon[42541]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:33.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:33 smithi016 ceph-mon[42541]: osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] boot 2024-05-08T23:14:33.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:33 smithi016 ceph-mon[42541]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T23:14:33.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:33 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:14:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:33 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:14:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:33 smithi189 ceph-mon[46919]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:33 smithi189 ceph-mon[46919]: osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] boot 2024-05-08T23:14:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:33 smithi189 ceph-mon[46919]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T23:14:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:33 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:14:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:33 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:14:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:33 smithi155 ceph-mon[47105]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:33 smithi155 ceph-mon[47105]: osd.2 [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] boot 2024-05-08T23:14:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:33 smithi155 ceph-mon[47105]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T23:14:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:14:34.049 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:33 smithi016 sudo[67475]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:14:34.049 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:33 smithi016 sudo[67475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:34.049 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:33 smithi016 sudo[67475]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:34.049 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:33 smithi016 sudo[67475]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:34.076+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0f20095ec0 con 0x7f0f28102ba0 2024-05-08T23:14:34.364 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:34 smithi016 sudo[67527]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:14:34.364 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:34 smithi016 sudo[67527]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:34.364 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:34 smithi016 sudo[67527]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:34.364 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:14:34 smithi016 sudo[67527]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:34.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:34.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:14:34.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T23:14:34.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3211680995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]: dispatch 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]: dispatch 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]': finished 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:34.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:34 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1511574935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:34.376 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:34.376 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:14:34.376 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T23:14:34.376 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T23:14:34.376 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3211680995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]: dispatch 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]: dispatch 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]': finished 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:34.377 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:34 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1511574935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3211680995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]: dispatch 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]: dispatch 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810b101a-3d2e-4761-be92-dc7bc786a11c"}]': finished 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T23:14:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:34.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:34 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1511574935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57078]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57078]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57078]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57078]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57081]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57081]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57081]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:34.715 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:14:34 smithi155 sudo[57081]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:35.138 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:35 smithi189 ceph-mon[46919]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T23:14:35.138 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:35.139 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:35 smithi189 ceph-mon[46919]: mgrmap e16: x(active, since 111s), standbys: z, y 2024-05-08T23:14:35.139 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:35.139 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:35 smithi189 ceph-mon[46919]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:35.139 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:35.139 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:34 smithi189 sudo[57958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:14:35.139 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:34 smithi189 sudo[57958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:35.139 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:34 smithi189 sudo[57958]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:35.139 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:34 smithi189 sudo[57958]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:35.139 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:35 smithi189 sudo[57961]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:14:35.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 ceph-mon[42541]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T23:14:35.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:35.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 ceph-mon[42541]: mgrmap e16: x(active, since 111s), standbys: z, y 2024-05-08T23:14:35.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:35.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 ceph-mon[42541]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:35.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:35.418 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:35 smithi189 sudo[57961]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:35.419 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:35 smithi189 sudo[57961]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:35.419 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:14:35 smithi189 sudo[57961]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:35 smithi155 ceph-mon[47105]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T23:14:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:35 smithi155 ceph-mon[47105]: mgrmap e16: x(active, since 111s), standbys: z, y 2024-05-08T23:14:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:35 smithi155 ceph-mon[47105]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 sudo[68273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:14:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 sudo[68273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 sudo[68273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:36.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:35 smithi016 sudo[68273]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:36.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 sudo[68460]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:14:36.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 sudo[68460]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:36.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 sudo[68460]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:36.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 sudo[68460]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57964]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57964]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57964]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57964]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57967]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57967]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:36.597 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57967]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 sudo[57967]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:36.598 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:36 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:36.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:36.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:36 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 sudo[57084]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 sudo[57084]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 sudo[57084]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:36.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:36 smithi155 sudo[57084]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 sudo[57087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T23:14:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 sudo[57087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T23:14:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 sudo[57087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T23:14:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 sudo[57087]: pam_unix(sudo:session): session closed for user root 2024-05-08T23:14:37.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:37.840 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:37 smithi016 ceph-mon[42541]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:37.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:37 smithi189 ceph-mon[46919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:14:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:37 smithi155 ceph-mon[47105]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T23:14:39.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:39 smithi016 ceph-mon[42541]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:39.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T23:14:39.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:39.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:39 smithi189 ceph-mon[46919]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:39.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T23:14:39.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:39.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:39 smithi155 ceph-mon[47105]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-05-08T23:14:39.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T23:14:39.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:40.710 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:40 smithi016 ceph-mon[42541]: Deploying daemon osd.3 on smithi016 2024-05-08T23:14:40.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:40 smithi189 ceph-mon[46919]: Deploying daemon osd.3 on smithi016 2024-05-08T23:14:40.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:40 smithi155 ceph-mon[47105]: Deploying daemon osd.3 on smithi016 2024-05-08T23:14:41.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:41 smithi016 ceph-mon[42541]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 196 KiB/s wr, 14 op/s 2024-05-08T23:14:41.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:41 smithi189 ceph-mon[46919]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 196 KiB/s wr, 14 op/s 2024-05-08T23:14:41.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:41 smithi155 ceph-mon[47105]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 196 KiB/s wr, 14 op/s 2024-05-08T23:14:43.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:43 smithi016 ceph-mon[42541]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-08T23:14:43.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:43.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:43.849 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:43 smithi189 ceph-mon[46919]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-08T23:14:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:43 smithi155 ceph-mon[47105]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-05-08T23:14:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:44.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.974+0000 7f0f1effd700 1 -- 172.21.15.16:0/2103490926 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f2802ce40 con 0x7f0f140a1110 2024-05-08T23:14:44.974 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0f140a1110 msgr2=0x7f0f140a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0f140a1110 0x7f0f140a35d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0f28102060 tx=0x7f0f24009260 comp rx=0 tx=0).stop 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 msgr2=0x7f0f2819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 0x7f0f2819e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000ef60 tx=0x7f0f2000ef90 comp rx=0 tx=0).stop 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 shutdown_connections 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f28102ba0 0x7f0f2819e3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0f140a1110 0x7f0f140a35d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f28101000 0x7f0f28196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 --2- 172.21.15.16:0/2103490926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f28102200 0x7f0f281973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:44.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 >> 172.21.15.16:0/2103490926 conn(0x7f0f280fc5c0 msgr2=0x7f0f281055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 shutdown_connections 2024-05-08T23:14:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:14:44.977+0000 7f0f30135700 1 -- 172.21.15.16:0/2103490926 wait complete. 2024-05-08T23:14:45.348 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T23:14:45.350 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_3... 2024-05-08T23:14:45.351 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T23:14:45.388 INFO:journalctl@ceph.osd.3.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:14:45.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:45.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:45.549 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-05-08T23:14:45.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:45.550 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:45 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.583 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:45.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:45 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 98 KiB/s wr, 7 op/s 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:14:45.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:45 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:46.847 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T23:14:46.847 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T23:14:46.847 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-05-08T23:14:46.847 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-05-08T23:14:46.847 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113266 s, 926 MB/s 2024-05-08T23:14:46.847 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-05-08T23:14:47.113 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:14:46 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3[69694]: 2024-05-08T23:14:46.753+0000 7fb6566867c0 -1 osd.3 0 log_to_monitors true 2024-05-08T23:14:47.236 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-05-08T23:14:47.457 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:14:47.855 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:47 smithi155 ceph-mon[47105]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-08T23:14:47.855 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:47 smithi155 ceph-mon[47105]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T23:14:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:47 smithi016 ceph-mon[42541]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-08T23:14:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:47 smithi016 ceph-mon[42541]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T23:14:47.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:47 smithi189 ceph-mon[46919]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-08T23:14:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:47 smithi189 ceph-mon[46919]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T23:14:48.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 -- 172.21.15.155:0/1928000724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04106d50 msgr2=0x7f9b04109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:48.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/1928000724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04106d50 0x7f9b04109160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b0a0 tx=0x7f9afc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:14:48.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 -- 172.21.15.155:0/1928000724 shutdown_connections 2024-05-08T23:14:48.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/1928000724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04106d50 0x7f9b04109160 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:48.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/1928000724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b04101af0 0x7f9b04103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/1928000724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04104420 0x7f9b04106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 -- 172.21.15.155:0/1928000724 >> 172.21.15.155:0/1928000724 conn(0x7f9b040fb4a0 msgr2=0x7f9b040fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 -- 172.21.15.155:0/1928000724 shutdown_connections 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.367+0000 7f9b09a8b700 1 -- 172.21.15.155:0/1928000724 wait complete. 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 Processor -- start 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 -- start start 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 0x7f9b04196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b04104420 0x7f9b04197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04106d50 0x7f9b0419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0419eae0 con 0x7f9b04104420 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0419ec20 con 0x7f9b04101af0 2024-05-08T23:14:48.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b09a8b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0419edb0 con 0x7f9b04106d50 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b037fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 0x7f9b04196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b037fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 0x7f9b04196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:57578/0 (socket says 172.21.15.155:57578) 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b037fe700 1 -- 172.21.15.155:0/2063817861 learned_addr learned my addr 172.21.15.155:0/2063817861 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.368+0000 7f9b02ffd700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b04104420 0x7f9b04197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b03fff700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04106d50 0x7f9b0419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b037fe700 1 -- 172.21.15.155:0/2063817861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04106d50 msgr2=0x7f9b0419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b037fe700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04106d50 0x7f9b0419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b037fe700 1 -- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b04104420 msgr2=0x7f9b04197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b037fe700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b04104420 0x7f9b04197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b037fe700 1 -- 172.21.15.155:0/2063817861 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009d00 con 0x7f9b04101af0 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b03fff700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04106d50 0x7f9b0419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b037fe700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 0x7f9b04196ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b530 tx=0x7f9aec00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:48.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec011830 con 0x7f9b04101af0 2024-05-08T23:14:48.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0419f090 con 0x7f9b04101af0 2024-05-08T23:14:48.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9aec011e20 con 0x7f9b04101af0 2024-05-08T23:14:48.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00f2e0 con 0x7f9b04101af0 2024-05-08T23:14:48.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.369+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0419f5e0 con 0x7f9b04101af0 2024-05-08T23:14:48.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.370+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b040fd020 con 0x7f9b04101af0 2024-05-08T23:14:48.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.371+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9aec011990 con 0x7f9b04101af0 2024-05-08T23:14:48.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.372+0000 7f9b00ff9700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9af00a11a0 0x7f9af00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:14:48.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.372+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3373+0+0 (secure 0 0 0) 0x7f9aec0cb570 con 0x7f9b04101af0 2024-05-08T23:14:48.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.372+0000 7f9b02ffd700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9af00a11a0 0x7f9af00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:14:48.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.373+0000 7f9b02ffd700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9af00a11a0 0x7f9af00a3660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005950 tx=0x7f9af400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:14:48.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.375+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9aec095fb0 con 0x7f9b04101af0 2024-05-08T23:14:48.575 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:14:48.575+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b040613e0 con 0x7f9af00a11a0 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi016 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi016 to 8559M 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:48.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:48.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:48.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:48.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi016 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi016 to 8559M 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:48.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:48.936 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T23:14:48.936 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T23:14:48.936 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:48.936 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:14:48.936 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi016 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi016 to 8559M 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:14:48.937 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:49.596 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-08T23:14:49.596 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:14:49 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3[69694]: 2024-05-08T23:14:49.134+0000 7fb64bcea700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T23:14:49.596 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:14:49 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3[69694]: 2024-05-08T23:14:49.140+0000 7fb643ad7700 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:14:49.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] boot 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: osdmap e23: 4 total, 4 up, 4 in 2024-05-08T23:14:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:49 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] boot 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: osdmap e23: 4 total, 4 up, 4 in 2024-05-08T23:14:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:49 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: from='osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008]' entity='osd.3' 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: osd.3 [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] boot 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: osdmap e23: 4 total, 4 up, 4 in 2024-05-08T23:14:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:49 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:14:50.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:14:50.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:14:50.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/3673937143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]: dispatch 2024-05-08T23:14:50.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]: dispatch 2024-05-08T23:14:50.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]': finished 2024-05-08T23:14:50.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: osdmap e24: 5 total, 4 up, 5 in 2024-05-08T23:14:50.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:50.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:14:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:14:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/3673937143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]: dispatch 2024-05-08T23:14:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]: dispatch 2024-05-08T23:14:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]': finished 2024-05-08T23:14:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: osdmap e24: 5 total, 4 up, 5 in 2024-05-08T23:14:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/3673937143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]: dispatch 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]: dispatch 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1578371-9303-4583-95e0-6c7812f396ad"}]': finished 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: osdmap e24: 5 total, 4 up, 5 in 2024-05-08T23:14:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:51.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:51 smithi016 ceph-mon[42541]: pgmap v78: 1 pgs: 1 remapped+peering; 577 KiB data, 485 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:51.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:51 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/177378696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:51.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:51 smithi016 ceph-mon[42541]: osdmap e25: 5 total, 4 up, 5 in 2024-05-08T23:14:51.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:51 smithi189 ceph-mon[46919]: pgmap v78: 1 pgs: 1 remapped+peering; 577 KiB data, 485 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:51 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/177378696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:51 smithi189 ceph-mon[46919]: osdmap e25: 5 total, 4 up, 5 in 2024-05-08T23:14:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:51 smithi155 ceph-mon[47105]: pgmap v78: 1 pgs: 1 remapped+peering; 577 KiB data, 485 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:51 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/177378696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:14:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:51 smithi155 ceph-mon[47105]: osdmap e25: 5 total, 4 up, 5 in 2024-05-08T23:14:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:53.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:53 smithi189 ceph-mon[46919]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T23:14:53.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:53.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:53 smithi189 ceph-mon[46919]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 485 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:53.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:53 smithi155 ceph-mon[47105]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T23:14:53.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:53.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:53 smithi155 ceph-mon[47105]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 485 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:53.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:53 smithi016 ceph-mon[42541]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T23:14:53.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:53.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:53 smithi016 ceph-mon[42541]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 485 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:54.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:54 smithi189 ceph-mon[46919]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T23:14:54.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:54.670 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:54 smithi155 ceph-mon[47105]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T23:14:54.670 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:54.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:54 smithi016 ceph-mon[42541]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T23:14:54.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:14:55.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:55 smithi189 ceph-mon[46919]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 484 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:55.699 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:55 smithi155 ceph-mon[47105]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 484 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:55 smithi016 ceph-mon[42541]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 484 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:14:56.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T23:14:56.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:56 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:56.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T23:14:56.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:56 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:56.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T23:14:56.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:56 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:14:57.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:57 smithi155 ceph-mon[47105]: Deploying daemon osd.4 on smithi155 2024-05-08T23:14:57.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:57 smithi155 ceph-mon[47105]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:14:57.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:57 smithi189 ceph-mon[46919]: Deploying daemon osd.4 on smithi155 2024-05-08T23:14:57.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:57 smithi189 ceph-mon[46919]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:14:57.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:57 smithi016 ceph-mon[42541]: Deploying daemon osd.4 on smithi155 2024-05-08T23:14:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:57 smithi016 ceph-mon[42541]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:14:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:14:59 smithi016 ceph-mon[42541]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T23:14:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:14:59 smithi189 ceph-mon[46919]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T23:14:59.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:14:59 smithi155 ceph-mon[47105]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T23:15:00.714 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:00 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:00.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:00 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:00.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:00 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:00.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:00 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:00.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:00 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:00.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:00 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:00.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:00 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:00 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:00 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:01.578 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.578+0000 7f9b00ff9700 1 -- 172.21.15.155:0/2063817861 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9b040613e0 con 0x7f9af00a11a0 2024-05-08T23:15:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-05-08T23:15:01.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9af00a11a0 msgr2=0x7f9af00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:01.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9af00a11a0 0x7f9af00a3660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9af4005950 tx=0x7f9af400b410 comp rx=0 tx=0).stop 2024-05-08T23:15:01.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 msgr2=0x7f9b04196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:01.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 0x7f9b04196ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b530 tx=0x7f9aec00b840 comp rx=0 tx=0).stop 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 shutdown_connections 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9af00a11a0 0x7f9af00a3660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b04106d50 0x7f9b0419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9b04101af0 0x7f9b04196ff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 --2- 172.21.15.155:0/2063817861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b04104420 0x7f9b04197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 >> 172.21.15.155:0/2063817861 conn(0x7f9b040fb4a0 msgr2=0x7f9b041025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 shutdown_connections 2024-05-08T23:15:01.581 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:01.580+0000 7f9b09a8b700 1 -- 172.21.15.155:0/2063817861 wait complete. 2024-05-08T23:15:01.870 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:01.871 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:01 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:01.964 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T23:15:01.966 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-05-08T23:15:01.966 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T23:15:01.997 INFO:journalctl@ceph.osd.4.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.000 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.001 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:01 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:02.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:01 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:02.199 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:15:03.567 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T23:15:03.567 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T23:15:03.567 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-05-08T23:15:03.567 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-05-08T23:15:03.567 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111647 s, 939 MB/s 2024-05-08T23:15:03.567 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-05-08T23:15:03.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:03 smithi189 ceph-mon[46919]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:03 smithi189 ceph-mon[46919]: from='osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:15:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:03 smithi189 ceph-mon[46919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:15:03.921 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-05-08T23:15:03.945 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:03 smithi155 ceph-mon[47105]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:03.946 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:03 smithi155 ceph-mon[47105]: from='osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:15:03.946 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:03 smithi155 ceph-mon[47105]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:15:03.946 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:15:03 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4[60168]: 2024-05-08T23:15:03.567+0000 7fdacab407c0 -1 osd.4 0 log_to_monitors true 2024-05-08T23:15:04.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:03 smithi016 ceph-mon[42541]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:03 smithi016 ceph-mon[42541]: from='osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:15:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:03 smithi016 ceph-mon[42541]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T23:15:04.179 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:15:04.965 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:15:04 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4[60168]: 2024-05-08T23:15:04.705+0000 7fdac01a4700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T23:15:04.965 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:15:04 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4[60168]: 2024-05-08T23:15:04.711+0000 7fdabbf99700 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:15:04.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T23:15:04.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T23:15:04.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:04.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:04.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:04 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.085 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:05.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:04 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:05.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:04 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 -- 172.21.15.189:0/374335463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a41043c0 msgr2=0x7f07a41067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 --2- 172.21.15.189:0/374335463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a41043c0 0x7f07a41067b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0794009990 tx=0x7f0794009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 -- 172.21.15.189:0/374335463 shutdown_connections 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 --2- 172.21.15.189:0/374335463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a4106cf0 0x7f07a4109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 --2- 172.21.15.189:0/374335463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4101a90 0x7f07a4103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 --2- 172.21.15.189:0/374335463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a41043c0 0x7f07a41067b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:05.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 -- 172.21.15.189:0/374335463 >> 172.21.15.189:0/374335463 conn(0x7f07a40fb4c0 msgr2=0x7f07a40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:05.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 -- 172.21.15.189:0/374335463 shutdown_connections 2024-05-08T23:15:05.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.252+0000 7f07aa17c700 1 -- 172.21.15.189:0/374335463 wait complete. 2024-05-08T23:15:05.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 Processor -- start 2024-05-08T23:15:05.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 -- start start 2024-05-08T23:15:05.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 0x7f07a4196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:05.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41043c0 0x7f07a4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:05.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4106cf0 0x7f07a419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:05.256 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a419ea10 con 0x7f07a4106cf0 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a419eb50 con 0x7f07a41043c0 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.253+0000 7f07aa17c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a419ece0 con 0x7f07a4101a90 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 0x7f07a4196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 0x7f07a4196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:34046/0 (socket says 172.21.15.189:34046) 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 -- 172.21.15.189:0/705865088 learned_addr learned my addr 172.21.15.189:0/705865088 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a3fff700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4106cf0 0x7f07a419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a2ffd700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41043c0 0x7f07a4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 -- 172.21.15.189:0/705865088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41043c0 msgr2=0x7f07a4197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41043c0 0x7f07a4197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 -- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4106cf0 msgr2=0x7f07a419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4106cf0 0x7f07a419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 -- 172.21.15.189:0/705865088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940095f0 con 0x7f07a4101a90 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a2ffd700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41043c0 0x7f07a4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:15:05.257 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a37fe700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 0x7f07a4196ed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f079800d980 tx=0x7f079800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:05.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798009b40 con 0x7f07a4101a90 2024-05-08T23:15:05.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f079800b6e0 con 0x7f07a4101a90 2024-05-08T23:15:05.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a419efc0 con 0x7f07a4101a90 2024-05-08T23:15:05.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800f620 con 0x7f07a4101a90 2024-05-08T23:15:05.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.254+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a419f3d0 con 0x7f07a4101a90 2024-05-08T23:15:05.258 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.255+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a40fd040 con 0x7f07a4101a90 2024-05-08T23:15:05.259 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.256+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0798010460 con 0x7f07a4101a90 2024-05-08T23:15:05.259 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.257+0000 7f07a0ff9700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f078c0a1120 0x7f078c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:05.259 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.257+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f07980cb680 con 0x7f07a4101a90 2024-05-08T23:15:05.259 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.257+0000 7f07a2ffd700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f078c0a1120 0x7f078c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:05.259 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.258+0000 7f07a2ffd700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f078c0a1120 0x7f078c0a35e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07940095c0 tx=0x7f0794009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:05.261 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.261+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0798095f80 con 0x7f07a4101a90 2024-05-08T23:15:05.464 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:05.463+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f07a402ceb0 con 0x7f078c0a1120 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi155 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi155 to 8557M 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:05.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:05.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi155 2024-05-08T23:15:05.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi155 to 8557M 2024-05-08T23:15:05.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:05.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:15:05.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T23:15:05.990 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:05.991 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:05.991 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:05.991 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:05.991 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi155 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi155 to 8557M 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:06.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:06 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:15:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:06 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:15:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:06 smithi155 ceph-mon[47105]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:06 smithi155 ceph-mon[47105]: osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] boot 2024-05-08T23:15:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:06 smithi155 ceph-mon[47105]: osdmap e30: 5 total, 5 up, 5 in 2024-05-08T23:15:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:06 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:06 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:15:07.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:06 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:15:07.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:06 smithi016 ceph-mon[42541]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:07.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:06 smithi016 ceph-mon[42541]: osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] boot 2024-05-08T23:15:07.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:06 smithi016 ceph-mon[42541]: osdmap e30: 5 total, 5 up, 5 in 2024-05-08T23:15:07.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:06 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:06 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:15:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:06 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:15:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:06 smithi189 ceph-mon[46919]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:06 smithi189 ceph-mon[46919]: osd.4 [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] boot 2024-05-08T23:15:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:06 smithi189 ceph-mon[46919]: osdmap e30: 5 total, 5 up, 5 in 2024-05-08T23:15:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:06 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/1184174456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]: dispatch 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]: dispatch 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]': finished 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T23:15:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:07 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:08.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:15:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T23:15:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/1184174456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]: dispatch 2024-05-08T23:15:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]: dispatch 2024-05-08T23:15:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]': finished 2024-05-08T23:15:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T23:15:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:07 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/1184174456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]: dispatch 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]: dispatch 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6855d48-5018-4f4a-a7e6-d97ce06a38e8"}]': finished 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: osdmap e32: 6 total, 5 up, 6 in 2024-05-08T23:15:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:07 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:09.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/3684731324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:09.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:08 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/3684731324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:09.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:08 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/3684731324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:10.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:09 smithi016 ceph-mon[42541]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:09 smithi189 ceph-mon[46919]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:09 smithi155 ceph-mon[47105]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:12.072 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:11 smithi189 ceph-mon[46919]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:11 smithi016 ceph-mon[42541]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:12.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:11 smithi155 ceph-mon[47105]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:13 smithi189 ceph-mon[46919]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T23:15:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:13 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:14.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:13 smithi016 ceph-mon[42541]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T23:15:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:13 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:13 smithi155 ceph-mon[47105]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T23:15:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:13 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:14.975 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:14 smithi189 ceph-mon[46919]: Deploying daemon osd.5 on smithi189 2024-05-08T23:15:15.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:14 smithi016 ceph-mon[42541]: Deploying daemon osd.5 on smithi189 2024-05-08T23:15:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:14 smithi155 ceph-mon[47105]: Deploying daemon osd.5 on smithi189 2024-05-08T23:15:16.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:15 smithi016 ceph-mon[42541]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:15 smithi189 ceph-mon[46919]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:15 smithi155 ceph-mon[47105]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:18.051 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:17 smithi189 ceph-mon[46919]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:18.051 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:17 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:18.051 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:17 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:18.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:17 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:17 smithi016 ceph-mon[42541]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:17 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:17 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:17 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:17 smithi155 ceph-mon[47105]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:17 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:17 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:17 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:19.175 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.174+0000 7f07a0ff9700 1 -- 172.21.15.189:0/705865088 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f07a402ceb0 con 0x7f078c0a1120 2024-05-08T23:15:19.175 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-05-08T23:15:19.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f078c0a1120 msgr2=0x7f078c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f078c0a1120 0x7f078c0a35e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07940095c0 tx=0x7f0794009530 comp rx=0 tx=0).stop 2024-05-08T23:15:19.177 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 msgr2=0x7f07a4196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:19.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 0x7f07a4196ed0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f079800d980 tx=0x7f079800dd40 comp rx=0 tx=0).stop 2024-05-08T23:15:19.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 shutdown_connections 2024-05-08T23:15:19.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4106cf0 0x7f07a419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f078c0a1120 0x7f078c0a35e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07a4101a90 0x7f07a4196ed0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.178 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 --2- 172.21.15.189:0/705865088 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07a41043c0 0x7f07a4197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:19.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 >> 172.21.15.189:0/705865088 conn(0x7f07a40fb4c0 msgr2=0x7f07a4102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:19.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 shutdown_connections 2024-05-08T23:15:19.179 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:19.177+0000 7f07aa17c700 1 -- 172.21.15.189:0/705865088 wait complete. 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:19.563 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T23:15:19.566 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-05-08T23:15:19.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T23:15:19.595 INFO:journalctl@ceph.osd.5.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:15:19.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:19.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:15:21.169 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:15:20 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:15:20.912+0000 7f91a90197c0 -1 osd.5 0 log_to_monitors true 2024-05-08T23:15:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T23:15:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T23:15:21.173 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-08T23:15:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-08T23:15:21.174 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0090274 s, 1.2 GB/s 2024-05-08T23:15:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-08T23:15:21.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-05-08T23:15:21.821 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:15:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:21 smithi016 ceph-mon[42541]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:21.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:21 smithi016 ceph-mon[42541]: from='osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:15:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:21 smithi016 ceph-mon[42541]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:15:21.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:21 smithi189 ceph-mon[46919]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:21 smithi189 ceph-mon[46919]: from='osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:15:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:21 smithi189 ceph-mon[46919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:15:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:21 smithi155 ceph-mon[47105]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:21 smithi155 ceph-mon[47105]: from='osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:15:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:21 smithi155 ceph-mon[47105]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi189 2024-05-08T23:15:22.600 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi189 to 8557M 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:22.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:22 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 -- 172.21.15.16:0/1057424168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 msgr2=0x7ff55c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 --2- 172.21.15.16:0/1057424168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c108360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff55400b0a0 tx=0x7ff55400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 -- 172.21.15.16:0/1057424168 shutdown_connections 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 --2- 172.21.15.16:0/1057424168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c108360 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 --2- 172.21.15.16:0/1057424168 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0ffe90 0x7ff55c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 --2- 172.21.15.16:0/1057424168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff55c101080 0x7ff55c101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 -- 172.21.15.16:0/1057424168 >> 172.21.15.16:0/1057424168 conn(0x7ff55c0fb4a0 msgr2=0x7ff55c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 -- 172.21.15.16:0/1057424168 shutdown_connections 2024-05-08T23:15:22.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.815+0000 7ff56191f700 1 -- 172.21.15.16:0/1057424168 wait complete. 2024-05-08T23:15:22.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 Processor -- start 2024-05-08T23:15:22.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 -- start start 2024-05-08T23:15:22.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0ffe90 0x7ff55c103e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff55c101080 0x7ff55c1024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1031d0 con 0x7ff55c101a40 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c103340 con 0x7ff55c0ffe90 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff56191f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c1a0fc0 con 0x7ff55c101080 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff55b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff55b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55982/0 (socket says 172.21.15.16:55982) 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff55b7fe700 1 -- 172.21.15.16:0/1389946551 learned_addr learned my addr 172.21.15.16:0/1389946551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff55affd700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0ffe90 0x7ff55c103e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.816+0000 7ff55a7fc700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff55c101080 0x7ff55c1024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff55b7fe700 1 -- 172.21.15.16:0/1389946551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff55c101080 msgr2=0x7ff55c1024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff55b7fe700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff55c101080 0x7ff55c1024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff55b7fe700 1 -- 172.21.15.16:0/1389946551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0ffe90 msgr2=0x7ff55c103e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:22.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff55b7fe700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0ffe90 0x7ff55c103e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff55b7fe700 1 -- 172.21.15.16:0/1389946551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554009d00 con 0x7ff55c101a40 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff55b7fe700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c102a30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff554009840 tx=0x7ff55400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55400e040 con 0x7ff55c101a40 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff554004c50 con 0x7ff55c101a40 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c1a11f0 con 0x7ff55c101a40 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5540052a0 con 0x7ff55c101a40 2024-05-08T23:15:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.817+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c1a1660 con 0x7ff55c101a40 2024-05-08T23:15:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.818+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c04ec60 con 0x7ff55c101a40 2024-05-08T23:15:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.819+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff55401d470 con 0x7ff55c101a40 2024-05-08T23:15:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.820+0000 7ff56091d700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff5440a1120 0x7ff5440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.820+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4253+0+0 (secure 0 0 0) 0x7ff5540cce10 con 0x7ff55c101a40 2024-05-08T23:15:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.820+0000 7ff55affd700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff5440a1120 0x7ff5440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:22.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.820+0000 7ff55affd700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff5440a1120 0x7ff5440a35e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff55c100bf0 tx=0x7ff54c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:22.825+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff554097610 con 0x7ff55c101a40 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi189 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:22.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:22.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi189 to 8557M 2024-05-08T23:15:22.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:22.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:22.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:22 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi189 2024-05-08T23:15:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi189 to 8557M 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:22.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:22 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:23.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:23.047+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff55c0fd6c0 con 0x7ff5440a1120 2024-05-08T23:15:23.600 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:15:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:15:23.254+0000 7f919e67d700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T23:15:23.600 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:15:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:15:23.263+0000 7f919646a700 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:15:23.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:23.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:15:23.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T23:15:23.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.775 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:23.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:23.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:23.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='osd.5 ' entity='osd.5' 2024-05-08T23:15:23.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] boot 2024-05-08T23:15:23.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: osdmap e35: 6 total, 6 up, 6 in 2024-05-08T23:15:23.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='osd.5 ' entity='osd.5' 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] boot 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: osdmap e35: 6 total, 6 up, 6 in 2024-05-08T23:15:23.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-08T23:15:23.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:15:23.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T23:15:23.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='osd.5 ' entity='osd.5' 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: osd.5 [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] boot 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: osdmap e35: 6 total, 6 up, 6 in 2024-05-08T23:15:23.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:15:24.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:24 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:15:24.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:24 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:15:24.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:24 smithi016 ceph-mon[42541]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:24.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:24 smithi016 ceph-mon[42541]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T23:15:24.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:24 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:15:24.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:24 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:15:24.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:24 smithi189 ceph-mon[46919]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:24.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:24 smithi189 ceph-mon[46919]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T23:15:24.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:24 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:15:24.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:24 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:15:24.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:24 smithi155 ceph-mon[47105]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:24.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:24 smithi155 ceph-mon[47105]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T23:15:25.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:25 smithi016 ceph-mon[42541]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:25.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/115175224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f82436c-9ec7-4fb4-9c5d-5d815b86a408"}]: dispatch 2024-05-08T23:15:25.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/115175224' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f82436c-9ec7-4fb4-9c5d-5d815b86a408"}]': finished 2024-05-08T23:15:25.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:25 smithi016 ceph-mon[42541]: osdmap e37: 7 total, 6 up, 7 in 2024-05-08T23:15:25.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:25 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:25.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:25 smithi189 ceph-mon[46919]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:25.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/115175224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f82436c-9ec7-4fb4-9c5d-5d815b86a408"}]: dispatch 2024-05-08T23:15:25.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/115175224' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f82436c-9ec7-4fb4-9c5d-5d815b86a408"}]': finished 2024-05-08T23:15:25.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:25 smithi189 ceph-mon[46919]: osdmap e37: 7 total, 6 up, 7 in 2024-05-08T23:15:25.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:25 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:25 smithi155 ceph-mon[47105]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 521 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/115175224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f82436c-9ec7-4fb4-9c5d-5d815b86a408"}]: dispatch 2024-05-08T23:15:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/115175224' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f82436c-9ec7-4fb4-9c5d-5d815b86a408"}]': finished 2024-05-08T23:15:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:25 smithi155 ceph-mon[47105]: osdmap e37: 7 total, 6 up, 7 in 2024-05-08T23:15:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:25 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:26.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:26 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2434648168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:26 smithi016 ceph-mon[42541]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T23:15:26.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2434648168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:26 smithi189 ceph-mon[46919]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T23:15:26.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:26.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2434648168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:26.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:26 smithi155 ceph-mon[47105]: osdmap e38: 7 total, 6 up, 7 in 2024-05-08T23:15:26.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:27 smithi016 ceph-mon[42541]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 521 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:27 smithi016 ceph-mon[42541]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T23:15:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:27.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:27 smithi189 ceph-mon[46919]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 521 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:27 smithi189 ceph-mon[46919]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T23:15:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:27 smithi155 ceph-mon[47105]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 521 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:27 smithi155 ceph-mon[47105]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T23:15:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:29 smithi016 ceph-mon[42541]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 522 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:29.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:29 smithi189 ceph-mon[46919]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 522 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:29 smithi155 ceph-mon[47105]: pgmap v112: 1 pgs: 1 remapped+peering; 577 KiB data, 522 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:31.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:31 smithi016 ceph-mon[42541]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:15:31.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T23:15:31.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:31.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:31 smithi189 ceph-mon[46919]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:15:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T23:15:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:31 smithi155 ceph-mon[47105]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T23:15:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T23:15:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:32.795 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:32 smithi016 ceph-mon[42541]: Deploying daemon osd.6 on smithi016 2024-05-08T23:15:32.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:32 smithi189 ceph-mon[46919]: Deploying daemon osd.6 on smithi016 2024-05-08T23:15:32.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:32 smithi155 ceph-mon[47105]: Deploying daemon osd.6 on smithi016 2024-05-08T23:15:33.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:33 smithi189 ceph-mon[46919]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T23:15:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:33 smithi155 ceph-mon[47105]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T23:15:34.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:33 smithi016 ceph-mon[42541]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T23:15:35.911 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:35 smithi016 ceph-mon[42541]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T23:15:35.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:35.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:35.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:35 smithi189 ceph-mon[46919]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T23:15:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:36.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:35 smithi155 ceph-mon[47105]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T23:15:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.144+0000 7ff56091d700 1 -- 172.21.15.16:0/1389946551 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff55c0fd6c0 con 0x7ff5440a1120 2024-05-08T23:15:37.145 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff5440a1120 msgr2=0x7ff5440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff5440a1120 0x7ff5440a35e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff55c100bf0 tx=0x7ff54c00b410 comp rx=0 tx=0).stop 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 msgr2=0x7ff55c102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c102a30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff554009840 tx=0x7ff55400ba50 comp rx=0 tx=0).stop 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 shutdown_connections 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff55c101a40 0x7ff55c102a30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff55c0ffe90 0x7ff55c103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff5440a1120 0x7ff5440a35e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 --2- 172.21.15.16:0/1389946551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff55c101080 0x7ff55c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 >> 172.21.15.16:0/1389946551 conn(0x7ff55c0fb4a0 msgr2=0x7ff55c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 shutdown_connections 2024-05-08T23:15:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:15:37.147+0000 7ff56191f700 1 -- 172.21.15.16:0/1389946551 wait complete. 2024-05-08T23:15:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:37 smithi189 ceph-mon[46919]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:37.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:37.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:37.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.424 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:37 smithi016 ceph-mon[42541]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:37.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:37 smithi155 ceph-mon[47105]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:15:37.517 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T23:15:37.520 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_2... 2024-05-08T23:15:37.520 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T23:15:37.559 INFO:journalctl@ceph.osd.6.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:15:37.754 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:15:38.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:38.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:38.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:38 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:38.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:38.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:38.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:38 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:38 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:38.863 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:15:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:15:38.547+0000 7f5c31cab7c0 -1 osd.6 0 log_to_monitors true 2024-05-08T23:15:39.064 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T23:15:39.064 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T23:15:39.065 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-05-08T23:15:39.065 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-05-08T23:15:39.065 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126327 s, 830 MB/s 2024-05-08T23:15:39.065 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-05-08T23:15:39.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:39 smithi155 ceph-mon[47105]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T23:15:39.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:39 smithi155 ceph-mon[47105]: from='osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:15:39.402 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:39 smithi155 ceph-mon[47105]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:15:39.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:39 smithi189 ceph-mon[46919]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T23:15:39.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:39 smithi189 ceph-mon[46919]: from='osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:15:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:39 smithi189 ceph-mon[46919]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:15:39.469 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-05-08T23:15:39.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:39 smithi016 ceph-mon[42541]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T23:15:39.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:39 smithi016 ceph-mon[42541]: from='osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:15:39.573 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:39 smithi016 ceph-mon[42541]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T23:15:39.689 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:40.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:40.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:40.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:40.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:40.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 -- 172.21.15.155:0/2484050240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 msgr2=0x7f9f9c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 --2- 172.21.15.155:0/2484050240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c1014e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009990 tx=0x7f9f8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 -- 172.21.15.155:0/2484050240 shutdown_connections 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 --2- 172.21.15.155:0/2484050240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c101a20 0x7f9f9c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 --2- 172.21.15.155:0/2484050240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c0fff00 0x7f9f9c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 --2- 172.21.15.155:0/2484050240 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c1014e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 -- 172.21.15.155:0/2484050240 >> 172.21.15.155:0/2484050240 conn(0x7f9f9c0fb4c0 msgr2=0x7f9f9c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 -- 172.21.15.155:0/2484050240 shutdown_connections 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.643+0000 7f9fa2045700 1 -- 172.21.15.155:0/2484050240 wait complete. 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 Processor -- start 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 -- start start 2024-05-08T23:15:40.646 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c0fff00 0x7f9f9c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c101a20 0x7f9f9c19e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19ea60 con 0x7f9f9c0fff00 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19eba0 con 0x7f9f9c101060 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9fa2045700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19ed30 con 0x7f9f9c101a20 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:49590/0 (socket says 172.21.15.155:49590) 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9f9affd700 1 -- 172.21.15.155:0/832786423 learned_addr learned my addr 172.21.15.155:0/832786423 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9f9b7fe700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c0fff00 0x7f9f9c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.644+0000 7f9f9bfff700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c101a20 0x7f9f9c19e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9affd700 1 -- 172.21.15.155:0/832786423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c101a20 msgr2=0x7f9f9c19e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9affd700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c101a20 0x7f9f9c19e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:40.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9affd700 1 -- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c0fff00 msgr2=0x7f9f9c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9affd700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c0fff00 0x7f9f9c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9affd700 1 -- 172.21.15.155:0/832786423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0095f0 con 0x7f9f9c101060 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9bfff700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c101a20 0x7f9f9c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f9affd700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c197460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0038e0 tx=0x7f9f8c005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c01d070 con 0x7f9f9c101060 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f8c00bd50 con 0x7f9f9c101060 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c19efb0 con 0x7f9f9c101060 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00fe20 con 0x7f9f9c101060 2024-05-08T23:15:40.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.645+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c19f3c0 con 0x7f9f9c101060 2024-05-08T23:15:40.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.646+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c04ec60 con 0x7f9f9c101060 2024-05-08T23:15:40.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.647+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9f8c00f5f0 con 0x7f9f9c101060 2024-05-08T23:15:40.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.648+0000 7f9f98ff9700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f840a11a0 0x7f9f840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:40.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.648+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f9f8c0cd130 con 0x7f9f9c101060 2024-05-08T23:15:40.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.648+0000 7f9f9b7fe700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f840a11a0 0x7f9f840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:40.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.648+0000 7f9f9b7fe700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f840a11a0 0x7f9f840a3660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f90005950 tx=0x7f9f9000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:40.651 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.651+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f8c097780 con 0x7f9f9c101060 2024-05-08T23:15:40.851 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:40.851+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f9c069980 con 0x7f9f840a11a0 2024-05-08T23:15:41.114 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:15:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:15:40.709+0000 7f5c2730f700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T23:15:41.114 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:15:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:15:40.719+0000 7f5c1f0fc700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi016 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi016 to 5706M 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='osd.6 ' entity='osd.6' 2024-05-08T23:15:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi016 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi016 to 5706M 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='osd.6 ' entity='osd.6' 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:41.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi016 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi016 to 5706M 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='osd.6 ' entity='osd.6' 2024-05-08T23:15:41.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:41.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:41.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:41.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:42.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:42 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:15:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:42 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:15:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:42 smithi189 ceph-mon[46919]: osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] boot 2024-05-08T23:15:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:42 smithi189 ceph-mon[46919]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T23:15:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:42 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:15:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:42 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:15:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:42 smithi155 ceph-mon[47105]: osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] boot 2024-05-08T23:15:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:42 smithi155 ceph-mon[47105]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T23:15:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:42 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:15:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:42 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:15:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:42 smithi016 ceph-mon[42541]: osd.6 [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] boot 2024-05-08T23:15:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:42 smithi016 ceph-mon[42541]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T23:15:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:15:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:43 smithi189 ceph-mon[46919]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:43 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/3132585162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]: dispatch 2024-05-08T23:15:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:43 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]: dispatch 2024-05-08T23:15:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:43 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]': finished 2024-05-08T23:15:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:43 smithi189 ceph-mon[46919]: osdmap e43: 8 total, 7 up, 8 in 2024-05-08T23:15:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:43 smithi155 ceph-mon[47105]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:43 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/3132585162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]: dispatch 2024-05-08T23:15:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:43 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]: dispatch 2024-05-08T23:15:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:43 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]': finished 2024-05-08T23:15:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:43 smithi155 ceph-mon[47105]: osdmap e43: 8 total, 7 up, 8 in 2024-05-08T23:15:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:43.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:43 smithi016 ceph-mon[42541]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:43 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/3132585162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]: dispatch 2024-05-08T23:15:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:43 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]: dispatch 2024-05-08T23:15:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:43 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96cf8304-3a9c-49d8-8789-543a1ddcc7d5"}]': finished 2024-05-08T23:15:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:43 smithi016 ceph-mon[42541]: osdmap e43: 8 total, 7 up, 8 in 2024-05-08T23:15:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:44.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/2300639895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/2300639895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:44.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/2300639895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:15:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:45 smithi155 ceph-mon[47105]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:45.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:45 smithi016 ceph-mon[42541]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:45.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:45 smithi189 ceph-mon[46919]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:47.681 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:47 smithi155 ceph-mon[47105]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:47 smithi016 ceph-mon[42541]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:47.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:47 smithi189 ceph-mon[46919]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:48.714 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T23:15:48.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:48.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T23:15:48.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:48.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T23:15:48.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:49.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:49 smithi155 ceph-mon[47105]: Deploying daemon osd.7 on smithi155 2024-05-08T23:15:49.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:49 smithi155 ceph-mon[47105]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:49.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:49 smithi016 ceph-mon[42541]: Deploying daemon osd.7 on smithi155 2024-05-08T23:15:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:49 smithi016 ceph-mon[42541]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:49.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:49 smithi189 ceph-mon[46919]: Deploying daemon osd.7 on smithi155 2024-05-08T23:15:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:49 smithi189 ceph-mon[46919]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:51.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:51 smithi016 ceph-mon[42541]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:51.883 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:51 smithi155 ceph-mon[47105]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:51.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:51 smithi189 ceph-mon[46919]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:53.107 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:53.107 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:53.107 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:53.107 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:53 smithi155 ceph-mon[47105]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:53.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:53.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:53.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:53.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:53 smithi016 ceph-mon[42541]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:53.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:53.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:53.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:53.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:53 smithi189 ceph-mon[46919]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:54.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.355+0000 7f9f98ff9700 1 -- 172.21.15.155:0/832786423 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9f9c069980 con 0x7f9f840a11a0 2024-05-08T23:15:54.356 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-05-08T23:15:54.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f840a11a0 msgr2=0x7f9f840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f840a11a0 0x7f9f840a3660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f90005950 tx=0x7f9f9000a300 comp rx=0 tx=0).stop 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 msgr2=0x7f9f9c197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c197460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0038e0 tx=0x7f9f8c005810 comp rx=0 tx=0).stop 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 shutdown_connections 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f840a11a0 0x7f9f840a3660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f9c101a20 0x7f9f9c19e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c0fff00 0x7f9f9c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 --2- 172.21.15.155:0/832786423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f9c101060 0x7f9f9c197460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 >> 172.21.15.155:0/832786423 conn(0x7f9f9c0fb4c0 msgr2=0x7f9f9c069270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 shutdown_connections 2024-05-08T23:15:54.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:15:54.358+0000 7f9fa2045700 1 -- 172.21.15.155:0/832786423 wait complete. 2024-05-08T23:15:54.728 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T23:15:54.730 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-05-08T23:15:54.730 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T23:15:54.773 INFO:journalctl@ceph.osd.7.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:15:54.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:54 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:54 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:54.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:54.939 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.940 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:15:54.940 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.940 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:54 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:54.976 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:15:55.714 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:55 smithi155 ceph-mon[47105]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:55.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:55 smithi189 ceph-mon[46919]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:55 smithi016 ceph-mon[42541]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:56.323 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T23:15:56.323 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T23:15:56.323 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-05-08T23:15:56.324 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-05-08T23:15:56.324 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152662 s, 687 MB/s 2024-05-08T23:15:56.324 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-05-08T23:15:56.332 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:15:56 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:15:56.012+0000 7fc4b2d307c0 -1 osd.7 0 log_to_monitors true 2024-05-08T23:15:56.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:56 smithi189 ceph-mon[46919]: from='osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:15:56.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:56 smithi189 ceph-mon[46919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:15:56.714 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-05-08T23:15:56.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:56 smithi016 ceph-mon[42541]: from='osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:15:56.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:56 smithi016 ceph-mon[42541]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:15:56.935 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:15:56.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:56 smithi155 ceph-mon[47105]: from='osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:15:56.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:56 smithi155 ceph-mon[47105]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi155 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi155 to 5704M 2024-05-08T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:57.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:57 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.918 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 -- 172.21.15.189:0/4092980269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4100fff00 msgr2=0x7ff410100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.918 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 --2- 172.21.15.189:0/4092980269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4100fff00 0x7ff410100320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff404009990 tx=0x7ff404009ca0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.918 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 -- 172.21.15.189:0/4092980269 shutdown_connections 2024-05-08T23:15:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 --2- 172.21.15.189:0/4092980269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101a20 0x7ff410108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 --2- 172.21.15.189:0/4092980269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4100fff00 0x7ff410100320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 --2- 172.21.15.189:0/4092980269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff410101060 0x7ff4101014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.917+0000 7ff418034700 1 -- 172.21.15.189:0/4092980269 >> 172.21.15.189:0/4092980269 conn(0x7ff4100fb4c0 msgr2=0x7ff4100fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:15:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.918+0000 7ff418034700 1 -- 172.21.15.189:0/4092980269 shutdown_connections 2024-05-08T23:15:57.919 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.918+0000 7ff418034700 1 -- 172.21.15.189:0/4092980269 wait complete. 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi155 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi155 to 5704M 2024-05-08T23:15:57.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:57.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:57.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:57 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.923 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.918+0000 7ff418034700 1 Processor -- start 2024-05-08T23:15:57.923 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.918+0000 7ff418034700 1 -- start start 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4100fff00 0x7ff410196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101060 0x7ff410197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 0x7ff41019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41019e8f0 con 0x7ff410101060 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41019ea30 con 0x7ff4100fff00 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41019ebf0 con 0x7ff410101a20 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 0x7ff41019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff415dd0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4100fff00 0x7ff410196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 0x7ff41019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:35854/0 (socket says 172.21.15.189:35854) 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 -- 172.21.15.189:0/3497913288 learned_addr learned my addr 172.21.15.189:0/3497913288 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4155cf700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101060 0x7ff410197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 -- 172.21.15.189:0/3497913288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4100fff00 msgr2=0x7ff410196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.924 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4100fff00 0x7ff410196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 -- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101060 msgr2=0x7ff410197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101060 0x7ff410197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 -- 172.21.15.189:0/3497913288 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040095f0 con 0x7ff410101a20 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff415dd0700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4100fff00 0x7ff410196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4155cf700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101060 0x7ff410197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff4165d1700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 0x7ff41019e320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff408007ee0 tx=0x7ff40800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408017070 con 0x7ff410101a20 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff40800f040 con 0x7ff410101a20 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41019eed0 con 0x7ff410101a20 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408005710 con 0x7ff410101a20 2024-05-08T23:15:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.919+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41019f2e0 con 0x7ff410101a20 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.920+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41004ec60 con 0x7ff410101a20 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.921+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff408005870 con 0x7ff410101a20 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.922+0000 7ff402ffd700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.922+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5117+0+0 (secure 0 0 0) 0x7ff4080cbb50 con 0x7ff410101a20 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.922+0000 7ff415dd0700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.923+0000 7ff415dd0700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff404003780 tx=0x7ff404005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:15:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:57.925+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff408095ec0 con 0x7ff410101a20 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi155 2024-05-08T23:15:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi155 to 5704M 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:15:57.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:57 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:15:58.128 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:15:58.127+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff410069960 con 0x7ff3fc0a11a0 2024-05-08T23:15:58.613 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:15:58 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:15:58.310+0000 7fc4a8394700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T23:15:58.613 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:15:58 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:15:58.319+0000 7fc4a0181700 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:15:58.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='osd.7 ' entity='osd.7' 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] boot 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: osdmap e46: 8 total, 8 up, 8 in 2024-05-08T23:15:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:58 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.890 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:15:58.890 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T23:15:58.890 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.890 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.890 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:58.890 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:58.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:58.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='osd.7 ' entity='osd.7' 2024-05-08T23:15:58.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] boot 2024-05-08T23:15:58.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: osdmap e46: 8 total, 8 up, 8 in 2024-05-08T23:15:58.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:58 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:15:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T23:15:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:15:58.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:15:58.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:15:58.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='osd.7 ' entity='osd.7' 2024-05-08T23:15:58.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: osd.7 [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] boot 2024-05-08T23:15:58.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: osdmap e46: 8 total, 8 up, 8 in 2024-05-08T23:15:58.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:58 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:15:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:59 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:15:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:59 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:15:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:59 smithi189 ceph-mon[46919]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:59 smithi189 ceph-mon[46919]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:15:59 smithi189 ceph-mon[46919]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T23:15:59.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:59 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:15:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:59 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:15:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:59 smithi155 ceph-mon[47105]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:15:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:59 smithi155 ceph-mon[47105]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:15:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:15:59 smithi155 ceph-mon[47105]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T23:16:00.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:59 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:16:00.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:59 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:16:00.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:59 smithi016 ceph-mon[42541]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:00.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:59 smithi016 ceph-mon[42541]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 626 GiB / 626 GiB avail 2024-05-08T23:16:00.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:15:59 smithi016 ceph-mon[42541]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T23:16:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:00 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/2280065174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]: dispatch 2024-05-08T23:16:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:00 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]: dispatch 2024-05-08T23:16:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:00 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]': finished 2024-05-08T23:16:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:00 smithi189 ceph-mon[46919]: osdmap e48: 9 total, 8 up, 9 in 2024-05-08T23:16:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:00 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:00 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/2280065174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]: dispatch 2024-05-08T23:16:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:00 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]: dispatch 2024-05-08T23:16:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:00 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]': finished 2024-05-08T23:16:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:00 smithi155 ceph-mon[47105]: osdmap e48: 9 total, 8 up, 9 in 2024-05-08T23:16:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:00 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:01.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:00 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/2280065174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]: dispatch 2024-05-08T23:16:01.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:00 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]: dispatch 2024-05-08T23:16:01.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:00 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a625c37-f744-4a80-ac8f-44990dbfd83d"}]': finished 2024-05-08T23:16:01.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:00 smithi016 ceph-mon[42541]: osdmap e48: 9 total, 8 up, 9 in 2024-05-08T23:16:01.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:00 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:01.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:01 smithi189 ceph-mon[46919]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:01.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:01 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/250696075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:01.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:01 smithi155 ceph-mon[47105]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:01.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:01 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/250696075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:02.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:01 smithi016 ceph-mon[42541]: pgmap v137: 1 pgs: 1 remapped; 577 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:01 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/250696075' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:02.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:02 smithi189 ceph-mon[46919]: osdmap e49: 9 total, 8 up, 9 in 2024-05-08T23:16:02.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:02 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:02.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:02 smithi155 ceph-mon[47105]: osdmap e49: 9 total, 8 up, 9 in 2024-05-08T23:16:02.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:02 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:03.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:02 smithi016 ceph-mon[42541]: osdmap e49: 9 total, 8 up, 9 in 2024-05-08T23:16:03.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:02 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:03.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:03 smithi189 ceph-mon[46919]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:03.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:03 smithi189 ceph-mon[46919]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T23:16:03.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:03 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:03 smithi155 ceph-mon[47105]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:03 smithi155 ceph-mon[47105]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T23:16:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:03 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:04.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:03 smithi016 ceph-mon[42541]: pgmap v139: 1 pgs: 1 remapped; 577 KiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:03 smithi016 ceph-mon[42541]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T23:16:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:05.868 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:05 smithi189 ceph-mon[46919]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:05 smithi155 ceph-mon[47105]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:06.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:05 smithi016 ceph-mon[42541]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:06.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:06 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:16:06.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:06 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:06 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:16:06.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:06 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:06 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T23:16:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:06 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:07 smithi189 ceph-mon[46919]: Deploying daemon osd.8 on smithi189 2024-05-08T23:16:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:07 smithi189 ceph-mon[46919]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:07 smithi155 ceph-mon[47105]: Deploying daemon osd.8 on smithi189 2024-05-08T23:16:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:07 smithi155 ceph-mon[47105]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:08.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:07 smithi016 ceph-mon[42541]: Deploying daemon osd.8 on smithi189 2024-05-08T23:16:08.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:07 smithi016 ceph-mon[42541]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:09.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:09 smithi189 ceph-mon[46919]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:09.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:09 smithi155 ceph-mon[47105]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:09 smithi016 ceph-mon[42541]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:11.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:11 smithi189 ceph-mon[46919]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:11.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:11 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:11.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:11 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:11.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:11 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:11.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:11 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:11.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:11 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:11.781 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.781+0000 7ff402ffd700 1 -- 172.21.15.189:0/3497913288 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff410069960 con 0x7ff3fc0a11a0 2024-05-08T23:16:11.781 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-05-08T23:16:11.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff3fc0a11a0 msgr2=0x7ff3fc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:11.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff404003780 tx=0x7ff404005fb0 comp rx=0 tx=0).stop 2024-05-08T23:16:11.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 msgr2=0x7ff41019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:11.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 0x7ff41019e320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff408007ee0 tx=0x7ff40800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 shutdown_connections 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff3fc0a11a0 0x7ff3fc0a3660 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff410101a20 0x7ff41019e320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4100fff00 0x7ff410196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 --2- 172.21.15.189:0/3497913288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff410101060 0x7ff410197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 >> 172.21.15.189:0/3497913288 conn(0x7ff4100fb4c0 msgr2=0x7ff410069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 shutdown_connections 2024-05-08T23:16:11.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:11.784+0000 7ff418034700 1 -- 172.21.15.189:0/3497913288 wait complete. 2024-05-08T23:16:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:11 smithi155 ceph-mon[47105]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:11 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:11 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:11 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:11 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:11 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:11 smithi016 ceph-mon[42541]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:11 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:12.211 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T23:16:12.214 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-05-08T23:16:12.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:16:12.244 INFO:journalctl@ceph.osd.8.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:16:12.483 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:16:12.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:12.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:12.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:12 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:13.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:13.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:13.039 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:12 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:12 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:13 smithi189 ceph-mon[46919]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:13 smithi189 ceph-mon[46919]: from='osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:16:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:13 smithi189 ceph-mon[46919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:16:13.920 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:16:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:16:13.568+0000 7f3881df07c0 -1 osd.8 0 log_to_monitors true 2024-05-08T23:16:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:16:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:16:14.010 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-05-08T23:16:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-05-08T23:16:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00914038 s, 1.1 GB/s 2024-05-08T23:16:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-05-08T23:16:14.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:13 smithi016 ceph-mon[42541]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:14.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:13 smithi016 ceph-mon[42541]: from='osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:16:14.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:13 smithi016 ceph-mon[42541]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:16:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:13 smithi155 ceph-mon[47105]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:13 smithi155 ceph-mon[47105]: from='osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:16:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:13 smithi155 ceph-mon[47105]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T23:16:14.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-05-08T23:16:14.629 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:16:15.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:16:15.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T23:16:15.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:15.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:16:15.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:16:15.077 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:15.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:14 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.169 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:16:14 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:16:14.799+0000 7f3877454700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T23:16:15.169 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:16:14 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:16:14.810+0000 7f3873249700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:16:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:16:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T23:16:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:16:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:15.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:14 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T23:16:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T23:16:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:14 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 -- 172.21.15.16:0/1520621177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c0ff680 msgr2=0x7f1b7c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 --2- 172.21.15.16:0/1520621177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c0ff680 0x7f1b7c0ffaa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1b780099c0 tx=0x7f1b78009cd0 comp rx=0 tx=0).stop 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 -- 172.21.15.16:0/1520621177 shutdown_connections 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 --2- 172.21.15.16:0/1520621177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c101220 0x7f1b7c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 --2- 172.21.15.16:0/1520621177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c0ff680 0x7f1b7c0ffaa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 --2- 172.21.15.16:0/1520621177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b7c100880 0x7f1b7c100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 -- 172.21.15.16:0/1520621177 >> 172.21.15.16:0/1520621177 conn(0x7f1b7c0fac40 msgr2=0x7f1b7c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 -- 172.21.15.16:0/1520621177 shutdown_connections 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.630+0000 7f1b835dd700 1 -- 172.21.15.16:0/1520621177 wait complete. 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 Processor -- start 2024-05-08T23:16:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 -- start start 2024-05-08T23:16:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0ff680 0x7f1b7c196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 0x7f1b7c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b7c101220 0x7f1b7c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c19e9d0 con 0x7f1b7c100880 2024-05-08T23:16:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c19eb10 con 0x7f1b7c0ff680 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b835dd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c19eca0 con 0x7f1b7c101220 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b80b78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 0x7f1b7c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.631+0000 7f1b80b78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 0x7f1b7c1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55918/0 (socket says 172.21.15.16:55918) 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 -- 172.21.15.16:0/3924789288 learned_addr learned my addr 172.21.15.16:0/3924789288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b81379700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0ff680 0x7f1b7c196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b81b7a700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b7c101220 0x7f1b7c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 -- 172.21.15.16:0/3924789288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b7c101220 msgr2=0x7f1b7c19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b7c101220 0x7f1b7c19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 -- 172.21.15.16:0/3924789288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0ff680 msgr2=0x7f1b7c196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0ff680 0x7f1b7c196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 -- 172.21.15.16:0/3924789288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b780095f0 con 0x7f1b7c100880 2024-05-08T23:16:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b80b78700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 0x7f1b7c1973d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00ddf0 tx=0x7f1b6c00a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00ac90 con 0x7f1b7c100880 2024-05-08T23:16:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b6c00adf0 con 0x7f1b7c100880 2024-05-08T23:16:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c19ef80 con 0x7f1b7c100880 2024-05-08T23:16:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c0167c0 con 0x7f1b7c100880 2024-05-08T23:16:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.632+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c19f440 con 0x7f1b7c100880 2024-05-08T23:16:15.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.633+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c04ec60 con 0x7f1b7c100880 2024-05-08T23:16:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.635+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1b6c016b30 con 0x7f1b7c100880 2024-05-08T23:16:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.636+0000 7f1b727fc700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1b680a11a0 0x7f1b680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.636+0000 7f1b81379700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1b680a11a0 0x7f1b680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.636+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f1b6c0cbdd0 con 0x7f1b7c100880 2024-05-08T23:16:15.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.636+0000 7f1b81379700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1b680a11a0 0x7f1b680a3660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b78006010 tx=0x7f1b78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:15.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.639+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1b6c095f90 con 0x7f1b7c100880 2024-05-08T23:16:15.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:15.866+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b7c1075d0 con 0x7f1b680a11a0 2024-05-08T23:16:16.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi189 2024-05-08T23:16:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi189 to 5704M 2024-05-08T23:16:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:16:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:16:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:15 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi189 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi189 to 5704M 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:15 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi189 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi189 to 5704M 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:15 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:16:17.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:16:17.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] boot 2024-05-08T23:16:17.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: osdmap e53: 9 total, 9 up, 9 in 2024-05-08T23:16:17.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:17.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:17.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:17.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:16 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] boot 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: osdmap e53: 9 total, 9 up, 9 in 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:17.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:17.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:16 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: osd.8 [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] boot 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: osdmap e53: 9 total, 9 up, 9 in 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:17.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:16 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:17 smithi016 ceph-mon[42541]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:17 smithi016 ceph-mon[42541]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T23:16:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:17 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1825920626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1821a996-5473-46aa-8f43-49fadd38fbe0"}]: dispatch 2024-05-08T23:16:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:17 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1825920626' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1821a996-5473-46aa-8f43-49fadd38fbe0"}]': finished 2024-05-08T23:16:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:17 smithi016 ceph-mon[42541]: osdmap e55: 10 total, 9 up, 10 in 2024-05-08T23:16:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:17 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:18.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:17 smithi189 ceph-mon[46919]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:17 smithi189 ceph-mon[46919]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T23:16:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:17 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1825920626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1821a996-5473-46aa-8f43-49fadd38fbe0"}]: dispatch 2024-05-08T23:16:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:17 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1825920626' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1821a996-5473-46aa-8f43-49fadd38fbe0"}]': finished 2024-05-08T23:16:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:17 smithi189 ceph-mon[46919]: osdmap e55: 10 total, 9 up, 10 in 2024-05-08T23:16:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:17 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:17 smithi155 ceph-mon[47105]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-08T23:16:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:17 smithi155 ceph-mon[47105]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T23:16:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:17 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1825920626' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1821a996-5473-46aa-8f43-49fadd38fbe0"}]: dispatch 2024-05-08T23:16:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:17 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1825920626' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1821a996-5473-46aa-8f43-49fadd38fbe0"}]': finished 2024-05-08T23:16:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:17 smithi155 ceph-mon[47105]: osdmap e55: 10 total, 9 up, 10 in 2024-05-08T23:16:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:17 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:19.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:18 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4232945319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:19.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:18 smithi016 ceph-mon[42541]: osdmap e56: 10 total, 9 up, 10 in 2024-05-08T23:16:19.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:18 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:19.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:18 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4232945319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:19.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:18 smithi189 ceph-mon[46919]: osdmap e56: 10 total, 9 up, 10 in 2024-05-08T23:16:19.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:18 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:19.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:18 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4232945319' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:19.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:18 smithi155 ceph-mon[47105]: osdmap e56: 10 total, 9 up, 10 in 2024-05-08T23:16:19.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:18 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:19 smithi016 ceph-mon[42541]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:19 smithi016 ceph-mon[42541]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T23:16:20.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:19 smithi189 ceph-mon[46919]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:20.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:19 smithi189 ceph-mon[46919]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T23:16:20.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:19 smithi155 ceph-mon[47105]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:19 smithi155 ceph-mon[47105]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T23:16:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:21 smithi016 ceph-mon[42541]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:16:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:21 smithi189 ceph-mon[46919]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:16:22.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:21 smithi155 ceph-mon[47105]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:16:24.093 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:23 smithi016 ceph-mon[42541]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-08T23:16:24.093 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:16:24.093 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:23 smithi189 ceph-mon[46919]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-08T23:16:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:16:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:23 smithi155 ceph-mon[47105]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 285 MiB used, 804 GiB / 805 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-05-08T23:16:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T23:16:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:24.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:24 smithi016 ceph-mon[42541]: Deploying daemon osd.9 on smithi016 2024-05-08T23:16:25.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:24 smithi189 ceph-mon[46919]: Deploying daemon osd.9 on smithi016 2024-05-08T23:16:25.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:24 smithi155 ceph-mon[47105]: Deploying daemon osd.9 on smithi016 2024-05-08T23:16:26.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:25 smithi016 ceph-mon[42541]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-08T23:16:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:25 smithi189 ceph-mon[46919]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-08T23:16:26.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:25 smithi155 ceph-mon[47105]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-05-08T23:16:27.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:27 smithi016 ceph-mon[42541]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:16:27.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:27.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:27 smithi189 ceph-mon[46919]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:16:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:28.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:28.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:27 smithi155 ceph-mon[47105]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T23:16:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.865+0000 7f1b727fc700 1 -- 172.21.15.16:0/3924789288 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b7c1075d0 con 0x7f1b680a11a0 2024-05-08T23:16:29.866 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1b680a11a0 msgr2=0x7f1b680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1b680a11a0 0x7f1b680a3660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b78006010 tx=0x7f1b78000bc0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 msgr2=0x7f1b7c1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 0x7f1b7c1973d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00ddf0 tx=0x7f1b6c00a530 comp rx=0 tx=0).stop 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 shutdown_connections 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1b680a11a0 0x7f1b680a3660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b7c101220 0x7f1b7c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b7c0ff680 0x7f1b7c196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 --2- 172.21.15.16:0/3924789288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7c100880 0x7f1b7c1973d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 >> 172.21.15.16:0/3924789288 conn(0x7f1b7c0fac40 msgr2=0x7f1b7c105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 shutdown_connections 2024-05-08T23:16:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:16:29.868+0000 7f1b835dd700 1 -- 172.21.15.16:0/3924789288 wait complete. 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:16:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.264 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T23:16:30.267 INFO:tasks.cephadm:Deploying osd.10 on smithi155 with /dev/vg_nvme/lv_1... 2024-05-08T23:16:30.267 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:16:30.304 INFO:journalctl@ceph.osd.9.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:16:30.500 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:16:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:31.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:31.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:31.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:31.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:31.859 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:16:31.859 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:16:31.859 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-05-08T23:16:31.859 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-05-08T23:16:31.859 INFO:teuthology.orchestra.run.smithi155.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135979 s, 771 MB/s 2024-05-08T23:16:31.859 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-05-08T23:16:32.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:31 smithi016 ceph-mon[42541]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:32.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:31 smithi016 ceph-mon[42541]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:16:32.114 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:16:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:16:31.653+0000 7faf9779d7c0 -1 osd.9 0 log_to_monitors true 2024-05-08T23:16:32.127 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:31 smithi155 ceph-mon[47105]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:32.127 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:31 smithi155 ceph-mon[47105]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:16:32.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:31 smithi189 ceph-mon[46919]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:32.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:31 smithi189 ceph-mon[46919]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T23:16:32.243 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-05-08T23:16:32.464 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:16:33.114 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:16:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:16:32.846+0000 7faf8ce01700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T23:16:33.130 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:16:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:16:32.862+0000 7faf88bf6700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:16:33.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:16:33.130 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:16:33.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:16:33.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:33.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:33.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:16:33.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:16:33.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:33.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T23:16:33.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T23:16:33.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:33.190 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T23:16:33.191 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:33.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 -- 172.21.15.155:0/1868625021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc108ee0 msgr2=0x7f29dc10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 --2- 172.21.15.155:0/1868625021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc108ee0 0x7f29dc10b350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29d400b0a0 tx=0x7f29d400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 -- 172.21.15.155:0/1868625021 shutdown_connections 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 --2- 172.21.15.155:0/1868625021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc108ee0 0x7f29dc10b350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 --2- 172.21.15.155:0/1868625021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc074f50 0x7f29dc0733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 --2- 172.21.15.155:0/1868625021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 0x7f29dc073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 -- 172.21.15.155:0/1868625021 >> 172.21.15.155:0/1868625021 conn(0x7f29dc0fc580 msgr2=0x7f29dc0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 -- 172.21.15.155:0/1868625021 shutdown_connections 2024-05-08T23:16:33.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.367+0000 7f29e139c700 1 -- 172.21.15.155:0/1868625021 wait complete. 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 Processor -- start 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 -- start start 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 0x7f29dc199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 0x7f29dc199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc108ee0 0x7f29dc1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc1a0b90 con 0x7f29dc073980 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc1a0cd0 con 0x7f29dc074f50 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29e139c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc1a0e60 con 0x7f29dc108ee0 2024-05-08T23:16:33.371 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29da7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 0x7f29dc199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29da7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 0x7f29dc199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:54972/0 (socket says 172.21.15.155:54972) 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29da7fc700 1 -- 172.21.15.155:0/3169538161 learned_addr learned my addr 172.21.15.155:0/3169538161 (peer_addr_for_me v2:172.21.15.155:0/0) 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.368+0000 7f29daffd700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 0x7f29dc199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29db7fe700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc108ee0 0x7f29dc1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29da7fc700 1 -- 172.21.15.155:0/3169538161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc108ee0 msgr2=0x7f29dc1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29da7fc700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc108ee0 0x7f29dc1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29da7fc700 1 -- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 msgr2=0x7f29dc199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29da7fc700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 0x7f29dc199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29da7fc700 1 -- 172.21.15.155:0/3169538161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009d00 con 0x7f29dc074f50 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29daffd700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 0x7f29dc199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29da7fc700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 0x7f29dc199590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c9b0 tx=0x7f29cc00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc007b00 con 0x7f29dc074f50 2024-05-08T23:16:33.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29cc007c60 con 0x7f29dc074f50 2024-05-08T23:16:33.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc1a1140 con 0x7f29dc074f50 2024-05-08T23:16:33.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc018660 con 0x7f29dc074f50 2024-05-08T23:16:33.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.369+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc1a1690 con 0x7f29dc074f50 2024-05-08T23:16:33.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.370+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc107cf0 con 0x7f29dc074f50 2024-05-08T23:16:33.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.371+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f29cc00f450 con 0x7f29dc074f50 2024-05-08T23:16:33.373 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.371+0000 7f29c3fff700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29c40a11a0 0x7f29c40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:33.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.372+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f29cc0cbfa0 con 0x7f29dc074f50 2024-05-08T23:16:33.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.372+0000 7f29daffd700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29c40a11a0 0x7f29c40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:33.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.372+0000 7f29daffd700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29c40a11a0 0x7f29c40a3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005950 tx=0x7f29d00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:33.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.376+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f29cc096050 con 0x7f29dc074f50 2024-05-08T23:16:33.577 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:33.577+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f29dc02ce10 con 0x7f29c40a11a0 2024-05-08T23:16:34.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi016 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi016 to 4279M 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:34.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:34.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi016 2024-05-08T23:16:34.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi016 to 4279M 2024-05-08T23:16:34.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:34.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:34.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:34.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:34.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi016 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi016 to 4279M 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:35.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:34 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:16:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:34 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:16:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:34 smithi016 ceph-mon[42541]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:34 smithi016 ceph-mon[42541]: osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] boot 2024-05-08T23:16:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:34 smithi016 ceph-mon[42541]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T23:16:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:34 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:34 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:16:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:34 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:16:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:34 smithi189 ceph-mon[46919]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:34 smithi189 ceph-mon[46919]: osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] boot 2024-05-08T23:16:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:34 smithi189 ceph-mon[46919]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T23:16:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:34 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:34 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:16:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:34 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:16:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:34 smithi155 ceph-mon[47105]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:34 smithi155 ceph-mon[47105]: osd.9 [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] boot 2024-05-08T23:16:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:34 smithi155 ceph-mon[47105]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T23:16:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:34 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:16:36.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:36.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T23:16:36.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/33713768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]: dispatch 2024-05-08T23:16:36.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]: dispatch 2024-05-08T23:16:36.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]': finished 2024-05-08T23:16:36.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T23:16:36.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:36.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T23:16:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/33713768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]: dispatch 2024-05-08T23:16:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]: dispatch 2024-05-08T23:16:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]': finished 2024-05-08T23:16:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T23:16:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 804 GiB / 805 GiB avail 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/33713768' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]: dispatch 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]: dispatch 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "913decb0-6802-4e12-ace9-bdd7f341456e"}]': finished 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T23:16:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:36 smithi189 ceph-mon[46919]: from='client.? 172.21.15.155:0/2230512897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:37.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:36 smithi155 ceph-mon[47105]: from='client.? 172.21.15.155:0/2230512897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:37.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:36 smithi016 ceph-mon[42541]: from='client.? 172.21.15.155:0/2230512897' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:38.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:37 smithi189 ceph-mon[46919]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:38.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:37 smithi155 ceph-mon[47105]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:37 smithi016 ceph-mon[42541]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:40.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:39 smithi189 ceph-mon[46919]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:39 smithi155 ceph-mon[47105]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:40.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:39 smithi016 ceph-mon[42541]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:41.938 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:41 smithi155 ceph-mon[47105]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:41.938 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:16:41.938 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:42.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:41 smithi189 ceph-mon[46919]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:16:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:42.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:41 smithi016 ceph-mon[42541]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:42.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T23:16:42.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:42.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:42 smithi155 ceph-mon[47105]: Deploying daemon osd.10 on smithi155 2024-05-08T23:16:43.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:42 smithi189 ceph-mon[46919]: Deploying daemon osd.10 on smithi155 2024-05-08T23:16:43.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:42 smithi016 ceph-mon[42541]: Deploying daemon osd.10 on smithi155 2024-05-08T23:16:44.152 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:43 smithi155 ceph-mon[47105]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:44.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:43 smithi189 ceph-mon[46919]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:44.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:43 smithi016 ceph-mon[42541]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:45.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:46.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:45 smithi155 ceph-mon[47105]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:46.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:45 smithi189 ceph-mon[46919]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:46.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:45 smithi016 ceph-mon[42541]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:47.038 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 10 on host 'smithi155' 2024-05-08T23:16:47.039 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.037+0000 7f29c3fff700 1 -- 172.21.15.155:0/3169538161 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f29dc02ce10 con 0x7f29c40a11a0 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29c40a11a0 msgr2=0x7f29c40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29c40a11a0 0x7f29c40a3660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005950 tx=0x7f29d00058e0 comp rx=0 tx=0).stop 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 msgr2=0x7f29dc199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 0x7f29dc199590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c9b0 tx=0x7f29cc00cd70 comp rx=0 tx=0).stop 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 shutdown_connections 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29c40a11a0 0x7f29c40a3660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:47.040 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc108ee0 0x7f29dc1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:47.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc073980 0x7f29dc199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:47.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 --2- 172.21.15.155:0/3169538161 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc074f50 0x7f29dc199590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:47.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 >> 172.21.15.155:0/3169538161 conn(0x7f29dc0fc580 msgr2=0x7f29dc104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:47.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 shutdown_connections 2024-05-08T23:16:47.041 INFO:teuthology.orchestra.run.smithi155.stderr:2024-05-08T23:16:47.040+0000 7f29e139c700 1 -- 172.21.15.155:0/3169538161 wait complete. 2024-05-08T23:16:47.445 DEBUG:teuthology.orchestra.run.smithi155:osd.10> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T23:16:47.447 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-05-08T23:16:47.447 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T23:16:47.479 INFO:journalctl@ceph.osd.10.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:16:47.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:47.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:47.655 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:47.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.656 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:47.679 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:16:47.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.800 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.801 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:47.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:16:49.011 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T23:16:49.011 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T23:16:49.011 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-05-08T23:16:49.011 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-05-08T23:16:49.011 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126729 s, 827 MB/s 2024-05-08T23:16:49.011 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-05-08T23:16:49.215 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:16:48 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:16:48.830+0000 7ff21e7dc7c0 -1 osd.10 0 log_to_monitors true 2024-05-08T23:16:49.395 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-05-08T23:16:49.615 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:16:49.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:49 smithi016 ceph-mon[42541]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:49.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:49 smithi016 ceph-mon[42541]: from='osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:16:49.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:49 smithi016 ceph-mon[42541]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:16:49.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:49 smithi189 ceph-mon[46919]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:49 smithi189 ceph-mon[46919]: from='osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:16:49.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:49 smithi189 ceph-mon[46919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:16:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:49 smithi155 ceph-mon[47105]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:49 smithi155 ceph-mon[47105]: from='osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:16:49.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:49 smithi155 ceph-mon[47105]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T23:16:50.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 -- 172.21.15.189:0/1988705792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480101000 msgr2=0x7f7480101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:50.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 --2- 172.21.15.189:0/1988705792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480101000 0x7f7480101420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f74780098f0 tx=0x7f7478009c00 comp rx=0 tx=0).stop 2024-05-08T23:16:50.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 -- 172.21.15.189:0/1988705792 shutdown_connections 2024-05-08T23:16:50.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 --2- 172.21.15.189:0/1988705792 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480102ba0 0x7f7480109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:50.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 --2- 172.21.15.189:0/1988705792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480101000 0x7f7480101420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 --2- 172.21.15.189:0/1988705792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102200 0x7f7480102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 -- 172.21.15.189:0/1988705792 >> 172.21.15.189:0/1988705792 conn(0x7f74800fc5a0 msgr2=0x7f74800fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 -- 172.21.15.189:0/1988705792 shutdown_connections 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.735+0000 7f74859a0700 1 -- 172.21.15.189:0/1988705792 wait complete. 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 Processor -- start 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 -- start start 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480101000 0x7f7480196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 0x7f7480197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102ba0 0x7f748019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748019e9b0 con 0x7f7480102ba0 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748019eaf0 con 0x7f7480101000 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f74859a0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748019ec80 con 0x7f7480102200 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f747e7fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 0x7f7480197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:50.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f747e7fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 0x7f7480197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:45978/0 (socket says 172.21.15.189:45978) 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f747e7fc700 1 -- 172.21.15.189:0/1640431728 learned_addr learned my addr 172.21.15.189:0/1640431728 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f747effd700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480101000 0x7f7480196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.736+0000 7f747f7fe700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102ba0 0x7f748019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747e7fc700 1 -- 172.21.15.189:0/1640431728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480101000 msgr2=0x7f7480196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747e7fc700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480101000 0x7f7480196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747e7fc700 1 -- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102ba0 msgr2=0x7f748019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747e7fc700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102ba0 0x7f748019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747e7fc700 1 -- 172.21.15.189:0/1640431728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478009520 con 0x7f7480102200 2024-05-08T23:16:50.741 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747f7fe700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102ba0 0x7f748019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f747e7fc700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 0x7f7480197400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f746800d980 tx=0x7f746800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468009b40 con 0x7f7480102200 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f746800b6e0 con 0x7f7480102200 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748019ef60 con 0x7f7480102200 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800f3c0 con 0x7f7480102200 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.737+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748019f3d0 con 0x7f7480102200 2024-05-08T23:16:50.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.738+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74801085c0 con 0x7f7480102200 2024-05-08T23:16:50.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.739+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7468010460 con 0x7f7480102200 2024-05-08T23:16:50.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.740+0000 7f748499e700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f746c0a11a0 0x7f746c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:16:50.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.740+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f74680cc100 con 0x7f7480102200 2024-05-08T23:16:50.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.740+0000 7f747effd700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f746c0a11a0 0x7f746c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:16:50.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.740+0000 7f747effd700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f746c0a11a0 0x7f746c0a3660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7478009fd0 tx=0x7f7478019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:16:50.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.743+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7468095f60 con 0x7f7480102200 2024-05-08T23:16:50.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi155 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi155 to 4278M 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:50.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:50.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:50 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.946 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:16:50.946+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f748002ce10 con 0x7f746c0a11a0 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi155 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi155 to 4278M 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:50.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:50.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:50.967 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:50 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi155 2024-05-08T23:16:51.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi155 to 4278M 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:16:51.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:50 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:16:51.619 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:16:51 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:16:51.307+0000 7ff214641700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T23:16:51.619 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:16:51 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:16:51.321+0000 7ff20bc2d700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:51.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:51 smithi189 ceph-mon[46919]: from='osd.10 ' entity='osd.10' 2024-05-08T23:16:51.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:51.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:51 smithi155 ceph-mon[47105]: from='osd.10 ' entity='osd.10' 2024-05-08T23:16:52.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:52.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:51 smithi016 ceph-mon[42541]: from='osd.10 ' entity='osd.10' 2024-05-08T23:16:52.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:16:52.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:16:52.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:52.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:52.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] boot 2024-05-08T23:16:52.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T23:16:52.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:52 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] boot 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T23:16:52.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:52 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:53.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:16:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:16:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:16:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: osd.10 [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] boot 2024-05-08T23:16:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T23:16:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:52 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/2610401674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]: dispatch 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]: dispatch 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]': finished 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: osdmap e67: 12 total, 11 up, 12 in 2024-05-08T23:16:53.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:53.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T23:16:53.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:53.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:53 smithi155 ceph-mon[47105]: from='client.? 172.21.15.189:0/3588217322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:54.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/2610401674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]: dispatch 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]: dispatch 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]': finished 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: osdmap e67: 12 total, 11 up, 12 in 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:53 smithi016 ceph-mon[42541]: from='client.? 172.21.15.189:0/3588217322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:54.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/2610401674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]: dispatch 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]: dispatch 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd"}]': finished 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: osdmap e67: 12 total, 11 up, 12 in 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:53 smithi189 ceph-mon[46919]: from='client.? 172.21.15.189:0/3588217322' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T23:16:55.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:55 smithi016 ceph-mon[42541]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T23:16:55.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:55 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:55.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:55 smithi016 ceph-mon[42541]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 202 KiB/s, 0 objects/s recovering 2024-05-08T23:16:55.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:55 smithi189 ceph-mon[46919]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T23:16:55.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:55 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:55.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:55 smithi189 ceph-mon[46919]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 202 KiB/s, 0 objects/s recovering 2024-05-08T23:16:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:55 smithi155 ceph-mon[47105]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T23:16:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:55 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:16:55.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:55 smithi155 ceph-mon[47105]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 202 KiB/s, 0 objects/s recovering 2024-05-08T23:16:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:57 smithi016 ceph-mon[42541]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:16:57.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:57 smithi189 ceph-mon[46919]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:16:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:57 smithi155 ceph-mon[47105]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 343 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T23:16:59.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:59 smithi016 ceph-mon[42541]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-08T23:16:59.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:59 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:16:59.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:59 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:59.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:16:59 smithi016 ceph-mon[42541]: Deploying daemon osd.11 on smithi189 2024-05-08T23:16:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:59 smithi189 ceph-mon[46919]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-08T23:16:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:59 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:16:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:59 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:16:59 smithi189 ceph-mon[46919]: Deploying daemon osd.11 on smithi189 2024-05-08T23:16:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:59 smithi155 ceph-mon[47105]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-05-08T23:16:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:59 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T23:16:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:59 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:16:59.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:16:59 smithi155 ceph-mon[47105]: Deploying daemon osd.11 on smithi189 2024-05-08T23:17:01.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:01 smithi016 ceph-mon[42541]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T23:17:01.891 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:01 smithi189 ceph-mon[46919]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T23:17:01.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:01 smithi155 ceph-mon[47105]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T23:17:03.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:03 smithi189 ceph-mon[46919]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T23:17:03.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:03 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:03.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:03 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:03.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:03 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:03 smithi155 ceph-mon[47105]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T23:17:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:03 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:03 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:03 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:04.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:03 smithi016 ceph-mon[42541]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T23:17:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:03 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:05.103 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-05-08T23:17:05.103 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.103+0000 7f748499e700 1 -- 172.21.15.189:0/1640431728 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f748002ce10 con 0x7f746c0a11a0 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f746c0a11a0 msgr2=0x7f746c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f746c0a11a0 0x7f746c0a3660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7478009fd0 tx=0x7f7478019040 comp rx=0 tx=0).stop 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 msgr2=0x7f7480197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 0x7f7480197400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f746800d980 tx=0x7f746800dc90 comp rx=0 tx=0).stop 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 shutdown_connections 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7480102ba0 0x7f748019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7480101000 0x7f7480196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f746c0a11a0 0x7f746c0a3660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 --2- 172.21.15.189:0/1640431728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7480102200 0x7f7480197400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:05.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 >> 172.21.15.189:0/1640431728 conn(0x7f74800fc5a0 msgr2=0x7f7480105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:05.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 shutdown_connections 2024-05-08T23:17:05.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:05.105+0000 7f74859a0700 1 -- 172.21.15.189:0/1640431728 wait complete. 2024-05-08T23:17:05.525 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T23:17:05.527 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T23:17:05.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:17:05.557 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.557 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.557 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:05.557 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:05.557 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.557 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:05.558 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.558 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.558 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:05 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:05.564 INFO:journalctl@ceph.osd.11.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:17:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:05 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:05 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 -- 172.21.15.16:0/514037419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 msgr2=0x7f1a8c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 --2- 172.21.15.16:0/514037419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c100320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009990 tx=0x7f1a88009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 -- 172.21.15.16:0/514037419 shutdown_connections 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 --2- 172.21.15.16:0/514037419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 0x7f1a8c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 --2- 172.21.15.16:0/514037419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c100320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 --2- 172.21.15.16:0/514037419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c101060 0x7f1a8c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 -- 172.21.15.16:0/514037419 >> 172.21.15.16:0/514037419 conn(0x7f1a8c0fb4a0 msgr2=0x7f1a8c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 -- 172.21.15.16:0/514037419 shutdown_connections 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.798+0000 7f1a937d1700 1 -- 172.21.15.16:0/514037419 wait complete. 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 Processor -- start 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 -- start start 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c101060 0x7f1a8c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 0x7f1a8c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c19e9e0 con 0x7f1a8c0fff00 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c19eb20 con 0x7f1a8c101a20 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a937d1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c19ecb0 con 0x7f1a8c101060 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.799+0000 7f1a9156d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52486/0 (socket says 172.21.15.16:52486) 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 -- 172.21.15.16:0/3872665498 learned_addr learned my addr 172.21.15.16:0/3872665498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a90d6c700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c101060 0x7f1a8c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a91d6e700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 0x7f1a8c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 -- 172.21.15.16:0/3872665498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c101060 msgr2=0x7f1a8c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c101060 0x7f1a8c197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 -- 172.21.15.16:0/3872665498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 msgr2=0x7f1a8c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 0x7f1a8c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 -- 172.21.15.16:0/3872665498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a880095f0 con 0x7f1a8c0fff00 2024-05-08T23:17:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a91d6e700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 0x7f1a8c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a9156d700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c196ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b610 tx=0x7f1a88004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8801d070 con 0x7f1a8c0fff00 2024-05-08T23:17:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c19ef30 con 0x7f1a8c0fff00 2024-05-08T23:17:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a8800bdd0 con 0x7f1a8c0fff00 2024-05-08T23:17:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88022db0 con 0x7f1a8c0fff00 2024-05-08T23:17:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.800+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c19f290 con 0x7f1a8c0fff00 2024-05-08T23:17:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.801+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c04ec60 con 0x7f1a8c0fff00 2024-05-08T23:17:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.803+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1a88022570 con 0x7f1a8c0fff00 2024-05-08T23:17:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.803+0000 7f1a827fc700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1a780a1120 0x7f1a780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.803+0000 7f1a90d6c700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1a780a1120 0x7f1a780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.803+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f1a880cde90 con 0x7f1a8c0fff00 2024-05-08T23:17:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.804+0000 7f1a90d6c700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1a780a1120 0x7f1a780a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009c60 tx=0x7f1a7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:06.807+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1a88097c90 con 0x7f1a8c0fff00 2024-05-08T23:17:06.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:17:06 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:17:06.606+0000 7f2d2a6cf7c0 -1 osd.11 0 log_to_monitors true 2024-05-08T23:17:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.018+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1a8c10b790 con 0x7f1a8c0fff00 2024-05-08T23:17:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.018+0000 7f1a827fc700 1 -- 172.21.15.16:0/3872665498 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1a8809b2b0 con 0x7f1a8c0fff00 2024-05-08T23:17:07.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.020+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1a780a1120 msgr2=0x7f1a780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.020+0000 7f1a937d1700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1a780a1120 0x7f1a780a35e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009c60 tx=0x7f1a7c009260 comp rx=0 tx=0).stop 2024-05-08T23:17:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.020+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 msgr2=0x7f1a8c196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c196ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b610 tx=0x7f1a88004710 comp rx=0 tx=0).stop 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 shutdown_connections 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a8c101a20 0x7f1a8c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0fff00 0x7f1a8c196ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1a780a1120 0x7f1a780a35e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 --2- 172.21.15.16:0/3872665498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a8c101060 0x7f1a8c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 >> 172.21.15.16:0/3872665498 conn(0x7f1a8c0fb4a0 msgr2=0x7f1a8c0689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 shutdown_connections 2024-05-08T23:17:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:07.021+0000 7f1a937d1700 1 -- 172.21.15.16:0/3872665498 wait complete. 2024-05-08T23:17:07.398 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1715210211,"num_in_osds":12,"osd_in_since":1715210212,"num_remapped_pgs":0} 2024-05-08T23:17:07.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:07 smithi016 ceph-mon[42541]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:07.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:07 smithi016 ceph-mon[42541]: from='osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:17:07.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:07 smithi016 ceph-mon[42541]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:17:07.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:07 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3872665498' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:07 smithi189 ceph-mon[46919]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:07 smithi189 ceph-mon[46919]: from='osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:17:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:07 smithi189 ceph-mon[46919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:17:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:07 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3872665498' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:07 smithi155 ceph-mon[47105]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:07 smithi155 ceph-mon[47105]: from='osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:17:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:07 smithi155 ceph-mon[47105]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T23:17:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:07 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3872665498' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:08.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:17:08.658 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: Detected new or changed devices on smithi189 2024-05-08T23:17:08.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi189 to 4278M 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:08.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:08 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: Detected new or changed devices on smithi189 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi189 to 4278M 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:08.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:08 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: Detected new or changed devices on smithi189 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:17:08.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi189 to 4278M 2024-05-08T23:17:08.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:08.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:08.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:08.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:08 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:09.622 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:17:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:17:09.289+0000 7f2d1fd33700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T23:17:09.622 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:17:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:17:09.304+0000 7f2d17b20700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T23:17:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 -- 172.21.15.16:0/3421468529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101080 msgr2=0x7f3e20101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:09.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 --2- 172.21.15.16:0/3421468529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101080 0x7f3e20101500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009990 tx=0x7f3e10009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 -- 172.21.15.16:0/3421468529 shutdown_connections 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 --2- 172.21.15.16:0/3421468529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e20101a40 0x7f3e20108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 --2- 172.21.15.16:0/3421468529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e200ffe90 0x7f3e201002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 --2- 172.21.15.16:0/3421468529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101080 0x7f3e20101500 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.653+0000 7f3e2820f700 1 -- 172.21.15.16:0/3421468529 >> 172.21.15.16:0/3421468529 conn(0x7f3e200fb4a0 msgr2=0x7f3e200fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 -- 172.21.15.16:0/3421468529 shutdown_connections 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 -- 172.21.15.16:0/3421468529 wait complete. 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 Processor -- start 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 -- start start 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e200ffe90 0x7f3e20196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e20101080 0x7f3e20197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 0x7f3e2019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:09.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2019e930 con 0x7f3e20101a40 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2019ea70 con 0x7f3e200ffe90 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.654+0000 7f3e2820f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2019ec00 con 0x7f3e20101080 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 0x7f3e2019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 0x7f3e2019e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52492/0 (socket says 172.21.15.16:52492) 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 -- 172.21.15.16:0/1935165287 learned_addr learned my addr 172.21.15.16:0/1935165287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e25fab700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e200ffe90 0x7f3e20196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e257aa700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e20101080 0x7f3e20197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 -- 172.21.15.16:0/1935165287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e20101080 msgr2=0x7f3e20197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e20101080 0x7f3e20197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 -- 172.21.15.16:0/1935165287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e200ffe90 msgr2=0x7f3e20196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e200ffe90 0x7f3e20196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 -- 172.21.15.16:0/1935165287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100095f0 con 0x7f3e20101a40 2024-05-08T23:17:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e267ac700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 0x7f3e2019e360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e18007f10 tx=0x7f3e1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18017070 con 0x7f3e20101a40 2024-05-08T23:17:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e1800f040 con 0x7f3e20101a40 2024-05-08T23:17:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2019eee0 con 0x7f3e20101a40 2024-05-08T23:17:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.655+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18005570 con 0x7f3e20101a40 2024-05-08T23:17:09.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.656+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2019f300 con 0x7f3e20101a40 2024-05-08T23:17:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.656+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2004ec60 con 0x7f3e20101a40 2024-05-08T23:17:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.658+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3e1800db20 con 0x7f3e20101a40 2024-05-08T23:17:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.659+0000 7f3e16ffd700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3e0c0a1120 0x7f3e0c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.659+0000 7f3e25fab700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3e0c0a1120 0x7f3e0c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.659+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f3e180cc0e0 con 0x7f3e20101a40 2024-05-08T23:17:09.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.660+0000 7f3e25fab700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3e0c0a1120 0x7f3e0c0a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e20100bf0 tx=0x7f3e1c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.663+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3e18095e30 con 0x7f3e20101a40 2024-05-08T23:17:09.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: purged_snaps scrub starts 2024-05-08T23:17:09.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: purged_snaps scrub ok 2024-05-08T23:17:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:17:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:17:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:09.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:09 smithi016 ceph-mon[42541]: from='osd.11 ' entity='osd.11' 2024-05-08T23:17:09.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.878+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f3e2010b610 con 0x7f3e20101a40 2024-05-08T23:17:09.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.879+0000 7f3e16ffd700 1 -- 172.21.15.16:0/1935165287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) v1 ==== 74+0+133 (secure 0 0 0) 0x7f3e18099450 con 0x7f3e20101a40 2024-05-08T23:17:09.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3e0c0a1120 msgr2=0x7f3e0c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3e0c0a1120 0x7f3e0c0a35e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e20100bf0 tx=0x7f3e1c00a3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 msgr2=0x7f3e2019e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 0x7f3e2019e360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e18007f10 tx=0x7f3e1800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 shutdown_connections 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e20101a40 0x7f3e2019e360 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e200ffe90 0x7f3e20196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3e0c0a1120 0x7f3e0c0a35e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 --2- 172.21.15.16:0/1935165287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e20101080 0x7f3e20197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 >> 172.21.15.16:0/1935165287 conn(0x7f3e200fb4a0 msgr2=0x7f3e200fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 shutdown_connections 2024-05-08T23:17:09.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:09.881+0000 7f3e2820f700 1 -- 172.21.15.16:0/1935165287 wait complete. 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: purged_snaps scrub starts 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: purged_snaps scrub ok 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:09 smithi189 ceph-mon[46919]: from='osd.11 ' entity='osd.11' 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: purged_snaps scrub starts 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: purged_snaps scrub ok 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:09 smithi155 ceph-mon[47105]: from='osd.11 ' entity='osd.11' 2024-05-08T23:17:10.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1715210211,"num_in_osds":12,"osd_in_since":1715210212,"num_remapped_pgs":0} 2024-05-08T23:17:10.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:10 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:10.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:10 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1935165287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:10.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:10 smithi189 ceph-mon[46919]: osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] boot 2024-05-08T23:17:10.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:10 smithi189 ceph-mon[46919]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T23:17:10.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:10 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:10.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:10 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:10 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1935165287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:10 smithi155 ceph-mon[47105]: osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] boot 2024-05-08T23:17:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:10 smithi155 ceph-mon[47105]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T23:17:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:10 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:10 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1935165287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:10 smithi016 ceph-mon[42541]: osd.11 [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] boot 2024-05-08T23:17:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:10 smithi016 ceph-mon[42541]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T23:17:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:10 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:17:11.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T23:17:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:17:11.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:11 smithi189 ceph-mon[46919]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:11.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:11 smithi189 ceph-mon[46919]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T23:17:11.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:11 smithi016 ceph-mon[42541]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:11.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:11 smithi016 ceph-mon[42541]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T23:17:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:11 smithi155 ceph-mon[47105]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-05-08T23:17:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:11 smithi155 ceph-mon[47105]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 -- 172.21.15.16:0/524722180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1001b0 msgr2=0x7f8e7c100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 --2- 172.21.15.16:0/524722180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c100630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009990 tx=0x7f8e70009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 -- 172.21.15.16:0/524722180 shutdown_connections 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 --2- 172.21.15.16:0/524722180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c100b70 0x7f8e7c100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 --2- 172.21.15.16:0/524722180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e7c0ff850 0x7f8e7c0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 --2- 172.21.15.16:0/524722180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c100630 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 -- 172.21.15.16:0/524722180 >> 172.21.15.16:0/524722180 conn(0x7f8e7c0fb450 msgr2=0x7f8e7c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 -- 172.21.15.16:0/524722180 shutdown_connections 2024-05-08T23:17:12.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.471+0000 7f8e82825700 1 -- 172.21.15.16:0/524722180 wait complete. 2024-05-08T23:17:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 Processor -- start 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 -- start start 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e7c0ff850 0x7f8e7c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 0x7f8e7c1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1a0a70 con 0x7f8e7c100b70 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1a0bb0 con 0x7f8e7c0ff850 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e82825700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1a0d40 con 0x7f8e7c1001b0 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e80dc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 0x7f8e7c1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.472+0000 7f8e80dc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 0x7f8e7c1a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52514/0 (socket says 172.21.15.16:52514) 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 -- 172.21.15.16:0/2322143482 learned_addr learned my addr 172.21.15.16:0/2322143482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:12.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e7bfff700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e7c0ff850 0x7f8e7c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e7b7fe700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 -- 172.21.15.16:0/2322143482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c1001b0 msgr2=0x7f8e7c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 -- 172.21.15.16:0/2322143482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e7c0ff850 msgr2=0x7f8e7c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e7c0ff850 0x7f8e7c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 -- 172.21.15.16:0/2322143482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700095f0 con 0x7f8e7c100b70 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e7b7fe700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e80dc2700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 0x7f8e7c1a04a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400ef60 tx=0x7f8e7400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74018070 con 0x7f8e7c100b70 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c1a0fc0 con 0x7f8e7c100b70 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e7400f040 con 0x7f8e7c100b70 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400cd10 con 0x7f8e7c100b70 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.473+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c1a1430 con 0x7f8e7c100b70 2024-05-08T23:17:12.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.474+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c10a630 con 0x7f8e7c100b70 2024-05-08T23:17:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.476+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8e74004ad0 con 0x7f8e7c100b70 2024-05-08T23:17:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.476+0000 7f8e797fa700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e640a11a0 0x7f8e640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.476+0000 7f8e7bfff700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e640a11a0 0x7f8e640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:12.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.476+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8e740cca20 con 0x7f8e7c100b70 2024-05-08T23:17:12.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.477+0000 7f8e7bfff700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e640a11a0 0x7f8e640a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005950 tx=0x7f8e6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:12.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.480+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8e74096640 con 0x7f8e7c100b70 2024-05-08T23:17:12.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.692+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8e7c04ec60 con 0x7f8e7c100b70 2024-05-08T23:17:12.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.693+0000 7f8e797fa700 1 -- 172.21.15.16:0/2322143482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8e74099c60 con 0x7f8e7c100b70 2024-05-08T23:17:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e640a11a0 msgr2=0x7f8e640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e640a11a0 0x7f8e640a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005950 tx=0x7f8e6c00b410 comp rx=0 tx=0).stop 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 msgr2=0x7f8e7c1a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 0x7f8e7c1a04a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400ef60 tx=0x7f8e7400ef90 comp rx=0 tx=0).stop 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 shutdown_connections 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c100b70 0x7f8e7c1a04a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e7c0ff850 0x7f8e7c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e640a11a0 0x7f8e640a3660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 --2- 172.21.15.16:0/2322143482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7c1001b0 0x7f8e7c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 >> 172.21.15.16:0/2322143482 conn(0x7f8e7c0fb450 msgr2=0x7f8e7c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 shutdown_connections 2024-05-08T23:17:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:12.695+0000 7f8e82825700 1 -- 172.21.15.16:0/2322143482 wait complete. 2024-05-08T23:17:13.065 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1715210230,"num_in_osds":12,"osd_in_since":1715210212,"num_remapped_pgs":0} 2024-05-08T23:17:13.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:17:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:17:13.777 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:13 smithi016 ceph-mon[42541]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:13.777 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:13 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2322143482' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:13.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:13 smithi189 ceph-mon[46919]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:13.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:13 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2322143482' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:13 smithi155 ceph-mon[47105]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:13 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2322143482' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T23:17:14.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 -- 172.21.15.16:0/1363039346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900106340 msgr2=0x7f490010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:14.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 --2- 172.21.15.16:0/1363039346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900106340 0x7f490010dec0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b0a0 tx=0x7f48f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 -- 172.21.15.16:0/1363039346 shutdown_connections 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 --2- 172.21.15.16:0/1363039346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900106340 0x7f490010dec0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 --2- 172.21.15.16:0/1363039346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900068bb0 0x7f4900068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 --2- 172.21.15.16:0/1363039346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4900069560 0x7f4900105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 -- 172.21.15.16:0/1363039346 >> 172.21.15.16:0/1363039346 conn(0x7f4900075f80 msgr2=0x7f4900076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 -- 172.21.15.16:0/1363039346 shutdown_connections 2024-05-08T23:17:14.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 -- 172.21.15.16:0/1363039346 wait complete. 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 Processor -- start 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.271+0000 7f4906a11700 1 -- start start 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4906a11700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 0x7f49000fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4906a11700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4900069560 0x7f49001004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4906a11700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900106340 0x7f4900100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4906a11700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a5770 con 0x7f4900068bb0 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4906a11700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a58e0 con 0x7f4900106340 2024-05-08T23:17:14.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4906a11700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001a5aa0 con 0x7f4900069560 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 0x7f49000fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 0x7f49000fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52530/0 (socket says 172.21.15.16:52530) 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 -- 172.21.15.16:0/3527852119 learned_addr learned my addr 172.21.15.16:0/3527852119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48ff7fe700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4900069560 0x7f49001004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4904fae700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900106340 0x7f4900100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 -- 172.21.15.16:0/3527852119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4900069560 msgr2=0x7f49001004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4900069560 0x7f49001004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 -- 172.21.15.16:0/3527852119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900106340 msgr2=0x7f4900100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900106340 0x7f4900100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 -- 172.21.15.16:0/3527852119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8009d00 con 0x7f4900068bb0 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f4904fae700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900106340 0x7f4900100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.272+0000 7f48fffff700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 0x7f49000fffa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c9b0 tx=0x7f48f000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:14.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.273+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0007b00 con 0x7f4900068bb0 2024-05-08T23:17:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.273+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49001a5d80 con 0x7f4900068bb0 2024-05-08T23:17:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.273+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48f0007c60 con 0x7f4900068bb0 2024-05-08T23:17:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.273+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0018660 con 0x7f4900068bb0 2024-05-08T23:17:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.273+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49001a6220 con 0x7f4900068bb0 2024-05-08T23:17:14.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.274+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900109530 con 0x7f4900068bb0 2024-05-08T23:17:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.275+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f48f000f450 con 0x7f4900068bb0 2024-05-08T23:17:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.276+0000 7f48fd7fa700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48e80a1120 0x7f48e80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.276+0000 7f48ff7fe700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48e80a1120 0x7f48e80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.276+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48f00cc760 con 0x7f4900068bb0 2024-05-08T23:17:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.276+0000 7f48ff7fe700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48e80a1120 0x7f48e80a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f48f4005fd0 tx=0x7f48f400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.280+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48f0096420 con 0x7f4900068bb0 2024-05-08T23:17:14.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.492+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4900101480 con 0x7f4900068bb0 2024-05-08T23:17:14.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.493+0000 7f48fd7fa700 1 -- 172.21.15.16:0/3527852119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18433 (secure 0 0 0) 0x7f48f0099a40 con 0x7f4900068bb0 2024-05-08T23:17:14.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:14.494 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:11.293661+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48e80a1120 msgr2=0x7f48e80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48e80a1120 0x7f48e80a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f48f4005fd0 tx=0x7f48f400af20 comp rx=0 tx=0).stop 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 msgr2=0x7f49000fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 0x7f49000fffa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f48f000c9b0 tx=0x7f48f000cd70 comp rx=0 tx=0).stop 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 shutdown_connections 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4900106340 0x7f4900100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900068bb0 0x7f49000fffa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48e80a1120 0x7f48e80a35e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 --2- 172.21.15.16:0/3527852119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4900069560 0x7f49001004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 >> 172.21.15.16:0/3527852119 conn(0x7f4900075f80 msgr2=0x7f4900107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 shutdown_connections 2024-05-08T23:17:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:14.496+0000 7f4906a11700 1 -- 172.21.15.16:0/3527852119 wait complete. 2024-05-08T23:17:14.781 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:14 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3527852119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:14.891 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:17:14.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T23:17:14.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:14 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3527852119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:14.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:14 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3527852119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:17:15.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:15 smithi016 ceph-mon[42541]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:15.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:15 smithi189 ceph-mon[46919]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:15.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:15 smithi155 ceph-mon[47105]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 -- 172.21.15.16:0/2698224637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7100692f0 msgr2=0x7fb71010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 --2- 172.21.15.16:0/2698224637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7100692f0 0x7fb71010de60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb70800b0a0 tx=0x7fb70800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 -- 172.21.15.16:0/2698224637 shutdown_connections 2024-05-08T23:17:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 --2- 172.21.15.16:0/2698224637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7100692f0 0x7fb71010de60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 --2- 172.21.15.16:0/2698224637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb710107020 0x7fb710107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 --2- 172.21.15.16:0/2698224637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710068950 0x7fb710068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 -- 172.21.15.16:0/2698224637 >> 172.21.15.16:0/2698224637 conn(0x7fb710075f80 msgr2=0x7fb710076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 -- 172.21.15.16:0/2698224637 shutdown_connections 2024-05-08T23:17:16.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.123+0000 7fb716571700 1 -- 172.21.15.16:0/2698224637 wait complete. 2024-05-08T23:17:16.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.124+0000 7fb716571700 1 Processor -- start 2024-05-08T23:17:16.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.124+0000 7fb716571700 1 -- start start 2024-05-08T23:17:16.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 0x7fb710197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7100692f0 0x7fb710197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710107020 0x7fb71019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71019c320 con 0x7fb710068950 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71019c490 con 0x7fb710107020 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71019c650 con 0x7fb7100692f0 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 0x7fb710197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 0x7fb710197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52540/0 (socket says 172.21.15.16:52540) 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 -- 172.21.15.16:0/1323895897 learned_addr learned my addr 172.21.15.16:0/1323895897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70f7fe700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7100692f0 0x7fb710197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb714b0e700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710107020 0x7fb71019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 -- 172.21.15.16:0/1323895897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7100692f0 msgr2=0x7fb710197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:16.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7100692f0 0x7fb710197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 -- 172.21.15.16:0/1323895897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710107020 msgr2=0x7fb71019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710107020 0x7fb71019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 -- 172.21.15.16:0/1323895897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708009d00 con 0x7fb710068950 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb714b0e700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710107020 0x7fb71019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70f7fe700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7100692f0 0x7fb710197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70ffff700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 0x7fb710197420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d950 tx=0x7fb70000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700009b40 con 0x7fb710068950 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb700009ca0 con 0x7fb710068950 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71019c930 con 0x7fb710068950 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f620 con 0x7fb710068950 2024-05-08T23:17:16.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.125+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7101a4190 con 0x7fb710068950 2024-05-08T23:17:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.127+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb710109540 con 0x7fb710068950 2024-05-08T23:17:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.128+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb700010460 con 0x7fb710068950 2024-05-08T23:17:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.128+0000 7fb70d7fa700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb6fc0a11a0 0x7fb6fc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.129+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7000cc6d0 con 0x7fb710068950 2024-05-08T23:17:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.129+0000 7fb70f7fe700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb6fc0a11a0 0x7fb6fc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:16.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.129+0000 7fb70f7fe700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb6fc0a11a0 0x7fb6fc0a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8005950 tx=0x7fb6f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.133+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb700096300 con 0x7fb710068950 2024-05-08T23:17:16.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.341+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7101026d0 con 0x7fb710068950 2024-05-08T23:17:16.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.342+0000 7fb70d7fa700 1 -- 172.21.15.16:0/1323895897 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7101026d0 con 0x7fb710068950 2024-05-08T23:17:16.342 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:17:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb6fc0a11a0 msgr2=0x7fb6fc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb6fc0a11a0 0x7fb6fc0a3660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8005950 tx=0x7fb6f80058e0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 msgr2=0x7fb710197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 0x7fb710197420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d950 tx=0x7fb70000dd10 comp rx=0 tx=0).stop 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 shutdown_connections 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb710107020 0x7fb71019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710068950 0x7fb710197420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb6fc0a11a0 0x7fb6fc0a3660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.344+0000 7fb716571700 1 --2- 172.21.15.16:0/1323895897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7100692f0 0x7fb710197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.345+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 >> 172.21.15.16:0/1323895897 conn(0x7fb710075f80 msgr2=0x7fb7100fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.345+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 shutdown_connections 2024-05-08T23:17:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:16.345+0000 7fb716571700 1 -- 172.21.15.16:0/1323895897 wait complete. 2024-05-08T23:17:16.756 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-05-08T23:17:16.756 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.b on smithi155 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.e on smithi155 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.h on smithi155 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.k on smithi155 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-05-08T23:17:16.757 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-05-08T23:17:16.758 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-05-08T23:17:16.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:16 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1323895897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:16.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:16 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1323895897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:16.995 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:17:17.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:16 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1323895897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:17.895 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:17 smithi189 ceph-mon[46919]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:17.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:17 smithi155 ceph-mon[47105]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:18.081 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 -- 172.21.15.189:0/108033996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101a20 msgr2=0x7fac1c108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 --2- 172.21.15.189:0/108033996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101a20 0x7fac1c108340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fac1400b0a0 tx=0x7fac1400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 -- 172.21.15.189:0/108033996 shutdown_connections 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 --2- 172.21.15.189:0/108033996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101a20 0x7fac1c108340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 --2- 172.21.15.189:0/108033996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c0fff00 0x7fac1c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 --2- 172.21.15.189:0/108033996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c101060 0x7fac1c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 -- 172.21.15.189:0/108033996 >> 172.21.15.189:0/108033996 conn(0x7fac1c0fb4a0 msgr2=0x7fac1c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 -- 172.21.15.189:0/108033996 shutdown_connections 2024-05-08T23:17:18.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 -- 172.21.15.189:0/108033996 wait complete. 2024-05-08T23:17:18.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.082+0000 7fac243cd700 1 Processor -- start 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 -- start start 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0fff00 0x7fac1c069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 0x7fac1c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c101a20 0x7fac1c19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19e7b0 con 0x7fac1c101a20 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19e920 con 0x7fac1c0fff00 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c19eae0 con 0x7fac1c101060 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 0x7fac1c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 0x7fac1c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:57172/0 (socket says 172.21.15.189:57172) 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 -- 172.21.15.189:0/1144024117 learned_addr learned my addr 172.21.15.189:0/1144024117 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac22169700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0fff00 0x7fac1c069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac2296a700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c101a20 0x7fac1c19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:18.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 -- 172.21.15.189:0/1144024117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0fff00 msgr2=0x7fac1c069480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0fff00 0x7fac1c069480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 -- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c101a20 msgr2=0x7fac1c19e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c101a20 0x7fac1c19e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 -- 172.21.15.189:0/1144024117 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009d00 con 0x7fac1c101060 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac2296a700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c101a20 0x7fac1c19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac21968700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 0x7fac1c19ba20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00d950 tx=0x7fac0c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c009940 con 0x7fac1c101060 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac0c010460 con 0x7fac1c101060 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.083+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c19edc0 con 0x7fac1c101060 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.084+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00f620 con 0x7fac1c101060 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.084+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c19f240 con 0x7fac1c101060 2024-05-08T23:17:18.086 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.085+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c107320 con 0x7fac1c101060 2024-05-08T23:17:18.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.086+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fac0c0105d0 con 0x7fac1c101060 2024-05-08T23:17:18.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.086+0000 7fac137fe700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fac080a11a0 0x7fac080a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:18.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.086+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fac0c0cc210 con 0x7fac1c101060 2024-05-08T23:17:18.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.086+0000 7fac22169700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fac080a11a0 0x7fac080a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:18.087 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.087+0000 7fac22169700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fac080a11a0 0x7fac080a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac18005950 tx=0x7fac1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:18.089 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.090+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fac0c095e40 con 0x7fac1c101060 2024-05-08T23:17:18.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:17 smithi016 ceph-mon[42541]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:18.290 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.291+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fac1c02cfb0 con 0x7fac080a11a0 2024-05-08T23:17:18.299 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.300+0000 7fac137fe700 1 -- 172.21.15.189:0/1144024117 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fac1c02cfb0 con 0x7fac080a11a0 2024-05-08T23:17:18.299 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-05-08T23:17:18.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fac080a11a0 msgr2=0x7fac080a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:18.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fac080a11a0 0x7fac080a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac18005950 tx=0x7fac1800b410 comp rx=0 tx=0).stop 2024-05-08T23:17:18.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 msgr2=0x7fac1c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:18.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 0x7fac1c19ba20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00d950 tx=0x7fac0c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:18.301 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 shutdown_connections 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c101a20 0x7fac1c19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0fff00 0x7fac1c069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fac080a11a0 0x7fac080a3660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 --2- 172.21.15.189:0/1144024117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac1c101060 0x7fac1c19ba20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 >> 172.21.15.189:0/1144024117 conn(0x7fac1c0fb4a0 msgr2=0x7fac1c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 shutdown_connections 2024-05-08T23:17:18.302 INFO:teuthology.orchestra.run.smithi189.stderr:2024-05-08T23:17:18.302+0000 7fac243cd700 1 -- 172.21.15.189:0/1144024117 wait complete. 2024-05-08T23:17:18.715 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T23:17:18.718 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T23:17:18.754 INFO:journalctl@ceph.mds.a.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:17:18.755 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T23:17:18.787 INFO:journalctl@ceph.mds.d.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:17:18.788 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T23:17:18.817 INFO:journalctl@ceph.mds.g.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:17:18.818 DEBUG:teuthology.orchestra.run.smithi155:mds.b> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T23:17:18.821 DEBUG:teuthology.orchestra.run.smithi155:mds.e> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T23:17:18.848 INFO:journalctl@ceph.mds.j.smithi016.stdout:-- Logs begin at Wed 2024-05-08 22:53:57 UTC. -- 2024-05-08T23:17:18.860 INFO:journalctl@ceph.mds.b.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:17:18.861 DEBUG:teuthology.orchestra.run.smithi155:mds.h> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T23:17:18.891 INFO:journalctl@ceph.mds.e.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:17:18.892 DEBUG:teuthology.orchestra.run.smithi155:mds.k> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T23:17:18.920 INFO:journalctl@ceph.mds.h.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:17:18.921 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T23:17:18.923 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T23:17:18.948 INFO:journalctl@ceph.mds.k.smithi155.stdout:-- Logs begin at Wed 2024-05-08 22:54:24 UTC. -- 2024-05-08T23:17:18.953 INFO:journalctl@ceph.mds.c.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:17:18.954 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T23:17:18.982 INFO:journalctl@ceph.mds.f.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:17:18.983 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T23:17:19.011 INFO:journalctl@ceph.mds.i.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:17:19.012 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T23:17:19.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:19.039 INFO:journalctl@ceph.mds.l.smithi189.stdout:-- Logs begin at Wed 2024-05-08 22:54:26 UTC. -- 2024-05-08T23:17:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 -- 172.21.15.16:0/2310801090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe901018f0 msgr2=0x7fbe90103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 --2- 172.21.15.16:0/2310801090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe901018f0 0x7fbe90103ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009990 tx=0x7fbe78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 -- 172.21.15.16:0/2310801090 shutdown_connections 2024-05-08T23:17:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 --2- 172.21.15.16:0/2310801090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe90106b50 0x7fbe90108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 --2- 172.21.15.16:0/2310801090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe901018f0 0x7fbe90103ce0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 --2- 172.21.15.16:0/2310801090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe90104220 0x7fbe90106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 -- 172.21.15.16:0/2310801090 >> 172.21.15.16:0/2310801090 conn(0x7fbe900fb300 msgr2=0x7fbe900fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 -- 172.21.15.16:0/2310801090 shutdown_connections 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 -- 172.21.15.16:0/2310801090 wait complete. 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.162+0000 7fbe96711700 1 Processor -- start 2024-05-08T23:17:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 -- start start 2024-05-08T23:17:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 0x7fbe90198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90104220 0x7fbe90199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe90106b50 0x7fbe901a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901a0a60 con 0x7fbe901018f0 2024-05-08T23:17:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901a0ba0 con 0x7fbe90104220 2024-05-08T23:17:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe96711700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901a0d30 con 0x7fbe90106b50 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 0x7fbe90198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 0x7fbe90198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56024/0 (socket says 172.21.15.16:56024) 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 -- 172.21.15.16:0/3349235829 learned_addr learned my addr 172.21.15.16:0/3349235829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8f7fe700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90104220 0x7fbe90199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe94cae700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe90106b50 0x7fbe901a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 -- 172.21.15.16:0/3349235829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe90106b50 msgr2=0x7fbe901a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe90106b50 0x7fbe901a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 -- 172.21.15.16:0/3349235829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90104220 msgr2=0x7fbe90199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90104220 0x7fbe90199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8ffff700 1 -- 172.21.15.16:0/3349235829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780095f0 con 0x7fbe901018f0 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.163+0000 7fbe8f7fe700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90104220 0x7fbe90199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.164+0000 7fbe8ffff700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 0x7fbe90198f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe78006010 tx=0x7fbe78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.164+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7801d070 con 0x7fbe901018f0 2024-05-08T23:17:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.164+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe7800bdb0 con 0x7fbe901018f0 2024-05-08T23:17:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.164+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe901a0fb0 con 0x7fbe901018f0 2024-05-08T23:17:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.164+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800fda0 con 0x7fbe901018f0 2024-05-08T23:17:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.164+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe901a1370 con 0x7fbe901018f0 2024-05-08T23:17:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.165+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe90068060 con 0x7fbe901018f0 2024-05-08T23:17:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.166+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbe7800f460 con 0x7fbe901018f0 2024-05-08T23:17:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.167+0000 7fbe8d7fa700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbe7c0a11a0 0x7fbe7c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.167+0000 7fbe8f7fe700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbe7c0a11a0 0x7fbe7c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.167+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe780cd6d0 con 0x7fbe901018f0 2024-05-08T23:17:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.167+0000 7fbe8f7fe700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbe7c0a11a0 0x7fbe7c0a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbe80011490 tx=0x7fbe80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.171+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbe78097390 con 0x7fbe901018f0 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:19.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:19.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:19.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:19.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:19.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:19 smithi016 ceph-mon[42541]: Deploying daemon mds.l on smithi189 2024-05-08T23:17:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.396+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe9010c550 con 0x7fbe901018f0 2024-05-08T23:17:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.398+0000 7fbe8d7fa700 1 -- 172.21.15.16:0/3349235829 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18433 (secure 0 0 0) 0x7fbe7809a9b0 con 0x7fbe901018f0 2024-05-08T23:17:19.398 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:19.398 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:11.293661+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbe7c0a11a0 msgr2=0x7fbe7c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbe7c0a11a0 0x7fbe7c0a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbe80011490 tx=0x7fbe80009260 comp rx=0 tx=0).stop 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 msgr2=0x7fbe90198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 0x7fbe90198f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe78006010 tx=0x7fbe78004730 comp rx=0 tx=0).stop 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 shutdown_connections 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fbe7c0a11a0 0x7fbe7c0a3660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe90106b50 0x7fbe901a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe901018f0 0x7fbe90198f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 --2- 172.21.15.16:0/3349235829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe90104220 0x7fbe90199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 >> 172.21.15.16:0/3349235829 conn(0x7fbe900fb300 msgr2=0x7fbe900ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 shutdown_connections 2024-05-08T23:17:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.400+0000 7fbe96711700 1 -- 172.21.15.16:0/3349235829 wait complete. 2024-05-08T23:17:19.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:17:19.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:17:19.544 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:17:19.544 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-05-08T23:17:19.544 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:19.545 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:19 smithi189 ceph-mon[46919]: Deploying daemon mds.l on smithi189 2024-05-08T23:17:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 -- 172.21.15.16:0/2234041323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 msgr2=0x7f6d18101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/2234041323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18101360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009990 tx=0x7f6d14009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 -- 172.21.15.16:0/2234041323 shutdown_connections 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/2234041323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d18102ab0 0x7f6d181093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/2234041323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18101360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/2234041323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d181020f0 0x7f6d18102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 -- 172.21.15.16:0/2234041323 >> 172.21.15.16:0/2234041323 conn(0x7f6d180fc4e0 msgr2=0x7f6d180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 -- 172.21.15.16:0/2234041323 shutdown_connections 2024-05-08T23:17:19.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 -- 172.21.15.16:0/2234041323 wait complete. 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.556+0000 7f6d1fff6700 1 Processor -- start 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 -- start start 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d181020f0 0x7f6d181973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d18102ab0 0x7f6d1819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1819e960 con 0x7f6d18100f20 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1819eaa0 con 0x7f6d181020f0 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1fff6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1819ec30 con 0x7f6d18102ab0 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56038/0 (socket says 172.21.15.16:56038) 2024-05-08T23:17:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 -- 172.21.15.16:0/228449473 learned_addr learned my addr 172.21.15.16:0/228449473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1e593700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d18102ab0 0x7f6d1819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1d591700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d181020f0 0x7f6d181973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 -- 172.21.15.16:0/228449473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d18102ab0 msgr2=0x7f6d1819e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d18102ab0 0x7f6d1819e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 -- 172.21.15.16:0/228449473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d181020f0 msgr2=0x7f6d181973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d181020f0 0x7f6d181973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 -- 172.21.15.16:0/228449473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140095f0 con 0x7f6d18100f20 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1d591700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d181020f0 0x7f6d181973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.557+0000 7f6d1dd92700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18196e80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d14003780 tx=0x7f6d14009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.558+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1401d070 con 0x7f6d18100f20 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.558+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d14003f80 con 0x7f6d18100f20 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.558+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1819eeb0 con 0x7f6d18100f20 2024-05-08T23:17:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.558+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14022c70 con 0x7f6d18100f20 2024-05-08T23:17:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.558+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1819f220 con 0x7f6d18100f20 2024-05-08T23:17:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.559+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d18108480 con 0x7f6d18100f20 2024-05-08T23:17:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.560+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6d14003af0 con 0x7f6d18100f20 2024-05-08T23:17:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.560+0000 7f6d0effd700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d040a1120 0x7f6d040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.561+0000 7f6d1d591700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d040a1120 0x7f6d040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.561+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d140ce0d0 con 0x7f6d18100f20 2024-05-08T23:17:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.561+0000 7f6d1d591700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d040a1120 0x7f6d040a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005950 tx=0x7f6d080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.564+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d1409b820 con 0x7f6d18100f20 2024-05-08T23:17:19.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:17:19.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-05-08T23:17:19.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:19.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:19 smithi155 ceph-mon[47105]: Deploying daemon mds.l on smithi189 2024-05-08T23:17:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.781+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d1810c790 con 0x7f6d18100f20 2024-05-08T23:17:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.781+0000 7f6d0effd700 1 -- 172.21.15.16:0/228449473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d1409b3b0 con 0x7f6d18100f20 2024-05-08T23:17:19.781 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d040a1120 msgr2=0x7f6d040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d040a1120 0x7f6d040a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6d08005950 tx=0x7f6d080058e0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 msgr2=0x7f6d18196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18196e80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d14003780 tx=0x7f6d14009580 comp rx=0 tx=0).stop 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 shutdown_connections 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d040a1120 0x7f6d040a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d18102ab0 0x7f6d1819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d18100f20 0x7f6d18196e80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 --2- 172.21.15.16:0/228449473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d181020f0 0x7f6d181973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 >> 172.21.15.16:0/228449473 conn(0x7f6d180fc4e0 msgr2=0x7f6d181055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 shutdown_connections 2024-05-08T23:17:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.784+0000 7f6d1fff6700 1 -- 172.21.15.16:0/228449473 wait complete. 2024-05-08T23:17:19.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 -- 172.21.15.16:0/2434828796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348101900 msgr2=0x7f23481081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 --2- 172.21.15.16:0/2434828796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348101900 0x7f23481081b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f234000b0d0 tx=0x7f234000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 -- 172.21.15.16:0/2434828796 shutdown_connections 2024-05-08T23:17:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 --2- 172.21.15.16:0/2434828796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348101900 0x7f23481081b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 --2- 172.21.15.16:0/2434828796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23480ffd70 0x7f2348100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 --2- 172.21.15.16:0/2434828796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348100f60 0x7f23481013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 -- 172.21.15.16:0/2434828796 >> 172.21.15.16:0/2434828796 conn(0x7f23480fb300 msgr2=0x7f23480fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 -- 172.21.15.16:0/2434828796 shutdown_connections 2024-05-08T23:17:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 -- 172.21.15.16:0/2434828796 wait complete. 2024-05-08T23:17:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 Processor -- start 2024-05-08T23:17:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.935+0000 7f234e3b5700 1 -- start start 2024-05-08T23:17:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234e3b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 0x7f2348199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234e3b5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2348100f60 0x7f2348199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234e3b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348101900 0x7f23481a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234e3b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481a0b40 con 0x7f23480ffd70 2024-05-08T23:17:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234e3b5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481a0c80 con 0x7f2348101900 2024-05-08T23:17:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234e3b5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481a0e10 con 0x7f2348100f60 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 0x7f2348199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 0x7f2348199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56056/0 (socket says 172.21.15.16:56056) 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 -- 172.21.15.16:0/1908781751 learned_addr learned my addr 172.21.15.16:0/1908781751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f23477fe700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2348100f60 0x7f2348199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234c952700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348101900 0x7f23481a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 -- 172.21.15.16:0/1908781751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2348100f60 msgr2=0x7f2348199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2348100f60 0x7f2348199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 -- 172.21.15.16:0/1908781751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348101900 msgr2=0x7f23481a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348101900 0x7f23481a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f2347fff700 1 -- 172.21.15.16:0/1908781751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009d00 con 0x7f23480ffd70 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.936+0000 7f234c952700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348101900 0x7f23481a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.937+0000 7f2347fff700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 0x7f2348199010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f233800e970 tx=0x7f233800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:19.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.937+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338009790 con 0x7f23480ffd70 2024-05-08T23:17:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.937+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f233800eec0 con 0x7f23480ffd70 2024-05-08T23:17:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.937+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338003c10 con 0x7f23480ffd70 2024-05-08T23:17:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.937+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23481a10f0 con 0x7f23480ffd70 2024-05-08T23:17:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.937+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23481a15b0 con 0x7f23480ffd70 2024-05-08T23:17:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.938+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2348068060 con 0x7f23480ffd70 2024-05-08T23:17:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.939+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2338003d90 con 0x7f23480ffd70 2024-05-08T23:17:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.940+0000 7f23457fa700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23300a1120 0x7f23300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.940+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2338014070 con 0x7f23480ffd70 2024-05-08T23:17:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.940+0000 7f23477fe700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23300a1120 0x7f23300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.940+0000 7f23477fe700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23300a1120 0x7f23300a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f233c005950 tx=0x7f233c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:19.943+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2338096440 con 0x7f23480ffd70 2024-05-08T23:17:20.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.210+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f234810b680 con 0x7f23480ffd70 2024-05-08T23:17:20.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.210+0000 7f23457fa700 1 -- 172.21.15.16:0/1908781751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f2338099a60 con 0x7f23480ffd70 2024-05-08T23:17:20.210 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:20.210 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T23:17:20.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.212+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23300a1120 msgr2=0x7f23300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.212+0000 7f234e3b5700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23300a1120 0x7f23300a35e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f233c005950 tx=0x7f233c00b410 comp rx=0 tx=0).stop 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 msgr2=0x7f2348199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 0x7f2348199010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f233800e970 tx=0x7f233800ec80 comp rx=0 tx=0).stop 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 shutdown_connections 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2348101900 0x7f23481a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23480ffd70 0x7f2348199010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23300a1120 0x7f23300a35e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 --2- 172.21.15.16:0/1908781751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2348100f60 0x7f2348199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 >> 172.21.15.16:0/1908781751 conn(0x7f23480fb300 msgr2=0x7f2348069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 shutdown_connections 2024-05-08T23:17:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:20.213+0000 7f234e3b5700 1 -- 172.21.15.16:0/1908781751 wait complete. 2024-05-08T23:17:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-05-08T23:17:20.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:20 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3349235829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:20.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:20 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/228449473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:20.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:20 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1908781751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:20.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:20 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3349235829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:20.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:20 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/228449473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:20.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:20 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1908781751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:20.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:20 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3349235829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:20.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:20 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/228449473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:20.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:20 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1908781751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:20.919 INFO:journalctl@ceph.mds.l.smithi189.stdout:May 08 23:17:20 smithi189 bash[83134]: ca1e9c349a88e674ba806b5e471232dff94aaf4eb54e3a233aa3c0b5e97e6571 2024-05-08T23:17:20.919 INFO:journalctl@ceph.mds.l.smithi189.stdout:May 08 23:17:20 smithi189 systemd[1]: Started Ceph mds.l for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:21.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6832/3253311334,v1:172.21.15.189:6833/3253311334] up:boot 2024-05-08T23:17:21.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: fsmap 1 up:standby 2024-05-08T23:17:21.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:17:21.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.573 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:21.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:21.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:21.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:21.574 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:21 smithi155 ceph-mon[47105]: Deploying daemon mds.k on smithi155 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6832/3253311334,v1:172.21.15.189:6833/3253311334] up:boot 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: fsmap 1 up:standby 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:21.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:21 smithi016 ceph-mon[42541]: Deploying daemon mds.k on smithi155 2024-05-08T23:17:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6832/3253311334,v1:172.21.15.189:6833/3253311334] up:boot 2024-05-08T23:17:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: fsmap 1 up:standby 2024-05-08T23:17:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:17:21.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:21.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:21.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:21.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:21.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:21.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:21 smithi189 ceph-mon[46919]: Deploying daemon mds.k on smithi155 2024-05-08T23:17:21.965 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:17:21 smithi155 systemd[1]: Starting Ceph mds.k for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:22.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:22.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 -- 172.21.15.16:0/2873170840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 msgr2=0x7fe4500ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 --2- 172.21.15.16:0/2873170840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe4500ffb70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe440009990 tx=0x7fe440009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 -- 172.21.15.16:0/2873170840 shutdown_connections 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 --2- 172.21.15.16:0/2873170840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 0x7fe450100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 --2- 172.21.15.16:0/2873170840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe4500ffb70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 --2- 172.21.15.16:0/2873170840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 0x7fe450100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.379+0000 7fe45501a700 1 -- 172.21.15.16:0/2873170840 >> 172.21.15.16:0/2873170840 conn(0x7fe4500fb2f0 msgr2=0x7fe4500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.380+0000 7fe45501a700 1 -- 172.21.15.16:0/2873170840 shutdown_connections 2024-05-08T23:17:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.380+0000 7fe45501a700 1 -- 172.21.15.16:0/2873170840 wait complete. 2024-05-08T23:17:22.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.380+0000 7fe45501a700 1 Processor -- start 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 -- start start 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe450198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 0x7fe4501993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 0x7fe4501a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501a09e0 con 0x7fe4500ff750 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501a0b20 con 0x7fe4501000b0 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe45501a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501a0cb0 con 0x7fe450100a70 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe450198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe450198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56068/0 (socket says 172.21.15.16:56068) 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 -- 172.21.15.16:0/4226066824 learned_addr learned my addr 172.21.15.16:0/4226066824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44e59c700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 0x7fe4501993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44f59e700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 0x7fe4501a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 -- 172.21.15.16:0/4226066824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 msgr2=0x7fe4501a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 0x7fe4501a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 -- 172.21.15.16:0/4226066824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 msgr2=0x7fe4501993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 0x7fe4501993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44ed9d700 1 -- 172.21.15.16:0/4226066824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400095f0 con 0x7fe4500ff750 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44e59c700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 0x7fe4501993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.381+0000 7fe44f59e700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 0x7fe4501a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.382+0000 7fe44ed9d700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe450198eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe4400059d0 tx=0x7fe440005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.382+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44001d070 con 0x7fe4500ff750 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.382+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe44000be50 con 0x7fe4500ff750 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.382+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4501a0f30 con 0x7fe4500ff750 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.382+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440022e90 con 0x7fe4500ff750 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.382+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4501a12a0 con 0x7fe4500ff750 2024-05-08T23:17:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.383+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450067c80 con 0x7fe4500ff750 2024-05-08T23:17:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.384+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe440022690 con 0x7fe4500ff750 2024-05-08T23:17:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.385+0000 7fe437fff700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe4380a1120 0x7fe4380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.385+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4400cda10 con 0x7fe4500ff750 2024-05-08T23:17:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.385+0000 7fe44e59c700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe4380a1120 0x7fe4380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.386+0000 7fe44e59c700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe4380a1120 0x7fe4380a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe444009cb0 tx=0x7fe444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.389+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe4400976d0 con 0x7fe4500ff750 2024-05-08T23:17:22.465 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:17:22 smithi155 ceph-mds[82828]: -- 172.21.15.155:0/3548124167 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55db041b81a0 con 0x55db0418e800 2024-05-08T23:17:22.465 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:17:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-k[82805]: starting mds.k at 2024-05-08T23:17:22.465 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:17:22 smithi155 bash[82790]: d176d04fc811dda6789201f9c8a8e7df6f1b36352b274fc163c204be8b7f7bde 2024-05-08T23:17:22.465 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:17:22 smithi155 systemd[1]: Started Ceph mds.k for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.672+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe450067c10 con 0x7fe4500ff750 2024-05-08T23:17:22.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.672+0000 7fe437fff700 1 -- 172.21.15.16:0/4226066824 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7fe44009acf0 con 0x7fe4500ff750 2024-05-08T23:17:22.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:22.673 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe4380a1120 msgr2=0x7fe4380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe4380a1120 0x7fe4380a35e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe444009cb0 tx=0x7fe444009260 comp rx=0 tx=0).stop 2024-05-08T23:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 msgr2=0x7fe450198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe450198eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe4400059d0 tx=0x7fe440005e80 comp rx=0 tx=0).stop 2024-05-08T23:17:22.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 shutdown_connections 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe4380a1120 0x7fe4380a35e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe450100a70 0x7fe4501a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4500ff750 0x7fe450198eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 --2- 172.21.15.16:0/4226066824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4501000b0 0x7fe4501993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 >> 172.21.15.16:0/4226066824 conn(0x7fe4500fb2f0 msgr2=0x7fe4500fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.675+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 shutdown_connections 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:22.676+0000 7fe45501a700 1 -- 172.21.15.16:0/4226066824 wait complete. 2024-05-08T23:17:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-05-08T23:17:23.260 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.260 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: Deploying daemon mds.j on smithi016 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6832/2420649836,v1:172.21.15.155:6833/2420649836] up:boot 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: fsmap 2 up:standby 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:17:23.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4226066824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: Deploying daemon mds.j on smithi016 2024-05-08T23:17:23.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:23.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6832/2420649836,v1:172.21.15.155:6833/2420649836] up:boot 2024-05-08T23:17:23.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: fsmap 2 up:standby 2024-05-08T23:17:23.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:17:23.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:23 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4226066824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: Deploying daemon mds.j on smithi016 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:23.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6832/2420649836,v1:172.21.15.155:6833/2420649836] up:boot 2024-05-08T23:17:23.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: fsmap 2 up:standby 2024-05-08T23:17:23.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:17:23.716 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:23 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4226066824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:23.825 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:17:23 smithi016 systemd[1]: Starting Ceph mds.j for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:24.364 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:17:23 smithi016 bash[94635]: 879126779f471d752d27e321a752bf62cf9134f2c895fc177b7154b801c3abf0 2024-05-08T23:17:24.364 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:17:23 smithi016 systemd[1]: Started Ceph mds.j for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:24.364 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:17:23 smithi016 ceph-mds[94673]: -- 172.21.15.16:0/665081515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559224661a0 con 0x55592243d000 2024-05-08T23:17:24.364 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:17:23 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-j[94650]: starting mds.j at 2024-05-08T23:17:24.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:24.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 -- 172.21.15.16:0/3325083659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0100f50 msgr2=0x7f09c01013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:24.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/3325083659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0100f50 0x7f09c01013b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009990 tx=0x7f09a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:24.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 -- 172.21.15.16:0/3325083659 shutdown_connections 2024-05-08T23:17:24.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/3325083659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c01018f0 0x7f09c01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:24.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/3325083659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:24.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/3325083659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0100f50 0x7f09c01013b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 -- 172.21.15.16:0/3325083659 >> 172.21.15.16:0/3325083659 conn(0x7f09c00fb2f0 msgr2=0x7f09c00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 -- 172.21.15.16:0/3325083659 shutdown_connections 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.842+0000 7f09c6dbe700 1 -- 172.21.15.16:0/3325083659 wait complete. 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 Processor -- start 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 -- start start 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0100f50 0x7f09c0199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:24.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c01018f0 0x7f09c01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a09f0 con 0x7f09c00ffd60 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a0b30 con 0x7f09c0100f50 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c6dbe700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a0cc0 con 0x7f09c01018f0 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56112/0 (socket says 172.21.15.16:56112) 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 -- 172.21.15.16:0/2517874019 learned_addr learned my addr 172.21.15.16:0/2517874019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c535b700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c01018f0 0x7f09c01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09bffff700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0100f50 0x7f09c0199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:24.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 -- 172.21.15.16:0/2517874019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c01018f0 msgr2=0x7f09c01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c01018f0 0x7f09c01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 -- 172.21.15.16:0/2517874019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0100f50 msgr2=0x7f09c0199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0100f50 0x7f09c0199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 -- 172.21.15.16:0/2517874019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0009520 con 0x7f09c00ffd60 2024-05-08T23:17:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09bffff700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0100f50 0x7f09c0199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.843+0000 7f09c4b5a700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0198ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ea50 tx=0x7f09b000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.844+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000cc90 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.844+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a80095f0 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.844+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09b000cdf0 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.844+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0018680 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.844+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01a1240 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.845+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c0067c10 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.846+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f09b0010430 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.847+0000 7f09bdffb700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f09ac0a1120 0x7f09ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.847+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09b0014070 con 0x7f09c00ffd60 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.847+0000 7f09bffff700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f09ac0a1120 0x7f09ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:24.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.847+0000 7f09bffff700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f09ac0a1120 0x7f09ac0a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09a800b2e0 tx=0x7f09a8005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:24.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:24.852+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f09b0096670 con 0x7f09c00ffd60 2024-05-08T23:17:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.130+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c010b450 con 0x7f09c00ffd60 2024-05-08T23:17:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.131+0000 7f09bdffb700 1 -- 172.21.15.16:0/2517874019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f09c010b450 con 0x7f09c00ffd60 2024-05-08T23:17:25.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:25.131 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-05-08T23:17:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f09ac0a1120 msgr2=0x7f09ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f09ac0a1120 0x7f09ac0a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09a800b2e0 tx=0x7f09a8005c80 comp rx=0 tx=0).stop 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 msgr2=0x7f09c0198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0198ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ea50 tx=0x7f09b000ee10 comp rx=0 tx=0).stop 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 shutdown_connections 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f09ac0a1120 0x7f09ac0a35e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c01018f0 0x7f09c01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09c00ffd60 0x7f09c0198ec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 --2- 172.21.15.16:0/2517874019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09c0100f50 0x7f09c0199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 >> 172.21.15.16:0/2517874019 conn(0x7f09c00fb2f0 msgr2=0x7f09c0068020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.133+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 shutdown_connections 2024-05-08T23:17:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:25.134+0000 7f09c6dbe700 1 -- 172.21.15.16:0/2517874019 wait complete. 2024-05-08T23:17:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-05-08T23:17:25.232 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: Deploying daemon mds.d on smithi016 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.16:6834/4018978287,v1:172.21.15.16:6835/4018978287] up:boot 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: fsmap 3 up:standby 2024-05-08T23:17:25.233 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:24 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: Deploying daemon mds.d on smithi016 2024-05-08T23:17:25.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:25.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.16:6834/4018978287,v1:172.21.15.16:6835/4018978287] up:boot 2024-05-08T23:17:25.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: fsmap 3 up:standby 2024-05-08T23:17:25.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:24 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: Deploying daemon mds.d on smithi016 2024-05-08T23:17:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:25.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.16:6834/4018978287,v1:172.21.15.16:6835/4018978287] up:boot 2024-05-08T23:17:25.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: fsmap 3 up:standby 2024-05-08T23:17:25.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:24 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:17:25.614 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:17:25 smithi016 systemd[1]: Starting Ceph mds.d for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:17:25 smithi016 bash[95021]: 99b1274150a58e58685802708d734323fc612b6e6b8650c313298e93b5706a2a 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:17:25 smithi016 ceph-mds[95059]: -- 172.21.15.16:0/617908292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559d1980c1a0 con 0x559d197e2800 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:17:25 smithi016 systemd[1]: Started Ceph mds.d for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:17:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-d[95036]: starting mds.d at 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2517874019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:25 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:26.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:26.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:26 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:26.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:26 smithi016 ceph-mon[42541]: Deploying daemon mds.e on smithi155 2024-05-08T23:17:26.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2517874019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:26.273 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:26 smithi155 ceph-mon[47105]: Deploying daemon mds.e on smithi155 2024-05-08T23:17:26.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2517874019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:26 smithi189 ceph-mon[46919]: Deploying daemon mds.e on smithi155 2024-05-08T23:17:27.118 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:17:27 smithi155 systemd[1]: Starting Ceph mds.e for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:27.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 -- 172.21.15.16:0/554566851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc100f80 msgr2=0x7f79dc1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 --2- 172.21.15.16:0/554566851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc100f80 0x7f79dc1013e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0099c0 tx=0x7f79cc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 -- 172.21.15.16:0/554566851 shutdown_connections 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 --2- 172.21.15.16:0/554566851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79dc101920 0x7f79dc1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 --2- 172.21.15.16:0/554566851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79dc0ffd90 0x7f79dc1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 --2- 172.21.15.16:0/554566851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc100f80 0x7f79dc1013e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.303+0000 7f79e2371700 1 -- 172.21.15.16:0/554566851 >> 172.21.15.16:0/554566851 conn(0x7f79dc0fb300 msgr2=0x7f79dc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.304+0000 7f79e2371700 1 -- 172.21.15.16:0/554566851 shutdown_connections 2024-05-08T23:17:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.304+0000 7f79e2371700 1 -- 172.21.15.16:0/554566851 wait complete. 2024-05-08T23:17:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.304+0000 7f79e2371700 1 Processor -- start 2024-05-08T23:17:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.304+0000 7f79e2371700 1 -- start start 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e2371700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79dc0ffd90 0x7f79dc195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e2371700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79dc100f80 0x7f79dc196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e2371700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 0x7f79dc19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e2371700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19d700 con 0x7f79dc101920 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e2371700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19d840 con 0x7f79dc100f80 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e2371700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc19d9d0 con 0x7f79dc0ffd90 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 0x7f79dc19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 0x7f79dc19d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49804/0 (socket says 172.21.15.16:49804) 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 -- 172.21.15.16:0/3328740272 learned_addr learned my addr 172.21.15.16:0/3328740272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79db7fe700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79dc100f80 0x7f79dc196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79dbfff700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79dc0ffd90 0x7f79dc195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 -- 172.21.15.16:0/3328740272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79dc0ffd90 msgr2=0x7f79dc195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79dc0ffd90 0x7f79dc195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 -- 172.21.15.16:0/3328740272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79dc100f80 msgr2=0x7f79dc196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79dc100f80 0x7f79dc196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.305+0000 7f79e090e700 1 -- 172.21.15.16:0/3328740272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0095f0 con 0x7f79dc101920 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.306+0000 7f79e090e700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 0x7f79dc19d130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f79d4007f10 tx=0x7f79d400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.306+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400fd30 con 0x7f79dc101920 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.306+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79d400fe90 con 0x7f79dc101920 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.306+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc069230 con 0x7f79dc101920 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.306+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4012e60 con 0x7f79dc101920 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.306+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc069740 con 0x7f79dc101920 2024-05-08T23:17:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.307+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc067380 con 0x7f79dc101920 2024-05-08T23:17:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.308+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f79d4012460 con 0x7f79dc101920 2024-05-08T23:17:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.309+0000 7f79d97fa700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f79c80a11a0 0x7f79c80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.309+0000 7f79dbfff700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f79c80a11a0 0x7f79c80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.309+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f79d40ce9a0 con 0x7f79dc101920 2024-05-08T23:17:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.309+0000 7f79dbfff700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f79c80a11a0 0x7f79c80a3660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79dc100de0 tx=0x7f79c4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.312+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f79d40985d0 con 0x7f79dc101920 2024-05-08T23:17:27.562 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:27 smithi016 ceph-mon[42541]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:27.562 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:27 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.16:6836/3189374355,v1:172.21.15.16:6837/3189374355] up:boot 2024-05-08T23:17:27.562 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:27 smithi016 ceph-mon[42541]: fsmap 4 up:standby 2024-05-08T23:17:27.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.584+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79dc10b450 con 0x7f79dc101920 2024-05-08T23:17:27.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.585+0000 7f79d97fa700 1 -- 172.21.15.16:0/3328740272 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f79d4016de0 con 0x7f79dc101920 2024-05-08T23:17:27.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:27.585 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2024-05-08T23:17:27.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f79c80a11a0 msgr2=0x7f79c80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f79c80a11a0 0x7f79c80a3660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f79dc100de0 tx=0x7f79c4007400 comp rx=0 tx=0).stop 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 msgr2=0x7f79dc19d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 0x7f79dc19d130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f79d4007f10 tx=0x7f79d400f5c0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 shutdown_connections 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79dc101920 0x7f79dc19d130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f79c80a11a0 0x7f79c80a3660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79dc0ffd90 0x7f79dc195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 --2- 172.21.15.16:0/3328740272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79dc100f80 0x7f79dc196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 >> 172.21.15.16:0/3328740272 conn(0x7f79dc0fb300 msgr2=0x7f79dc1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 shutdown_connections 2024-05-08T23:17:27.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:27.588+0000 7f79e2371700 1 -- 172.21.15.16:0/3328740272 wait complete. 2024-05-08T23:17:27.589 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-05-08T23:17:27.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:27 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:17:27.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:27 smithi189 ceph-mon[46919]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:27 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.16:6836/3189374355,v1:172.21.15.16:6837/3189374355] up:boot 2024-05-08T23:17:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:27 smithi189 ceph-mon[46919]: fsmap 4 up:standby 2024-05-08T23:17:27.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:27 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:27 smithi155 ceph-mon[47105]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:27 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.16:6836/3189374355,v1:172.21.15.16:6837/3189374355] up:boot 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:27 smithi155 ceph-mon[47105]: fsmap 4 up:standby 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:27 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:17:27 smithi155 ceph-mds[83176]: -- 172.21.15.155:0/557703113 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ad1c3501a0 con 0x55ad1c326800 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:17:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e[83153]: starting mds.e at 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:17:27 smithi155 bash[83138]: 1071281a85498cc9c3fdd927bc6f27ac7305cc706181f27ff2ddc212bd6779fe 2024-05-08T23:17:27.965 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:17:27 smithi155 systemd[1]: Started Ceph mds.e for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3328740272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:28.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:28.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: Deploying daemon mds.f on smithi189 2024-05-08T23:17:28.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6834/1260766455,v1:172.21.15.155:6835/1260766455] up:boot 2024-05-08T23:17:28.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: fsmap 5 up:standby 2024-05-08T23:17:28.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:28 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3328740272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: Deploying daemon mds.f on smithi189 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6834/1260766455,v1:172.21.15.155:6835/1260766455] up:boot 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: fsmap 5 up:standby 2024-05-08T23:17:28.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:28 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3328740272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: Deploying daemon mds.f on smithi189 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6834/1260766455,v1:172.21.15.155:6835/1260766455] up:boot 2024-05-08T23:17:28.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: fsmap 5 up:standby 2024-05-08T23:17:28.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:28 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:17:29.135 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:17:28 smithi189 systemd[1]: Starting Ceph mds.f for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:29.418 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mds[83515]: -- 172.21.15.189:0/3047343440 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55df80c081a0 con 0x55df80bdf000 2024-05-08T23:17:29.418 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:17:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f[83492]: starting mds.f at 2024-05-08T23:17:29.419 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:17:29 smithi189 bash[83477]: 100f3e4f55e2e6c10977ca4b7da9ed52e79a162cdb944ed5d1e9ccef852e25a9 2024-05-08T23:17:29.419 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:17:29 smithi189 systemd[1]: Started Ceph mds.f for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:29.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:29.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 -- 172.21.15.16:0/2482148353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc101960 msgr2=0x7f18dc103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:29.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 --2- 172.21.15.16:0/2482148353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc101960 0x7f18dc103d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009990 tx=0x7f18d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 -- 172.21.15.16:0/2482148353 shutdown_connections 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 --2- 172.21.15.16:0/2482148353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18dc106bc0 0x7f18dc108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 --2- 172.21.15.16:0/2482148353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc101960 0x7f18dc103d50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 --2- 172.21.15.16:0/2482148353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc104290 0x7f18dc106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 -- 172.21.15.16:0/2482148353 >> 172.21.15.16:0/2482148353 conn(0x7f18dc0fb2f0 msgr2=0x7f18dc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 -- 172.21.15.16:0/2482148353 shutdown_connections 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.749+0000 7f18e4767700 1 -- 172.21.15.16:0/2482148353 wait complete. 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 Processor -- start 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 -- start start 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18dc101960 0x7f18dc101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 0x7f18dc0ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:29.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc106bc0 0x7f18dc0ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1005f0 con 0x7f18dc104290 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc100760 con 0x7f18dc101960 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e4767700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1a0d10 con 0x7f18dc106bc0 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e1d02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 0x7f18dc0ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e1d02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 0x7f18dc0ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49818/0 (socket says 172.21.15.16:49818) 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e1d02700 1 -- 172.21.15.16:0/1487728421 learned_addr learned my addr 172.21.15.16:0/1487728421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e2d04700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc106bc0 0x7f18dc0ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.750+0000 7f18e2503700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18dc101960 0x7f18dc101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e1d02700 1 -- 172.21.15.16:0/1487728421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc106bc0 msgr2=0x7f18dc0ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e1d02700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc106bc0 0x7f18dc0ffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e1d02700 1 -- 172.21.15.16:0/1487728421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18dc101960 msgr2=0x7f18dc101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e1d02700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18dc101960 0x7f18dc101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e1d02700 1 -- 172.21.15.16:0/1487728421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80095f0 con 0x7f18dc104290 2024-05-08T23:17:29.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e1d02700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 0x7f18dc0ff910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d950 tx=0x7f18cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc00f890 con 0x7f18dc104290 2024-05-08T23:17:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18cc00fe80 con 0x7f18dc104290 2024-05-08T23:17:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc1a0fa0 con 0x7f18dc104290 2024-05-08T23:17:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc019360 con 0x7f18dc104290 2024-05-08T23:17:29.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.751+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc1a1420 con 0x7f18dc104290 2024-05-08T23:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.752+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc0fce70 con 0x7f18dc104290 2024-05-08T23:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.754+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f18cc010460 con 0x7f18dc104290 2024-05-08T23:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.754+0000 7f18d37fe700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18c80a1120 0x7f18c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.754+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18cc0cc190 con 0x7f18dc104290 2024-05-08T23:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.754+0000 7f18e2503700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18c80a1120 0x7f18c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:29.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.755+0000 7f18e2503700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18c80a1120 0x7f18c80a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005910 tx=0x7f18d800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:29.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:29.758+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f18cc095dc0 con 0x7f18dc104290 2024-05-08T23:17:29.820 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:29.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:29.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:29.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:29 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:29.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:29.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:29.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:29 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:29 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.028+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18dc067c10 con 0x7f18dc104290 2024-05-08T23:17:30.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.029+0000 7f18d37fe700 1 -- 172.21.15.16:0/1487728421 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5054 (secure 0 0 0) 0x7f18cc0993e0 con 0x7f18dc104290 2024-05-08T23:17:30.030 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:30.030 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18c80a1120 msgr2=0x7f18c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18c80a1120 0x7f18c80a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005910 tx=0x7f18d800b610 comp rx=0 tx=0).stop 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 msgr2=0x7f18dc0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 0x7f18dc0ff910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00d950 tx=0x7f18cc00dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 shutdown_connections 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18c80a1120 0x7f18c80a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc106bc0 0x7f18dc0ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18dc101960 0x7f18dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 --2- 172.21.15.16:0/1487728421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18dc104290 0x7f18dc0ff910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 >> 172.21.15.16:0/1487728421 conn(0x7f18dc0fb2f0 msgr2=0x7f18dc103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:30.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 shutdown_connections 2024-05-08T23:17:30.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:30.031+0000 7f18e4767700 1 -- 172.21.15.16:0/1487728421 wait complete. 2024-05-08T23:17:30.033 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-05-08T23:17:30.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:30 smithi189 ceph-mon[46919]: Deploying daemon mds.c on smithi189 2024-05-08T23:17:30.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:30 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6834/3239398256,v1:172.21.15.189:6835/3239398256] up:boot 2024-05-08T23:17:30.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:30 smithi189 ceph-mon[46919]: fsmap 6 up:standby 2024-05-08T23:17:30.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:30 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:17:30.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:30 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1487728421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:30.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:30 smithi016 ceph-mon[42541]: Deploying daemon mds.c on smithi189 2024-05-08T23:17:30.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:30 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6834/3239398256,v1:172.21.15.189:6835/3239398256] up:boot 2024-05-08T23:17:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:30 smithi016 ceph-mon[42541]: fsmap 6 up:standby 2024-05-08T23:17:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:30 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:17:30.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:30 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1487728421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:30 smithi155 ceph-mon[47105]: Deploying daemon mds.c on smithi189 2024-05-08T23:17:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:30 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6834/3239398256,v1:172.21.15.189:6835/3239398256] up:boot 2024-05-08T23:17:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:30 smithi155 ceph-mon[47105]: fsmap 6 up:standby 2024-05-08T23:17:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:30 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:17:30.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:30 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1487728421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:30.973 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:17:30 smithi189 systemd[1]: Starting Ceph mds.c for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:31.419 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mds[83856]: -- 172.21.15.189:0/3596669319 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b41f8341a0 con 0x55b41f80ac00 2024-05-08T23:17:31.419 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c[83833]: starting mds.c at 2024-05-08T23:17:31.419 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:17:31 smithi189 bash[83819]: 79d40cc3c22b93ddb1f8cb09c2875bff8044184b6e2f4f16e2be7be27d482561 2024-05-08T23:17:31.419 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:17:31 smithi189 systemd[1]: Started Ceph mds.c for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:31 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:31.864 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:31.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:31.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:31.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:31 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:31.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:31 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:32.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 -- 172.21.15.16:0/66971906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 msgr2=0x7fcee8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 --2- 172.21.15.16:0/66971906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8103d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009990 tx=0x7fcee4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 -- 172.21.15.16:0/66971906 shutdown_connections 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 --2- 172.21.15.16:0/66971906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee8106ba0 0x7fcee8108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 --2- 172.21.15.16:0/66971906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8103d30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 --2- 172.21.15.16:0/66971906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee8104270 0x7fcee8106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 -- 172.21.15.16:0/66971906 >> 172.21.15.16:0/66971906 conn(0x7fcee80fb2f0 msgr2=0x7fcee80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 -- 172.21.15.16:0/66971906 shutdown_connections 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.193+0000 7fceef2ac700 1 -- 172.21.15.16:0/66971906 wait complete. 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 Processor -- start 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 -- start start 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee8104270 0x7fcee81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee8106ba0 0x7fcee81a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81a0a90 con 0x7fcee8101940 2024-05-08T23:17:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81a0bd0 con 0x7fcee8106ba0 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.194+0000 7fceef2ac700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee81a0d60 con 0x7fcee8104270 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49838/0 (socket says 172.21.15.16:49838) 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 -- 172.21.15.16:0/2265793378 learned_addr learned my addr 172.21.15.16:0/2265793378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceec847700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee8104270 0x7fcee81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed849700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee8106ba0 0x7fcee81a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 -- 172.21.15.16:0/2265793378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee8104270 msgr2=0x7fcee81994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee8104270 0x7fcee81994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 -- 172.21.15.16:0/2265793378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee8106ba0 msgr2=0x7fcee81a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee8106ba0 0x7fcee81a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 -- 172.21.15.16:0/2265793378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40095f0 con 0x7fcee8101940 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceed048700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8198f60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b2e0 tx=0x7fcee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee401d070 con 0x7fcee8101940 2024-05-08T23:17:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee81a0fe0 con 0x7fcee8101940 2024-05-08T23:17:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcee400bdb0 con 0x7fcee8101940 2024-05-08T23:17:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400fdc0 con 0x7fcee8101940 2024-05-08T23:17:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.195+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee81a1350 con 0x7fcee8101940 2024-05-08T23:17:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.196+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee8067c80 con 0x7fcee8101940 2024-05-08T23:17:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.197+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcee400f560 con 0x7fcee8101940 2024-05-08T23:17:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.198+0000 7fcede7fc700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fced40a1120 0x7fced40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.198+0000 7fceec847700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fced40a1120 0x7fced40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.198+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcee40cda40 con 0x7fcee8101940 2024-05-08T23:17:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.199+0000 7fceec847700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fced40a1120 0x7fced40a35e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fced8005950 tx=0x7fced800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.202+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcee4097780 con 0x7fcee8101940 2024-05-08T23:17:32.379 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:17:32 smithi155 systemd[1]: Starting Ceph mds.h for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.469+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcee8067c10 con 0x7fcee8101940 2024-05-08T23:17:32.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.470+0000 7fcede7fc700 1 -- 172.21.15.16:0/2265793378 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5814 (secure 0 0 0) 0x7fcee409ada0 con 0x7fcee8101940 2024-05-08T23:17:32.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T23:17:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fced40a1120 msgr2=0x7fced40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fced40a1120 0x7fced40a35e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fced8005950 tx=0x7fced800b410 comp rx=0 tx=0).stop 2024-05-08T23:17:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 msgr2=0x7fcee8198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8198f60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b2e0 tx=0x7fcee4004730 comp rx=0 tx=0).stop 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 shutdown_connections 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcee8106ba0 0x7fcee81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee8101940 0x7fcee8198f60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fced40a1120 0x7fced40a35e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 --2- 172.21.15.16:0/2265793378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee8104270 0x7fcee81994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.472+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 >> 172.21.15.16:0/2265793378 conn(0x7fcee80fb2f0 msgr2=0x7fcee80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.473+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 shutdown_connections 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:32.473+0000 7fceef2ac700 1 -- 172.21.15.16:0/2265793378 wait complete. 2024-05-08T23:17:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-05-08T23:17:32.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:32 smithi016 ceph-mon[42541]: Deploying daemon mds.h on smithi155 2024-05-08T23:17:32.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:32 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] up:boot 2024-05-08T23:17:32.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:32 smithi016 ceph-mon[42541]: fsmap 7 up:standby 2024-05-08T23:17:32.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:32 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:17:32.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:32 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2265793378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:32.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:32 smithi189 ceph-mon[46919]: Deploying daemon mds.h on smithi155 2024-05-08T23:17:32.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:32 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] up:boot 2024-05-08T23:17:32.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:32 smithi189 ceph-mon[46919]: fsmap 7 up:standby 2024-05-08T23:17:32.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:32 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:17:32.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:32 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2265793378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:32.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:32 smithi155 ceph-mon[47105]: Deploying daemon mds.h on smithi155 2024-05-08T23:17:32.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:32 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] up:boot 2024-05-08T23:17:32.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:32 smithi155 ceph-mon[47105]: fsmap 7 up:standby 2024-05-08T23:17:32.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:32 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:17:32.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:32 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2265793378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:32.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:17:32 smithi155 ceph-mds[83518]: -- 172.21.15.155:0/3411493860 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561a377921a0 con 0x561a37768c00 2024-05-08T23:17:32.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:17:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h[83495]: starting mds.h at 2024-05-08T23:17:32.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:17:32 smithi155 bash[83479]: 720a5d77ec36ce55c3670335f4542e3fefea7d8e5771d75c2cf677c56b5705ab 2024-05-08T23:17:32.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:17:32 smithi155 systemd[1]: Started Ceph mds.h for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:34.071 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:34.072 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: Deploying daemon mds.i on smithi189 2024-05-08T23:17:34.072 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6836/1960878524,v1:172.21.15.155:6837/1960878524] up:boot 2024-05-08T23:17:34.072 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: fsmap 8 up:standby 2024-05-08T23:17:34.072 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:33 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: Deploying daemon mds.i on smithi189 2024-05-08T23:17:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6836/1960878524,v1:172.21.15.155:6837/1960878524] up:boot 2024-05-08T23:17:34.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: fsmap 8 up:standby 2024-05-08T23:17:34.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:33 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: Deploying daemon mds.i on smithi189 2024-05-08T23:17:34.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6836/1960878524,v1:172.21.15.155:6837/1960878524] up:boot 2024-05-08T23:17:34.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: fsmap 8 up:standby 2024-05-08T23:17:34.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:33 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:17:34.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 -- 172.21.15.16:0/3620332007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4180ff6e0 msgr2=0x7fd4180ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 --2- 172.21.15.16:0/3620332007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4180ff6e0 0x7fd4180ffb00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd414009990 tx=0x7fd414009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 -- 172.21.15.16:0/3620332007 shutdown_connections 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 --2- 172.21.15.16:0/3620332007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd418100a00 0x7fd418100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 --2- 172.21.15.16:0/3620332007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4180ff6e0 0x7fd4180ffb00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 --2- 172.21.15.16:0/3620332007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100040 0x7fd4181004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 -- 172.21.15.16:0/3620332007 >> 172.21.15.16:0/3620332007 conn(0x7fd4180fb2e0 msgr2=0x7fd4180fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 -- 172.21.15.16:0/3620332007 shutdown_connections 2024-05-08T23:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.633+0000 7fd420237700 1 -- 172.21.15.16:0/3620332007 wait complete. 2024-05-08T23:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 Processor -- start 2024-05-08T23:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 -- start start 2024-05-08T23:17:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4180ff6e0 0x7fd418198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 0x7fd418199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100a00 0x7fd4181a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181a0a40 con 0x7fd418100040 2024-05-08T23:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181a0b80 con 0x7fd4180ff6e0 2024-05-08T23:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd420237700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181a0d10 con 0x7fd418100a00 2024-05-08T23:17:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd41d7d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 0x7fd418199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd41d7d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 0x7fd418199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49858/0 (socket says 172.21.15.16:49858) 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd41d7d2700 1 -- 172.21.15.16:0/2723323787 learned_addr learned my addr 172.21.15.16:0/2723323787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd41dfd3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4180ff6e0 0x7fd418198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.634+0000 7fd41e7d4700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100a00 0x7fd4181a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41d7d2700 1 -- 172.21.15.16:0/2723323787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100a00 msgr2=0x7fd4181a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41d7d2700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100a00 0x7fd4181a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41d7d2700 1 -- 172.21.15.16:0/2723323787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4180ff6e0 msgr2=0x7fd418198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41d7d2700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4180ff6e0 0x7fd418198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41d7d2700 1 -- 172.21.15.16:0/2723323787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140095f0 con 0x7fd418100040 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41e7d4700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100a00 0x7fd4181a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd41d7d2700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 0x7fd418199450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e9a0 tx=0x7fd40c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c009790 con 0x7fd418100040 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd40c00cbd0 con 0x7fd418100040 2024-05-08T23:17:34.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4181a0ff0 con 0x7fd418100040 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c018680 con 0x7fd418100040 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.635+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4181a1470 con 0x7fd418100040 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.636+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41810a4c0 con 0x7fd418100040 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.638+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd40c00cd40 con 0x7fd418100040 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.639+0000 7fd40affd700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4040a1120 0x7fd4040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.639+0000 7fd41dfd3700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4040a1120 0x7fd4040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.639+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd40c014070 con 0x7fd418100040 2024-05-08T23:17:34.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.639+0000 7fd41dfd3700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4040a1120 0x7fd4040a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd414005910 tx=0x7fd41400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:34.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.642+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd40c096670 con 0x7fd418100040 2024-05-08T23:17:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.913+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd418067380 con 0x7fd418100040 2024-05-08T23:17:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.913+0000 7fd40affd700 1 -- 172.21.15.16:0/2723323787 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7336 (secure 0 0 0) 0x7fd40c099c90 con 0x7fd418100040 2024-05-08T23:17:34.914 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:34.914 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4040a1120 msgr2=0x7fd4040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4040a1120 0x7fd4040a35e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd414005910 tx=0x7fd41400b610 comp rx=0 tx=0).stop 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 msgr2=0x7fd418199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 0x7fd418199450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e9a0 tx=0x7fd40c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 shutdown_connections 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4040a1120 0x7fd4040a35e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100a00 0x7fd4181a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4180ff6e0 0x7fd418198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 --2- 172.21.15.16:0/2723323787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd418100040 0x7fd418199450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 >> 172.21.15.16:0/2723323787 conn(0x7fd4180fb2e0 msgr2=0x7fd4180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 shutdown_connections 2024-05-08T23:17:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:34.916+0000 7fd420237700 1 -- 172.21.15.16:0/2723323787 wait complete. 2024-05-08T23:17:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-05-08T23:17:35.168 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:17:34 smithi189 bash[84161]: 9ebee2d1419633bfaf989535c41e5ecfc56965f8590b7c4ed52300cbd0a03af9 2024-05-08T23:17:35.169 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:17:34 smithi189 systemd[1]: Started Ceph mds.i for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:35.801 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: Deploying daemon mds.g on smithi016 2024-05-08T23:17:35.802 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] up:boot 2024-05-08T23:17:35.802 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: fsmap 9 up:standby 2024-05-08T23:17:35.802 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:17:35.802 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:35 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2723323787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:36.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: Deploying daemon mds.g on smithi016 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] up:boot 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: fsmap 9 up:standby 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:17:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:35 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2723323787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: Deploying daemon mds.g on smithi016 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] up:boot 2024-05-08T23:17:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: fsmap 9 up:standby 2024-05-08T23:17:36.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:17:36.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:35 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2723323787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:36.355 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:17:36 smithi016 systemd[1]: Starting Ceph mds.g for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:36.863 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:17:36 smithi016 ceph-mds[95583]: -- 172.21.15.16:0/3589591856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556b02c601a0 con 0x556b02c36c00 2024-05-08T23:17:36.864 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:17:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-g[95560]: starting mds.g at 2024-05-08T23:17:36.864 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:17:36 smithi016 bash[95545]: cd707b9ad13a343dc0981ef8871685747704f3430734c9fbe3ef90ab95163c2d 2024-05-08T23:17:36.864 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:17:36 smithi016 systemd[1]: Started Ceph mds.g for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:36.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.077+0000 7f6210984700 1 -- 172.21.15.16:0/2715357058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c0ffd60 msgr2=0x7f620c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.077+0000 7f6210984700 1 --2- 172.21.15.16:0/2715357058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c0ffd60 0x7f620c100180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0099c0 tx=0x7f61fc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 -- 172.21.15.16:0/2715357058 shutdown_connections 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 --2- 172.21.15.16:0/2715357058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620c1018f0 0x7f620c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 --2- 172.21.15.16:0/2715357058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c0ffd60 0x7f620c100180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 --2- 172.21.15.16:0/2715357058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 0x7f620c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 -- 172.21.15.16:0/2715357058 >> 172.21.15.16:0/2715357058 conn(0x7f620c0fb2f0 msgr2=0x7f620c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 -- 172.21.15.16:0/2715357058 shutdown_connections 2024-05-08T23:17:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.078+0000 7f6210984700 1 -- 172.21.15.16:0/2715357058 wait complete. 2024-05-08T23:17:37.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 Processor -- start 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 -- start start 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620c0ffd60 0x7f620c068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 0x7f620c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 0x7f620c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1a0a10 con 0x7f620c1018f0 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1a0b50 con 0x7f620c100f50 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6210984700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1a0d10 con 0x7f620c0ffd60 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 0x7f620c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 0x7f620c1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53892/0 (socket says 172.21.15.16:53892) 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 -- 172.21.15.16:0/1675023588 learned_addr learned my addr 172.21.15.16:0/1675023588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f6209d9b700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 0x7f620c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620a59c700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620c0ffd60 0x7f620c068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 -- 172.21.15.16:0/1675023588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620c0ffd60 msgr2=0x7f620c068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620c0ffd60 0x7f620c068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 -- 172.21.15.16:0/1675023588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 msgr2=0x7f620c19deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 0x7f620c19deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.079+0000 7f620ad9d700 1 -- 172.21.15.16:0/1675023588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0095f0 con 0x7f620c1018f0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f6209d9b700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 0x7f620c19deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f620ad9d700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 0x7f620c1a0400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f620400ef30 tx=0x7f620400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6204018070 con 0x7f620c1018f0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f620400f040 con 0x7f620c1018f0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c1a0f90 con 0x7f620c1018f0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620400cc70 con 0x7f620c1018f0 2024-05-08T23:17:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.080+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c1a13c0 con 0x7f620c1018f0 2024-05-08T23:17:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.081+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c107270 con 0x7f620c1018f0 2024-05-08T23:17:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.082+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6204003a00 con 0x7f620c1018f0 2024-05-08T23:17:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.083+0000 7f61fb7fe700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f61f40a1120 0x7f61f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.083+0000 7f620a59c700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f61f40a1120 0x7f61f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.083+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62040cc540 con 0x7f620c1018f0 2024-05-08T23:17:37.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.084+0000 7f620a59c700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f61f40a1120 0x7f61f40a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009990 tx=0x7f61fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.086+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6204096200 con 0x7f620c1018f0 2024-05-08T23:17:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.360+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f620c067c10 con 0x7f620c1018f0 2024-05-08T23:17:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.361+0000 7f61fb7fe700 1 -- 172.21.15.16:0/1675023588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8091 (secure 0 0 0) 0x7f6204003cb0 con 0x7f620c1018f0 2024-05-08T23:17:37.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:37.362 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f61f40a1120 msgr2=0x7f61f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f61f40a1120 0x7f61f40a35e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009990 tx=0x7f61fc01a040 comp rx=0 tx=0).stop 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 msgr2=0x7f620c1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 0x7f620c1a0400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f620400ef30 tx=0x7f620400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 shutdown_connections 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f620c1018f0 0x7f620c1a0400 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f61f40a1120 0x7f61f40a35e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620c0ffd60 0x7f620c068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 --2- 172.21.15.16:0/1675023588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f620c100f50 0x7f620c19deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 >> 172.21.15.16:0/1675023588 conn(0x7f620c0fb2f0 msgr2=0x7f620c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 shutdown_connections 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:37.363+0000 7f6210984700 1 -- 172.21.15.16:0/1675023588 wait complete. 2024-05-08T23:17:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-05-08T23:17:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:37.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:37.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:37.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: Deploying daemon mds.a on smithi016 2024-05-08T23:17:37.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.16:6838/495357433,v1:172.21.15.16:6839/495357433] up:boot 2024-05-08T23:17:37.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: fsmap 10 up:standby 2024-05-08T23:17:37.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:17:37.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:37 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1675023588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: Deploying daemon mds.a on smithi016 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.16:6838/495357433,v1:172.21.15.16:6839/495357433] up:boot 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: fsmap 10 up:standby 2024-05-08T23:17:37.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:17:37.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:37 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1675023588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: Deploying daemon mds.a on smithi016 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.16:6838/495357433,v1:172.21.15.16:6839/495357433] up:boot 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: fsmap 10 up:standby 2024-05-08T23:17:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:17:37.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:37 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1675023588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:38.113 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:17:37 smithi016 systemd[1]: Starting Ceph mds.a for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:38.613 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:17:38 smithi016 bash[95929]: 851fd1374674080b05839ca86a5ee3a73a67a45f889e5bfc5355d236ceeacd4a 2024-05-08T23:17:38.614 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:17:38 smithi016 ceph-mds[95967]: -- 172.21.15.16:0/578790059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a38f11a1a0 con 0x55a38f0f0c00 2024-05-08T23:17:38.614 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:17:38 smithi016 systemd[1]: Started Ceph mds.a for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:38.614 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:17:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-a[95944]: starting mds.a at 2024-05-08T23:17:39.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 -- 172.21.15.16:0/2762698763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381042d0 msgr2=0x7fa8381066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/2762698763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381042d0 0x7fa8381066c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009990 tx=0x7fa82c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 -- 172.21.15.16:0/2762698763 shutdown_connections 2024-05-08T23:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/2762698763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 0x7fa838108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/2762698763 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8381019a0 0x7fa838103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/2762698763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381042d0 0x7fa8381066c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 -- 172.21.15.16:0/2762698763 >> 172.21.15.16:0/2762698763 conn(0x7fa8380fb2f0 msgr2=0x7fa8380fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 -- 172.21.15.16:0/2762698763 shutdown_connections 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.532+0000 7fa83e6b7700 1 -- 172.21.15.16:0/2762698763 wait complete. 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 Processor -- start 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 -- start start 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 0x7fa838198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8381042d0 0x7fa838199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 0x7fa8381a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381a0a50 con 0x7fa8381019a0 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381a0b90 con 0x7fa8381042d0 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83e6b7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381a0d20 con 0x7fa838106c00 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa837fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 0x7fa838198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa837fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 0x7fa838198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53928/0 (socket says 172.21.15.16:53928) 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa837fff700 1 -- 172.21.15.16:0/3029675971 learned_addr learned my addr 172.21.15.16:0/3029675971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa8377fe700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8381042d0 0x7fa838199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.533+0000 7fa83cc54700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 0x7fa8381a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa837fff700 1 -- 172.21.15.16:0/3029675971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 msgr2=0x7fa8381a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa837fff700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 0x7fa8381a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa837fff700 1 -- 172.21.15.16:0/3029675971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8381042d0 msgr2=0x7fa838199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa837fff700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8381042d0 0x7fa838199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa837fff700 1 -- 172.21.15.16:0/3029675971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009520 con 0x7fa8381019a0 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa83cc54700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 0x7fa8381a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa837fff700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 0x7fa838198f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa82800eb30 tx=0x7fa82800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828009880 con 0x7fa8381019a0 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8280044b0 con 0x7fa8381019a0 2024-05-08T23:17:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c0095f0 con 0x7fa8381019a0 2024-05-08T23:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828003df0 con 0x7fa8381019a0 2024-05-08T23:17:39.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.534+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8381a1290 con 0x7fa8381019a0 2024-05-08T23:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.535+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa838108780 con 0x7fa8381019a0 2024-05-08T23:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.536+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa828003f50 con 0x7fa8381019a0 2024-05-08T23:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.537+0000 7fa8357fa700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa8200a1120 0x7fa8200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.537+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa828014070 con 0x7fa8381019a0 2024-05-08T23:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.537+0000 7fa8377fe700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa8200a1120 0x7fa8200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.537+0000 7fa8377fe700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa8200a1120 0x7fa8200a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa82c006010 tx=0x7fa82c00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.540+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa828096550 con 0x7fa8381019a0 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: Deploying daemon mds.b on smithi155 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.16:6840/2654183877,v1:172.21.15.16:6841/2654183877] up:boot 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: fsmap 11 up:standby 2024-05-08T23:17:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:39 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:17:39.681 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:17:39 smithi155 systemd[1]: Starting Ceph mds.b for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: Deploying daemon mds.b on smithi155 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.16:6840/2654183877,v1:172.21.15.16:6841/2654183877] up:boot 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: fsmap 11 up:standby 2024-05-08T23:17:39.682 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:39 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:17:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.809+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa83810c830 con 0x7fa8381019a0 2024-05-08T23:17:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.810+0000 7fa8357fa700 1 -- 172.21.15.16:0/3029675971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8849 (secure 0 0 0) 0x7fa828099b70 con 0x7fa8381019a0 2024-05-08T23:17:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:39.811 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa8200a1120 msgr2=0x7fa8200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa8200a1120 0x7fa8200a35e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa82c006010 tx=0x7fa82c00b610 comp rx=0 tx=0).stop 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 msgr2=0x7fa838198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 0x7fa838198f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa82800eb30 tx=0x7fa82800ee40 comp rx=0 tx=0).stop 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 shutdown_connections 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa8200a1120 0x7fa8200a35e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa838106c00 0x7fa8381a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8381019a0 0x7fa838198f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 --2- 172.21.15.16:0/3029675971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8381042d0 0x7fa838199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 >> 172.21.15.16:0/3029675971 conn(0x7fa8380fb2f0 msgr2=0x7fa8381010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 shutdown_connections 2024-05-08T23:17:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:39.813+0000 7fa83e6b7700 1 -- 172.21.15.16:0/3029675971 wait complete. 2024-05-08T23:17:39.815 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-05-08T23:17:39.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: Deploying daemon mds.b on smithi155 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.16:6840/2654183877,v1:172.21.15.16:6841/2654183877] up:boot 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: fsmap 11 up:standby 2024-05-08T23:17:39.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:39 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:17:40.438 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mds[83862]: -- 172.21.15.155:0/2265286001 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5607a96681a0 con 0x5607a963ec00 2024-05-08T23:17:40.438 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: starting mds.b at 2024-05-08T23:17:40.438 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:17:40 smithi155 bash[83824]: 706614860b42eceae16b0b0ae2b0e96575ca64131c5fbc7cf2b958e2883b9cfb 2024-05-08T23:17:40.438 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:17:40 smithi155 systemd[1]: Started Ceph mds.b for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3029675971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:40 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3029675971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:40 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:40.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3029675971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:40.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:40.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:40 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:41 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] up:boot 2024-05-08T23:17:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:41 smithi155 ceph-mon[47105]: fsmap 12 up:standby 2024-05-08T23:17:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:17:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:41 smithi155 ceph-mon[47105]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:41.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:41 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:41.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:41 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] up:boot 2024-05-08T23:17:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:41 smithi016 ceph-mon[42541]: fsmap 12 up:standby 2024-05-08T23:17:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:17:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:41 smithi016 ceph-mon[42541]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:41.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:41 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:41.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:41 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] up:boot 2024-05-08T23:17:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:41 smithi189 ceph-mon[46919]: fsmap 12 up:standby 2024-05-08T23:17:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:17:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:41 smithi189 ceph-mon[46919]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:41.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:41 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.976+0000 7f83e26ff700 1 -- 172.21.15.16:0/56499484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc100e70 msgr2=0x7f83dc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.976+0000 7f83e26ff700 1 --2- 172.21.15.16:0/56499484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc100e70 0x7f83dc101290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009990 tx=0x7f83cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.976+0000 7f83e26ff700 1 -- 172.21.15.16:0/56499484 shutdown_connections 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.976+0000 7f83e26ff700 1 --2- 172.21.15.16:0/56499484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83dc102980 0x7f83dc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.976+0000 7f83e26ff700 1 --2- 172.21.15.16:0/56499484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc100e70 0x7f83dc101290 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.976+0000 7f83e26ff700 1 --2- 172.21.15.16:0/56499484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83dc101fc0 0x7f83dc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.977+0000 7f83e26ff700 1 -- 172.21.15.16:0/56499484 >> 172.21.15.16:0/56499484 conn(0x7f83dc0fc400 msgr2=0x7f83dc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.977+0000 7f83e26ff700 1 -- 172.21.15.16:0/56499484 shutdown_connections 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.977+0000 7f83e26ff700 1 -- 172.21.15.16:0/56499484 wait complete. 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.977+0000 7f83e26ff700 1 Processor -- start 2024-05-08T23:17:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.977+0000 7f83e26ff700 1 -- start start 2024-05-08T23:17:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83dc100e70 0x7f83dc196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 0x7f83dc1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83dc102980 0x7f83dc19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc19e880 con 0x7f83dc101fc0 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc19e9f0 con 0x7f83dc100e70 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc19ebb0 con 0x7f83dc102980 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 0x7f83dc1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 0x7f83dc1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53958/0 (socket says 172.21.15.16:53958) 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 -- 172.21.15.16:0/500790224 learned_addr learned my addr 172.21.15.16:0/500790224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e0c9c700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83dc102980 0x7f83dc19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83dbfff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83dc100e70 0x7f83dc196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 -- 172.21.15.16:0/500790224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83dc102980 msgr2=0x7f83dc19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83dc102980 0x7f83dc19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 -- 172.21.15.16:0/500790224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83dc100e70 msgr2=0x7f83dc196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83dc100e70 0x7f83dc196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 -- 172.21.15.16:0/500790224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0095f0 con 0x7f83dc101fc0 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83db7fe700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 0x7f83dc1972d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f83d000e9a0 tx=0x7f83d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0009790 con 0x7f83dc101fc0 2024-05-08T23:17:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83d000cbd0 con 0x7f83dc101fc0 2024-05-08T23:17:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.978+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc19ee90 con 0x7f83dc101fc0 2024-05-08T23:17:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.979+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0018680 con 0x7f83dc101fc0 2024-05-08T23:17:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.979+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc19f310 con 0x7f83dc101fc0 2024-05-08T23:17:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.979+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc1083d0 con 0x7f83dc101fc0 2024-05-08T23:17:41.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.981+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f83d000cd40 con 0x7f83dc101fc0 2024-05-08T23:17:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.981+0000 7f83d97fa700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f83c40a11a0 0x7f83c40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.981+0000 7f83dbfff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f83c40a11a0 0x7f83c40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.981+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83d0014070 con 0x7f83dc101fc0 2024-05-08T23:17:41.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.982+0000 7f83dbfff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f83c40a11a0 0x7f83c40a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83dc198210 tx=0x7f83cc00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:41.986+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f83d0052d60 con 0x7f83dc101fc0 2024-05-08T23:17:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.283+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83dc067380 con 0x7f83dc101fc0 2024-05-08T23:17:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.284+0000 7f83d97fa700 1 -- 172.21.15.16:0/500790224 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9607 (secure 0 0 0) 0x7f83d0096280 con 0x7f83dc101fc0 2024-05-08T23:17:42.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:42.285 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24563,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T23:17:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.286+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f83c40a11a0 msgr2=0x7f83c40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.286+0000 7f83e26ff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f83c40a11a0 0x7f83c40a3660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83dc198210 tx=0x7f83cc00b5b0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.286+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 msgr2=0x7f83dc1972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 0x7f83dc1972d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f83d000e9a0 tx=0x7f83d000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 shutdown_connections 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f83c40a11a0 0x7f83c40a3660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83dc102980 0x7f83dc19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83dc100e70 0x7f83dc196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 --2- 172.21.15.16:0/500790224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83dc101fc0 0x7f83dc1972d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 >> 172.21.15.16:0/500790224 conn(0x7f83dc0fc400 msgr2=0x7f83dc105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 shutdown_connections 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.287+0000 7f83e26ff700 1 -- 172.21.15.16:0/500790224 wait complete. 2024-05-08T23:17:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-05-08T23:17:42.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 -- 172.21.15.16:0/298644592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 msgr2=0x7fc7c8106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 --2- 172.21.15.16:0/298644592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8106630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009990 tx=0x7fc7b8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 -- 172.21.15.16:0/298644592 shutdown_connections 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 --2- 172.21.15.16:0/298644592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8106b70 0x7fc7c8108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 --2- 172.21.15.16:0/298644592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c8101910 0x7fc7c8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 --2- 172.21.15.16:0/298644592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8106630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.492+0000 7fc7d0759700 1 -- 172.21.15.16:0/298644592 >> 172.21.15.16:0/298644592 conn(0x7fc7c80fb300 msgr2=0x7fc7c80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 -- 172.21.15.16:0/298644592 shutdown_connections 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 -- 172.21.15.16:0/298644592 wait complete. 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 Processor -- start 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 -- start start 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8101910 0x7fc7c8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c8106b70 0x7fc7c81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81a09f0 con 0x7fc7c8104240 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81a0b30 con 0x7fc7c8101910 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.493+0000 7fc7d0759700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81a0cc0 con 0x7fc7c8106b70 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53972/0 (socket says 172.21.15.16:53972) 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 -- 172.21.15.16:0/2871389778 learned_addr learned my addr 172.21.15.16:0/2871389778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cecf6700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c8106b70 0x7fc7c81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7ce4f5700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8101910 0x7fc7c8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 -- 172.21.15.16:0/2871389778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c8106b70 msgr2=0x7fc7c81a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c8106b70 0x7fc7c81a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 -- 172.21.15.16:0/2871389778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8101910 msgr2=0x7fc7c8198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8101910 0x7fc7c8198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 -- 172.21.15.16:0/2871389778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80095f0 con 0x7fc7c8104240 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7cdcf4700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8199400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8004710 tx=0x7fc7b8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b801d070 con 0x7fc7c8104240 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c81a0f40 con 0x7fc7c8104240 2024-05-08T23:17:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7b800be70 con 0x7fc7c8104240 2024-05-08T23:17:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8021410 con 0x7fc7c8104240 2024-05-08T23:17:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.494+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81a1310 con 0x7fc7c8104240 2024-05-08T23:17:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.495+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8067ff0 con 0x7fc7c8104240 2024-05-08T23:17:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.497+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc7b800f680 con 0x7fc7c8104240 2024-05-08T23:17:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.498+0000 7fc7bf7fe700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7b40a1120 0x7fc7b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.498+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7b80cda40 con 0x7fc7c8104240 2024-05-08T23:17:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.498+0000 7fc7ce4f5700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7b40a1120 0x7fc7b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.498+0000 7fc7ce4f5700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7b40a1120 0x7fc7b40a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007c60 tx=0x7fc7c40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.501+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc7b8097660 con 0x7fc7c8104240 2024-05-08T23:17:42.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:42.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:42 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/500790224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:42.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:42 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/500790224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.736+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c810c4c0 con 0x7fc7c8104240 2024-05-08T23:17:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.737+0000 7fc7bf7fe700 1 -- 172.21.15.16:0/2871389778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18433 (secure 0 0 0) 0x7fc7b809ac80 con 0x7fc7c8104240 2024-05-08T23:17:42.738 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:42.738 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:11.293661+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.739+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7b40a1120 msgr2=0x7fc7b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.739+0000 7fc7d0759700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7b40a1120 0x7fc7b40a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007c60 tx=0x7fc7c40073d0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.739+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 msgr2=0x7fc7c8199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.739+0000 7fc7d0759700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8199400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8004710 tx=0x7fc7b8009580 comp rx=0 tx=0).stop 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 shutdown_connections 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7b40a1120 0x7fc7b40a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c8106b70 0x7fc7c81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7c8101910 0x7fc7c8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 --2- 172.21.15.16:0/2871389778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c8104240 0x7fc7c8199400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 >> 172.21.15.16:0/2871389778 conn(0x7fc7c80fb300 msgr2=0x7fc7c80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 shutdown_connections 2024-05-08T23:17:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.740+0000 7fc7d0759700 1 -- 172.21.15.16:0/2871389778 wait complete. 2024-05-08T23:17:42.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T23:17:42.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:17:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 -- 172.21.15.16:0/3022233632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec4101990 msgr2=0x7f4ec41082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 --2- 172.21.15.16:0/3022233632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec4101990 0x7f4ec41082b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00b0a0 tx=0x7f4ebc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 -- 172.21.15.16:0/3022233632 shutdown_connections 2024-05-08T23:17:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 --2- 172.21.15.16:0/3022233632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec4101990 0x7f4ec41082b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 --2- 172.21.15.16:0/3022233632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ec40ffe80 0x7f4ec41002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 --2- 172.21.15.16:0/3022233632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4100fd0 0x7f4ec4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 -- 172.21.15.16:0/3022233632 >> 172.21.15.16:0/3022233632 conn(0x7f4ec40fb3f0 msgr2=0x7f4ec40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 -- 172.21.15.16:0/3022233632 shutdown_connections 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 -- 172.21.15.16:0/3022233632 wait complete. 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.912+0000 7f4ecb758700 1 Processor -- start 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 -- start start 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 0x7f4ec4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4100fd0 0x7f4ec41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ec4101990 0x7f4ec41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a0ca0 con 0x7f4ec40ffe80 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a0de0 con 0x7f4ec4101990 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ecb758700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec41a0f70 con 0x7f4ec4100fd0 2024-05-08T23:17:42.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 0x7f4ec4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 0x7f4ec4199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53990/0 (socket says 172.21.15.16:53990) 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 -- 172.21.15.16:0/2028012568 learned_addr learned my addr 172.21.15.16:0/2028012568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec8cf3700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4100fd0 0x7f4ec41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec9cf5700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ec4101990 0x7f4ec41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 -- 172.21.15.16:0/2028012568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4100fd0 msgr2=0x7f4ec41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4100fd0 0x7f4ec41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 -- 172.21.15.16:0/2028012568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ec4101990 msgr2=0x7f4ec41a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ec4101990 0x7f4ec41a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 -- 172.21.15.16:0/2028012568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009520 con 0x7f4ec40ffe80 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.913+0000 7f4ec94f4700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 0x7f4ec4199170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000eb30 tx=0x7f4ec000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.914+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0009880 con 0x7f4ec40ffe80 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.914+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec00044b0 con 0x7f4ec40ffe80 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.914+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc009d00 con 0x7f4ec40ffe80 2024-05-08T23:17:42.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.914+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0003ce0 con 0x7f4ec40ffe80 2024-05-08T23:17:42.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.914+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec41a15d0 con 0x7f4ec40ffe80 2024-05-08T23:17:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.915+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec4068140 con 0x7f4ec40ffe80 2024-05-08T23:17:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.916+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4ec0003e40 con 0x7f4ec40ffe80 2024-05-08T23:17:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.917+0000 7f4eba7fc700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4eb00a1120 0x7f4eb00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.917+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4ec0014070 con 0x7f4ec40ffe80 2024-05-08T23:17:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.917+0000 7f4ec8cf3700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4eb00a1120 0x7f4eb00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:42.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.917+0000 7f4ec8cf3700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4eb00a1120 0x7f4eb00a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009c60 tx=0x7f4eb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:42.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:17:42.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:42 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/500790224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:42.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:42.922+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4ec0096500 con 0x7f4ec40ffe80 2024-05-08T23:17:43.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.178+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ec41a18b0 con 0x7f4ec40ffe80 2024-05-08T23:17:43.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.179+0000 7f4eba7fc700 1 -- 172.21.15.16:0/2028012568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ec0099b20 con 0x7f4ec40ffe80 2024-05-08T23:17:43.180 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4eb00a1120 msgr2=0x7f4eb00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4eb00a1120 0x7f4eb00a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009c60 tx=0x7f4eb4009260 comp rx=0 tx=0).stop 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 msgr2=0x7f4ec4199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 0x7f4ec4199170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000eb30 tx=0x7f4ec000ee40 comp rx=0 tx=0).stop 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 shutdown_connections 2024-05-08T23:17:43.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ec4101990 0x7f4ec41a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec40ffe80 0x7f4ec4199170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4eb00a1120 0x7f4eb00a35e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 --2- 172.21.15.16:0/2028012568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4100fd0 0x7f4ec41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.181+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 >> 172.21.15.16:0/2028012568 conn(0x7f4ec40fb3f0 msgr2=0x7f4ec4106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.182+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 shutdown_connections 2024-05-08T23:17:43.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.182+0000 7f4ecb758700 1 -- 172.21.15.16:0/2028012568 wait complete. 2024-05-08T23:17:43.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 -- 172.21.15.16:0/3756935144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878101fc0 msgr2=0x7f3878102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 --2- 172.21.15.16:0/3756935144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878101fc0 0x7f3878102440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3868009990 tx=0x7f3868009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 -- 172.21.15.16:0/3756935144 shutdown_connections 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 --2- 172.21.15.16:0/3756935144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 0x7f38781092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 --2- 172.21.15.16:0/3756935144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878100e70 0x7f3878101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 --2- 172.21.15.16:0/3756935144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878101fc0 0x7f3878102440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 -- 172.21.15.16:0/3756935144 >> 172.21.15.16:0/3756935144 conn(0x7f38780fc400 msgr2=0x7f38780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 -- 172.21.15.16:0/3756935144 shutdown_connections 2024-05-08T23:17:43.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.341+0000 7f387e127700 1 -- 172.21.15.16:0/3756935144 wait complete. 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 Processor -- start 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 -- start start 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 0x7f3878196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878101fc0 0x7f3878197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 0x7f387819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387819e7e0 con 0x7f3878100e70 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387819e950 con 0x7f3878101fc0 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f387e127700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387819eb10 con 0x7f3878102980 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f38777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 0x7f3878196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f38777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 0x7f3878196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54012/0 (socket says 172.21.15.16:54012) 2024-05-08T23:17:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f38777fe700 1 -- 172.21.15.16:0/3465788909 learned_addr learned my addr 172.21.15.16:0/3465788909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.342+0000 7f3876ffd700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878101fc0 0x7f3878197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f3877fff700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 0x7f387819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f38777fe700 1 -- 172.21.15.16:0/3465788909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 msgr2=0x7f387819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f38777fe700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 0x7f387819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f38777fe700 1 -- 172.21.15.16:0/3465788909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878101fc0 msgr2=0x7f3878197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f38777fe700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878101fc0 0x7f3878197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f38777fe700 1 -- 172.21.15.16:0/3465788909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680095f0 con 0x7f3878100e70 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f3877fff700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 0x7f387819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f3876ffd700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878101fc0 0x7f3878197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:43.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f38777fe700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 0x7f3878196cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e9a0 tx=0x7f386c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c009790 con 0x7f3878100e70 2024-05-08T23:17:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386c00cbd0 con 0x7f3878100e70 2024-05-08T23:17:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387819edf0 con 0x7f3878100e70 2024-05-08T23:17:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c018680 con 0x7f3878100e70 2024-05-08T23:17:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.343+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387819f270 con 0x7f3878100e70 2024-05-08T23:17:43.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.344+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38781082e0 con 0x7f3878100e70 2024-05-08T23:17:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.345+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f386c00cd40 con 0x7f3878100e70 2024-05-08T23:17:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.346+0000 7f3874ff9700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f38600a11a0 0x7f38600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.346+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f386c014070 con 0x7f3878100e70 2024-05-08T23:17:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.346+0000 7f3876ffd700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f38600a11a0 0x7f38600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.346+0000 7f3876ffd700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f38600a11a0 0x7f38600a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3868006010 tx=0x7f386800b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:43.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.350+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f386c052d60 con 0x7f3878100e70 2024-05-08T23:17:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.578+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3878068060 con 0x7f3878100e70 2024-05-08T23:17:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.579+0000 7f3874ff9700 1 -- 172.21.15.16:0/3465788909 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7f386c096280 con 0x7f3878100e70 2024-05-08T23:17:43.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:43.580 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-05-08T23:17:43.580 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:17:43.580 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-05-08T23:17:43.580 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-05-08T23:17:43.580 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:17:43.580 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f38600a11a0 msgr2=0x7f38600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f38600a11a0 0x7f38600a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3868006010 tx=0x7f386800b5d0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 msgr2=0x7f3878196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 0x7f3878196cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e9a0 tx=0x7f386c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 shutdown_connections 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f38600a11a0 0x7f38600a3660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3878102980 0x7f387819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878100e70 0x7f3878196cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 --2- 172.21.15.16:0/3465788909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3878101fc0 0x7f3878197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 >> 172.21.15.16:0/3465788909 conn(0x7f38780fc400 msgr2=0x7f3878105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 shutdown_connections 2024-05-08T23:17:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.581+0000 7f387e127700 1 -- 172.21.15.16:0/3465788909 wait complete. 2024-05-08T23:17:43.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:43.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2871389778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:43.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.587 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2028012568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.588 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:43 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3465788909' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:17:43.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T23:17:43.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.741+0000 7f48965c8700 1 -- 172.21.15.16:0/1784033546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48900ffe60 msgr2=0x7f4890100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.741+0000 7f48965c8700 1 --2- 172.21.15.16:0/1784033546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48900ffe60 0x7f4890100280 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4880009990 tx=0x7f4880009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 -- 172.21.15.16:0/1784033546 shutdown_connections 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 --2- 172.21.15.16:0/1784033546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4890101970 0x7f4890108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 --2- 172.21.15.16:0/1784033546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48900ffe60 0x7f4890100280 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 --2- 172.21.15.16:0/1784033546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4890100fb0 0x7f4890101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 -- 172.21.15.16:0/1784033546 >> 172.21.15.16:0/1784033546 conn(0x7f48900fb3f0 msgr2=0x7f48900fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:43.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 -- 172.21.15.16:0/1784033546 shutdown_connections 2024-05-08T23:17:43.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 -- 172.21.15.16:0/1784033546 wait complete. 2024-05-08T23:17:43.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 Processor -- start 2024-05-08T23:17:43.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.742+0000 7f48965c8700 1 -- start start 2024-05-08T23:17:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f48965c8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48900ffe60 0x7f4890196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f48965c8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 0x7f4890197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f48965c8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4890101970 0x7f489019e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f48965c8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019e940 con 0x7f4890100fb0 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f48965c8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019ea80 con 0x7f4890101970 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f48965c8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019ec10 con 0x7f48900ffe60 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 0x7f4890197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 0x7f4890197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54032/0 (socket says 172.21.15.16:54032) 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 -- 172.21.15.16:0/1666499744 learned_addr learned my addr 172.21.15.16:0/1666499744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488ffff700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48900ffe60 0x7f4890196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f4894b65700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4890101970 0x7f489019e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 -- 172.21.15.16:0/1666499744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48900ffe60 msgr2=0x7f4890196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48900ffe60 0x7f4890196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 -- 172.21.15.16:0/1666499744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4890101970 msgr2=0x7f489019e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4890101970 0x7f489019e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 -- 172.21.15.16:0/1666499744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800095f0 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.743+0000 7f488f7fe700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 0x7f4890197350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f488400d950 tx=0x7f488400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.744+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884009940 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.744+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4884010460 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.744+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489019eef0 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.744+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400f620 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.744+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489019f370 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.745+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890107310 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.746+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f48840105d0 con 0x7f4890100fb0 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.746+0000 7f488d7fa700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48780a1120 0x7f48780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.747+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f48840cc260 con 0x7f4890100fb0 2024-05-08T23:17:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.747+0000 7f488ffff700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48780a1120 0x7f48780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.747+0000 7f488ffff700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48780a1120 0x7f48780a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f488000c010 tx=0x7f4880003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:43.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.751+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4884095e90 con 0x7f4890100fb0 2024-05-08T23:17:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2871389778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:43.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2028012568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.921 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:43.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:43.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.922 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:43 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3465788909' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2871389778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2028012568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:17:43.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:17:43.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:17:43.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:43 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3465788909' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T23:17:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:43.967+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4890067cc0 con 0x7f4890100fb0 2024-05-08T23:17:44.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.594+0000 7f488d7fa700 1 -- 172.21.15.16:0/1666499744 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f48840994b0 con 0x7f4890100fb0 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.600+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48780a1120 msgr2=0x7f48780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.600+0000 7f48965c8700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48780a1120 0x7f48780a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f488000c010 tx=0x7f4880003720 comp rx=0 tx=0).stop 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.600+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 msgr2=0x7f4890197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.600+0000 7f48965c8700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 0x7f4890197350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f488400d950 tx=0x7f488400dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 shutdown_connections 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4890101970 0x7f489019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48780a1120 0x7f48780a35e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48900ffe60 0x7f4890196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 --2- 172.21.15.16:0/1666499744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4890100fb0 0x7f4890197350 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f488400d950 tx=0x7f488400dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 >> 172.21.15.16:0/1666499744 conn(0x7f48900fb3f0 msgr2=0x7f4890104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 shutdown_connections 2024-05-08T23:17:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.601+0000 7f48965c8700 1 -- 172.21.15.16:0/1666499744 wait complete. 2024-05-08T23:17:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-05-08T23:17:44.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 -- 172.21.15.16:0/1646151154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101990 msgr2=0x7f4e881082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/1646151154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101990 0x7f4e881082b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000b0a0 tx=0x7f4e8000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 -- 172.21.15.16:0/1646151154 shutdown_connections 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/1646151154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101990 0x7f4e881082b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/1646151154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e880ffe80 0x7f4e881002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/1646151154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88100fd0 0x7f4e88101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 -- 172.21.15.16:0/1646151154 >> 172.21.15.16:0/1646151154 conn(0x7f4e880fb3f0 msgr2=0x7f4e880fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 -- 172.21.15.16:0/1646151154 shutdown_connections 2024-05-08T23:17:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 -- 172.21.15.16:0/1646151154 wait complete. 2024-05-08T23:17:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.768+0000 7f4e8fab0700 1 Processor -- start 2024-05-08T23:17:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 -- start start 2024-05-08T23:17:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e880ffe80 0x7f4e88069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 0x7f4e8819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88101990 0x7f4e8819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8819e9c0 con 0x7f4e88100fd0 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8819eb30 con 0x7f4e88101990 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8fab0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8819ecf0 con 0x7f4e880ffe80 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 0x7f4e8819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 0x7f4e8819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54050/0 (socket says 172.21.15.16:54050) 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 -- 172.21.15.16:0/3315376615 learned_addr learned my addr 172.21.15.16:0/3315376615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d84c700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e880ffe80 0x7f4e88069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 -- 172.21.15.16:0/3315376615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e880ffe80 msgr2=0x7f4e88069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8e04d700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88101990 0x7f4e8819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e880ffe80 0x7f4e88069fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 -- 172.21.15.16:0/3315376615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88101990 msgr2=0x7f4e8819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88101990 0x7f4e8819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 -- 172.21.15.16:0/3315376615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80009d00 con 0x7f4e88100fd0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8e04d700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88101990 0x7f4e8819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e8d04b700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 0x7f4e8819bc30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400e9a0 tx=0x7f4e8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.769+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84009790 con 0x7f4e88100fd0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.770+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e8400cbd0 con 0x7f4e88100fd0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.770+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8819efd0 con 0x7f4e88100fd0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.770+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84018680 con 0x7f4e88100fd0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.770+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8819f4a0 con 0x7f4e88100fd0 2024-05-08T23:17:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.771+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e881072f0 con 0x7f4e88100fd0 2024-05-08T23:17:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.772+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4e8400cd40 con 0x7f4e88100fd0 2024-05-08T23:17:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.772+0000 7f4e7affd700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e740a11f0 0x7f4e740a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.773+0000 7f4e8d84c700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e740a11f0 0x7f4e740a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.773+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f4e84014070 con 0x7f4e88100fd0 2024-05-08T23:17:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.773+0000 7f4e8d84c700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e740a11f0 0x7f4e740a36b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009c60 tx=0x7f4e7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.777+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e840965c0 con 0x7f4e88100fd0 2024-05-08T23:17:44.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1666499744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:17:44.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1666499744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:17:44.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1666499744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T23:17:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:44.998+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f4e88068140 con 0x7f4e88100fd0 2024-05-08T23:17:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.600+0000 7f4e7affd700 1 -- 172.21.15.16:0/3315376615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4e84099be0 con 0x7f4e88100fd0 2024-05-08T23:17:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.610+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e740a11f0 msgr2=0x7f4e740a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.610+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e740a11f0 0x7f4e740a36b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009c60 tx=0x7f4e7c009260 comp rx=0 tx=0).stop 2024-05-08T23:17:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.610+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 msgr2=0x7f4e8819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.610+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 0x7f4e8819bc30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400e9a0 tx=0x7f4e8400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.613+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 shutdown_connections 2024-05-08T23:17:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.613+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e88101990 0x7f4e8819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.613+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e740a11f0 0x7f4e740a36b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.613+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e880ffe80 0x7f4e88069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.613+0000 7f4e8fab0700 1 --2- 172.21.15.16:0/3315376615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88100fd0 0x7f4e8819bc30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.613+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 >> 172.21.15.16:0/3315376615 conn(0x7f4e880fb3f0 msgr2=0x7f4e88104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.623+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 shutdown_connections 2024-05-08T23:17:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.623+0000 7f4e8fab0700 1 -- 172.21.15.16:0/3315376615 wait complete. 2024-05-08T23:17:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-05-08T23:17:45.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 -- 172.21.15.16:0/788750628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c104310 msgr2=0x7fab5c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 --2- 172.21.15.16:0/788750628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c104310 0x7fab5c106700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab44009990 tx=0x7fab44009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 -- 172.21.15.16:0/788750628 shutdown_connections 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 --2- 172.21.15.16:0/788750628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c106c40 0x7fab5c1090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 --2- 172.21.15.16:0/788750628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab5c1019e0 0x7fab5c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 --2- 172.21.15.16:0/788750628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c104310 0x7fab5c106700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 -- 172.21.15.16:0/788750628 >> 172.21.15.16:0/788750628 conn(0x7fab5c0fb3f0 msgr2=0x7fab5c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:45.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 -- 172.21.15.16:0/788750628 shutdown_connections 2024-05-08T23:17:45.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.795+0000 7fab60903700 1 -- 172.21.15.16:0/788750628 wait complete. 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 Processor -- start 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 -- start start 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 0x7fab5c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c104310 0x7fab5c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab5c106c40 0x7fab5c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c19e960 con 0x7fab5c1019e0 2024-05-08T23:17:45.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c19eaa0 con 0x7fab5c104310 2024-05-08T23:17:45.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab60903700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c19ec30 con 0x7fab5c106c40 2024-05-08T23:17:45.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab5a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 0x7fab5c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:45.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab5a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 0x7fab5c196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54070/0 (socket says 172.21.15.16:54070) 2024-05-08T23:17:45.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab5a59c700 1 -- 172.21.15.16:0/3914863318 learned_addr learned my addr 172.21.15.16:0/3914863318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:45.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.796+0000 7fab51d9b700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c104310 0x7fab5c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5ad9d700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab5c106c40 0x7fab5c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5a59c700 1 -- 172.21.15.16:0/3914863318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab5c106c40 msgr2=0x7fab5c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5a59c700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab5c106c40 0x7fab5c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5a59c700 1 -- 172.21.15.16:0/3914863318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c104310 msgr2=0x7fab5c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5a59c700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c104310 0x7fab5c197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5a59c700 1 -- 172.21.15.16:0/3914863318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440095f0 con 0x7fab5c1019e0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab5a59c700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 0x7fab5c196e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00e9a0 tx=0x7fab4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c009790 con 0x7fab5c1019e0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c19ef10 con 0x7fab5c1019e0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab4c010430 con 0x7fab5c1019e0 2024-05-08T23:17:45.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00cea0 con 0x7fab5c1019e0 2024-05-08T23:17:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.797+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c19f340 con 0x7fab5c1019e0 2024-05-08T23:17:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.798+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c0fcf70 con 0x7fab5c1019e0 2024-05-08T23:17:45.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.800+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fab4c0105a0 con 0x7fab5c1019e0 2024-05-08T23:17:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.800+0000 7fab53fff700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fab480a1120 0x7fab480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.800+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fab4c014070 con 0x7fab5c1019e0 2024-05-08T23:17:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.800+0000 7fab51d9b700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fab480a1120 0x7fab480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:45.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.801+0000 7fab51d9b700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fab480a1120 0x7fab480a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab44006010 tx=0x7fab4400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:45.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:45.804+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab4c0963d0 con 0x7fab5c1019e0 2024-05-08T23:17:45.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:45 smithi189 ceph-mon[46919]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:45.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:45 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1666499744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:17:45.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:45 smithi189 ceph-mon[46919]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T23:17:45.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:45 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3315376615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:17:45.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:45 smithi155 ceph-mon[47105]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:45.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:45 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1666499744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:17:45.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:45 smithi155 ceph-mon[47105]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T23:17:45.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:45 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3315376615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:17:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.075+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fab5c067cf0 con 0x7fab5c1019e0 2024-05-08T23:17:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.084+0000 7fab53fff700 1 -- 172.21.15.16:0/3914863318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fab4c0260c0 con 0x7fab5c1019e0 2024-05-08T23:17:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.091+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fab480a1120 msgr2=0x7fab480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.091+0000 7fab60903700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fab480a1120 0x7fab480a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab44006010 tx=0x7fab4400b610 comp rx=0 tx=0).stop 2024-05-08T23:17:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.091+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 msgr2=0x7fab5c196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.091+0000 7fab60903700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 0x7fab5c196e30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00e9a0 tx=0x7fab4c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.094+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 shutdown_connections 2024-05-08T23:17:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.094+0000 7fab60903700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fab480a1120 0x7fab480a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.094+0000 7fab60903700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab5c106c40 0x7fab5c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.094+0000 7fab60903700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5c1019e0 0x7fab5c196e30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.094+0000 7fab60903700 1 --2- 172.21.15.16:0/3914863318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab5c104310 0x7fab5c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.094+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 >> 172.21.15.16:0/3914863318 conn(0x7fab5c0fb3f0 msgr2=0x7fab5c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.096+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 shutdown_connections 2024-05-08T23:17:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.096+0000 7fab60903700 1 -- 172.21.15.16:0/3914863318 wait complete. 2024-05-08T23:17:46.101 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T23:17:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:45 smithi016 ceph-mon[42541]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:45 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1666499744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T23:17:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:45 smithi016 ceph-mon[42541]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T23:17:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:45 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3315376615' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T23:17:46.120 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T23:17:46.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 -- 172.21.15.16:0/3581538662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07681020f0 msgr2=0x7f0768102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 --2- 172.21.15.16:0/3581538662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07681020f0 0x7f0768102570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0758009990 tx=0x7f0758009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 -- 172.21.15.16:0/3581538662 shutdown_connections 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 --2- 172.21.15.16:0/3581538662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0768102ab0 0x7f07681093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 --2- 172.21.15.16:0/3581538662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 0x7f0768101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 --2- 172.21.15.16:0/3581538662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07681020f0 0x7f0768102570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.262+0000 7f076f59a700 1 -- 172.21.15.16:0/3581538662 >> 172.21.15.16:0/3581538662 conn(0x7f07680fc4e0 msgr2=0x7f07680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 -- 172.21.15.16:0/3581538662 shutdown_connections 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 -- 172.21.15.16:0/3581538662 wait complete. 2024-05-08T23:17:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 Processor -- start 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 -- start start 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 0x7f0768196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07681020f0 0x7f07681973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 0x7f076819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819e9a0 con 0x7f0768102ab0 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819eae0 con 0x7f07681020f0 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.263+0000 7f076f59a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819ec70 con 0x7f0768100f20 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076db37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 0x7f076819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076db37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 0x7f076819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54092/0 (socket says 172.21.15.16:54092) 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076db37700 1 -- 172.21.15.16:0/736199622 learned_addr learned my addr 172.21.15.16:0/736199622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 0x7f0768196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076cb35700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07681020f0 0x7f07681973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 -- 172.21.15.16:0/736199622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07681020f0 msgr2=0x7f07681973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07681020f0 0x7f07681973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 -- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 msgr2=0x7f076819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 0x7f076819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 -- 172.21.15.16:0/736199622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580095f0 con 0x7f0768100f20 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076db37700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 0x7f076819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076d336700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 0x7f0768196e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9c0 tx=0x7f076400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764004220 con 0x7f0768100f20 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076819ef50 con 0x7f0768100f20 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0764004380 con 0x7f0768100f20 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.264+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764003b40 con 0x7f0768100f20 2024-05-08T23:17:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.265+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076819f370 con 0x7f0768100f20 2024-05-08T23:17:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.265+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0768108500 con 0x7f0768100f20 2024-05-08T23:17:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.267+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0764003ca0 con 0x7f0768100f20 2024-05-08T23:17:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.268+0000 7f075e7fc700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07540a1120 0x7f07540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.268+0000 7f076cb35700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07540a1120 0x7f07540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.268+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0764014070 con 0x7f0768100f20 2024-05-08T23:17:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.268+0000 7f076cb35700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07540a1120 0x7f07540a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0758006010 tx=0x7f075800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.271+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0764096400 con 0x7f0768100f20 2024-05-08T23:17:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:46.502+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0768067460 con 0x7f0768100f20 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3315376615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3914863318' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3914863318' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:17:46.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:17:46.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:17:46.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/736199622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:17:46.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:46 smithi189 ceph-mon[46919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3315376615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3914863318' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3914863318' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/736199622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:17:46.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:46 smithi155 ceph-mon[47105]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:17:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.090+0000 7f075e7fc700 1 -- 172.21.15.16:0/736199622 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v77) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0764099a20 con 0x7f0768100f20 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07540a1120 msgr2=0x7f07540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07540a1120 0x7f07540a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0758006010 tx=0x7f075800b5b0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 msgr2=0x7f0768196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 0x7f0768196e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f076400e9c0 tx=0x7f076400ecd0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 shutdown_connections 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0768102ab0 0x7f076819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07540a1120 0x7f07540a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0768100f20 0x7f0768196e70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 --2- 172.21.15.16:0/736199622 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07681020f0 0x7f07681973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 >> 172.21.15.16:0/736199622 conn(0x7f07680fc4e0 msgr2=0x7f0768105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 shutdown_connections 2024-05-08T23:17:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.093+0000 7f076f59a700 1 -- 172.21.15.16:0/736199622 wait complete. 2024-05-08T23:17:47.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3315376615' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T23:17:47.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3914863318' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3914863318' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/736199622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:17:47.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:46 smithi016 ceph-mon[42541]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T23:17:47.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T23:17:47.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/2897880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a0104220 msgr2=0x7fc7a0106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/2897880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a0104220 0x7fc7a0106610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc790009990 tx=0x7fc790009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/2897880 shutdown_connections 2024-05-08T23:17:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/2897880 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7a0106b50 0x7fc7a0108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/2897880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a01018f0 0x7fc7a0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/2897880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a0104220 0x7fc7a0106610 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/2897880 >> 172.21.15.16:0/2897880 conn(0x7fc7a00fb2e0 msgr2=0x7fc7a00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/2897880 shutdown_connections 2024-05-08T23:17:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.259+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/2897880 wait complete. 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.260+0000 7fc7a4d8a700 1 Processor -- start 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.260+0000 7fc7a4d8a700 1 -- start start 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.260+0000 7fc7a4d8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 0x7fc7a0101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.260+0000 7fc7a4d8a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7a0104220 0x7fc7a00ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc7a4d8a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a0106b50 0x7fc7a00ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc7a4d8a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a0100580 con 0x7fc7a01018f0 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc7a4d8a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a01006f0 con 0x7fc7a0104220 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc7a4d8a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a01a0da0 con 0x7fc7a0106b50 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 0x7fc7a0101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 0x7fc7a0101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45022/0 (socket says 172.21.15.16:45022) 2024-05-08T23:17:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 -- 172.21.15.16:0/1279852785 learned_addr learned my addr 172.21.15.16:0/1279852785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79dd9b700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7a0104220 0x7fc7a00ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79ed9d700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a0106b50 0x7fc7a00ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 -- 172.21.15.16:0/1279852785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a0106b50 msgr2=0x7fc7a00ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a0106b50 0x7fc7a00ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 -- 172.21.15.16:0/1279852785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7a0104220 msgr2=0x7fc7a00ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7a0104220 0x7fc7a00ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 -- 172.21.15.16:0/1279852785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900095f0 con 0x7fc7a01018f0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79ed9d700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a0106b50 0x7fc7a00ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc79e59c700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 0x7fc7a0101210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc79400d950 tx=0x7fc79400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794009ae0 con 0x7fc7a01018f0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a01a0fd0 con 0x7fc7a01018f0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc794009c40 con 0x7fc7a01018f0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79400f620 con 0x7fc7a01018f0 2024-05-08T23:17:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.261+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a01a1450 con 0x7fc7a01018f0 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.262+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a00fce60 con 0x7fc7a01018f0 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.263+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc794010460 con 0x7fc7a01018f0 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.264+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7880a1110 0x7fc7880a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.264+0000 7fc79dd9b700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7880a1110 0x7fc7880a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.264+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fc7940cc820 con 0x7fc7a01018f0 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.265+0000 7fc79dd9b700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7880a1110 0x7fc7880a35d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc790003780 tx=0x7fc790005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.269+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc794095fd0 con 0x7fc7a01018f0 2024-05-08T23:17:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:47.486+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc7a0067c10 con 0x7fc7a01018f0 2024-05-08T23:17:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: pgmap v218: 97 pgs: 13 creating+peering, 1 active+clean, 83 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] up:active 2024-05-08T23:17:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T23:17:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:17:47.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:47.920 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:47 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1279852785' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: pgmap v218: 97 pgs: 13 creating+peering, 1 active+clean, 83 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] up:active 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:47.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:47 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1279852785' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: pgmap v218: 97 pgs: 13 creating+peering, 1 active+clean, 83 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] up:active 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1279852785' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T23:17:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.103+0000 7fc78f7fe700 1 -- 172.21.15.16:0/1279852785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v79) v1 ==== 228+0+0 (secure 0 0 0) 0x7fc794016100 con 0x7fc7a01018f0 2024-05-08T23:17:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.109+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7880a1110 msgr2=0x7fc7880a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.109+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7880a1110 0x7fc7880a35d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc790003780 tx=0x7fc790005fb0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.109+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 msgr2=0x7fc7a0101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.109+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 0x7fc7a0101210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc79400d950 tx=0x7fc79400dd10 comp rx=0 tx=0).stop 2024-05-08T23:17:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.110+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 shutdown_connections 2024-05-08T23:17:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.110+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7880a1110 0x7fc7880a35d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.110+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a0106b50 0x7fc7a00ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.110+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7a01018f0 0x7fc7a0101210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.110+0000 7fc7a4d8a700 1 --2- 172.21.15.16:0/1279852785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7a0104220 0x7fc7a00ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.110+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 >> 172.21.15.16:0/1279852785 conn(0x7fc7a00fb2e0 msgr2=0x7fc7a0103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.112+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 shutdown_connections 2024-05-08T23:17:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.112+0000 7fc7a4d8a700 1 -- 172.21.15.16:0/1279852785 wait complete. 2024-05-08T23:17:49.115 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-05-08T23:17:49.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T23:17:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.274+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/1518125277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0101a40 msgr2=0x7f9ed0103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.274+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/1518125277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0101a40 0x7f9ed0103e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009990 tx=0x7f9ec0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/1518125277 shutdown_connections 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/1518125277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ed0106ca0 0x7f9ed01090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/1518125277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0101a40 0x7f9ed0103e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/1518125277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ed0104370 0x7f9ed0106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/1518125277 >> 172.21.15.16:0/1518125277 conn(0x7f9ed00fb3f0 msgr2=0x7f9ed00fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/1518125277 shutdown_connections 2024-05-08T23:17:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/1518125277 wait complete. 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.275+0000 7f9ed5cc7700 1 Processor -- start 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 -- start start 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ed0101a40 0x7f9ed0196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ed0104370 0x7f9ed0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 0x7f9ed019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019e930 con 0x7f9ed0106ca0 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019ea70 con 0x7f9ed0104370 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ed5cc7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019ec00 con 0x7f9ed0101a40 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 0x7f9ed019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ecffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 0x7f9ed019e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45046/0 (socket says 172.21.15.16:45046) 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ecffff700 1 -- 172.21.15.16:0/3181571245 learned_addr learned my addr 172.21.15.16:0/3181571245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9ecf7fe700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ed0101a40 0x7f9ed0196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.276+0000 7f9eceffd700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ed0104370 0x7f9ed0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ecffff700 1 -- 172.21.15.16:0/3181571245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ed0101a40 msgr2=0x7f9ed0196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ecffff700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ed0101a40 0x7f9ed0196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ecffff700 1 -- 172.21.15.16:0/3181571245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ed0104370 msgr2=0x7f9ed0197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ecffff700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ed0104370 0x7f9ed0197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ecffff700 1 -- 172.21.15.16:0/3181571245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec00095f0 con 0x7f9ed0106ca0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ecffff700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 0x7f9ed019e360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8007ee0 tx=0x7f9ec800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8017070 con 0x7f9ed0106ca0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ec800f040 con 0x7f9ed0106ca0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed019eee0 con 0x7f9ed0106ca0 2024-05-08T23:17:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8005570 con 0x7f9ed0106ca0 2024-05-08T23:17:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.277+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed019f310 con 0x7f9ed0106ca0 2024-05-08T23:17:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.278+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00fcf70 con 0x7f9ed0106ca0 2024-05-08T23:17:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.279+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9ec800db20 con 0x7f9ed0106ca0 2024-05-08T23:17:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.280+0000 7f9eccff9700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9eb80a1120 0x7f9eb80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.280+0000 7f9ecf7fe700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9eb80a1120 0x7f9eb80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.280+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f9ec80cc890 con 0x7f9ed0106ca0 2024-05-08T23:17:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.280+0000 7f9ecf7fe700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9eb80a1120 0x7f9eb80a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0003780 tx=0x7f9ec0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.284+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9ec8095df0 con 0x7f9ed0106ca0 2024-05-08T23:17:49.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:49 smithi016 ceph-mon[42541]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:17:49.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:49 smithi016 ceph-mon[42541]: pgmap v221: 97 pgs: 13 creating+peering, 47 active+clean, 37 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-05-08T23:17:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:49 smithi189 ceph-mon[46919]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:17:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:49 smithi189 ceph-mon[46919]: pgmap v221: 97 pgs: 13 creating+peering, 47 active+clean, 37 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-05-08T23:17:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:49 smithi155 ceph-mon[47105]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T23:17:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:49 smithi155 ceph-mon[47105]: pgmap v221: 97 pgs: 13 creating+peering, 47 active+clean, 37 unknown; 578 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2024-05-08T23:17:49.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:49.500+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9ed010c640 con 0x7f9ed0106ca0 2024-05-08T23:17:50.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.111+0000 7f9eccff9700 1 -- 172.21.15.16:0/3181571245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v80) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9ec8099410 con 0x7f9ed0106ca0 2024-05-08T23:17:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.122+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9eb80a1120 msgr2=0x7f9eb80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.122+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9eb80a1120 0x7f9eb80a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0003780 tx=0x7f9ec0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.122+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 msgr2=0x7f9ed019e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.122+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 0x7f9ed019e360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8007ee0 tx=0x7f9ec800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.125+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 shutdown_connections 2024-05-08T23:17:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.125+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed0106ca0 0x7f9ed019e360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.125+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9eb80a1120 0x7f9eb80a35e0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0003780 tx=0x7f9ec0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.125+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ed0101a40 0x7f9ed0196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.125+0000 7f9ed5cc7700 1 --2- 172.21.15.16:0/3181571245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ed0104370 0x7f9ed0197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.125+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 >> 172.21.15.16:0/3181571245 conn(0x7f9ed00fb3f0 msgr2=0x7f9ed0103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.128+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 shutdown_connections 2024-05-08T23:17:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.130+0000 7f9ed5cc7700 1 -- 172.21.15.16:0/3181571245 wait complete. 2024-05-08T23:17:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T23:17:50.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 -- 172.21.15.16:0/3260198711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c104330 msgr2=0x7f874c106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 --2- 172.21.15.16:0/3260198711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c104330 0x7f874c106720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f873c009990 tx=0x7f873c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 -- 172.21.15.16:0/3260198711 shutdown_connections 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 --2- 172.21.15.16:0/3260198711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c106c60 0x7f874c1090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 --2- 172.21.15.16:0/3260198711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c101a00 0x7f874c103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 --2- 172.21.15.16:0/3260198711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c104330 0x7f874c106720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.296+0000 7f87534af700 1 -- 172.21.15.16:0/3260198711 >> 172.21.15.16:0/3260198711 conn(0x7f874c0fb3f0 msgr2=0x7f874c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 -- 172.21.15.16:0/3260198711 shutdown_connections 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 -- 172.21.15.16:0/3260198711 wait complete. 2024-05-08T23:17:50.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 Processor -- start 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 -- start start 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c101a00 0x7f874c101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c104330 0x7f874c0ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 0x7f874c0ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c100690 con 0x7f874c106c60 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c100800 con 0x7f874c104330 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f87534af700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a0e20 con 0x7f874c101a00 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.297+0000 7f8751a4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 0x7f874c0ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 0x7f874c0ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45058/0 (socket says 172.21.15.16:45058) 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 -- 172.21.15.16:0/2910488589 learned_addr learned my addr 172.21.15.16:0/2910488589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f875124b700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c101a00 0x7f874c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8750a4a700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c104330 0x7f874c0ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 -- 172.21.15.16:0/2910488589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c101a00 msgr2=0x7f874c101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c101a00 0x7f874c101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 -- 172.21.15.16:0/2910488589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c104330 msgr2=0x7f874c0ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c104330 0x7f874c0ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 -- 172.21.15.16:0/2910488589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0095f0 con 0x7f874c106c60 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f8751a4c700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 0x7f874c0ffef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8744007ee0 tx=0x7f874400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744017070 con 0x7f874c106c60 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f874400f040 con 0x7f874c106c60 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744005410 con 0x7f874c106c60 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c1a10b0 con 0x7f874c106c60 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.298+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a1570 con 0x7f874c106c60 2024-05-08T23:17:50.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.299+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c0fcf70 con 0x7f874c106c60 2024-05-08T23:17:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.301+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f874400dac0 con 0x7f874c106c60 2024-05-08T23:17:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.301+0000 7f87427fc700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f87380a11a0 0x7f87380a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.301+0000 7f875124b700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f87380a11a0 0x7f87380a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.301+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f87440cc7a0 con 0x7f874c106c60 2024-05-08T23:17:50.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.302+0000 7f875124b700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f87380a11a0 0x7f87380a3660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8748005950 tx=0x7f874800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.305+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8744095d60 con 0x7f874c106c60 2024-05-08T23:17:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:50 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1279852785' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:17:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:50 smithi016 ceph-mon[42541]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:17:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:50 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3181571245' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:17:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:50 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1279852785' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:17:50.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:50 smithi189 ceph-mon[46919]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:17:50.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:50 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3181571245' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:17:50.449 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:50 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1279852785' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T23:17:50.449 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:50 smithi155 ceph-mon[47105]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T23:17:50.449 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:50 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3181571245' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T23:17:50.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.578+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f874c067460 con 0x7f874c106c60 2024-05-08T23:17:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.584+0000 7f87427fc700 1 -- 172.21.15.16:0/2910488589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f874400dd70 con 0x7f874c106c60 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f87380a11a0 msgr2=0x7f87380a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f87380a11a0 0x7f87380a3660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8748005950 tx=0x7f874800a300 comp rx=0 tx=0).stop 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 msgr2=0x7f874c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 0x7f874c0ffef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8744007ee0 tx=0x7f874400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 shutdown_connections 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c106c60 0x7f874c0ffef0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f87380a11a0 0x7f87380a3660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c101a00 0x7f874c101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 --2- 172.21.15.16:0/2910488589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c104330 0x7f874c0ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 >> 172.21.15.16:0/2910488589 conn(0x7f874c0fb3f0 msgr2=0x7f874c103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 shutdown_connections 2024-05-08T23:17:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.587+0000 7f87534af700 1 -- 172.21.15.16:0/2910488589 wait complete. 2024-05-08T23:17:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-05-08T23:17:50.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:50.715 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:17:50 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: 2024-05-08T23:17:50.448+0000 7f9a4cd7c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T23:17:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 -- 172.21.15.16:0/3794592233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180100e70 msgr2=0x7f8180101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 --2- 172.21.15.16:0/3794592233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180100e70 0x7f8180101290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8170009990 tx=0x7f8170009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 -- 172.21.15.16:0/3794592233 shutdown_connections 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 --2- 172.21.15.16:0/3794592233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180102980 0x7f81801092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 --2- 172.21.15.16:0/3794592233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180100e70 0x7f8180101290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 --2- 172.21.15.16:0/3794592233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180101fc0 0x7f8180102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 -- 172.21.15.16:0/3794592233 >> 172.21.15.16:0/3794592233 conn(0x7f81800fc400 msgr2=0x7f81800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 -- 172.21.15.16:0/3794592233 shutdown_connections 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.740+0000 7f81851c0700 1 -- 172.21.15.16:0/3794592233 wait complete. 2024-05-08T23:17:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.741+0000 7f81851c0700 1 Processor -- start 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.741+0000 7f81851c0700 1 -- start start 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f81851c0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180100e70 0x7f8180196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f81851c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 0x7f81801971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f81851c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180102980 0x7f818019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f81851c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818019e790 con 0x7f8180101fc0 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f81851c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818019e900 con 0x7f8180102980 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f81851c0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818019eac0 con 0x7f8180100e70 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 0x7f81801971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 0x7f81801971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45074/0 (socket says 172.21.15.16:45074) 2024-05-08T23:17:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 -- 172.21.15.16:0/2370341892 learned_addr learned my addr 172.21.15.16:0/2370341892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817ed9d700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180100e70 0x7f8180196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817f59e700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180102980 0x7f818019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 -- 172.21.15.16:0/2370341892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180100e70 msgr2=0x7f8180196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180100e70 0x7f8180196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 -- 172.21.15.16:0/2370341892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180102980 msgr2=0x7f818019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180102980 0x7f818019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 -- 172.21.15.16:0/2370341892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700095f0 con 0x7f8180101fc0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817f59e700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180102980 0x7f818019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f817e59c700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 0x7f81801971e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f817400d980 tx=0x7f817400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.742+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174004220 con 0x7f8180101fc0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.743+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8174007640 con 0x7f8180101fc0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.743+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818019eda0 con 0x7f8180101fc0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.743+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81740055e0 con 0x7f8180101fc0 2024-05-08T23:17:50.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.743+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818019f1f0 con 0x7f8180101fc0 2024-05-08T23:17:50.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.744+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8180108380 con 0x7f8180101fc0 2024-05-08T23:17:50.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.745+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8174004750 con 0x7f8180101fc0 2024-05-08T23:17:50.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.746+0000 7f8167fff700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f81680a1120 0x7f81680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.746+0000 7f817ed9d700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f81680a1120 0x7f81680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.746+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81740cc9a0 con 0x7f8180101fc0 2024-05-08T23:17:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.746+0000 7f817ed9d700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f81680a1120 0x7f81680a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f817000c010 tx=0x7f8170003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:50.750+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8174095ee0 con 0x7f8180101fc0 2024-05-08T23:17:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.018+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8180067380 con 0x7f8180101fc0 2024-05-08T23:17:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.019+0000 7f8167fff700 1 -- 172.21.15.16:0/2370341892 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10816 (secure 0 0 0) 0x7f8174016020 con 0x7f8180101fc0 2024-05-08T23:17:51.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:51.020 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:17:50.580857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T23:17:51.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f81680a1120 msgr2=0x7f81680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f81680a1120 0x7f81680a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f817000c010 tx=0x7f8170003720 comp rx=0 tx=0).stop 2024-05-08T23:17:51.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 msgr2=0x7f81801971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 0x7f81801971e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f817400d980 tx=0x7f817400dc90 comp rx=0 tx=0).stop 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 shutdown_connections 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8180102980 0x7f818019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f81680a1120 0x7f81680a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180100e70 0x7f8180196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 --2- 172.21.15.16:0/2370341892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8180101fc0 0x7f81801971e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 >> 172.21.15.16:0/2370341892 conn(0x7f81800fc400 msgr2=0x7f81801054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 shutdown_connections 2024-05-08T23:17:51.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.021+0000 7f81851c0700 1 -- 172.21.15.16:0/2370341892 wait complete. 2024-05-08T23:17:51.024 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-05-08T23:17:51.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 -- 172.21.15.16:0/1393623601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd988106b40 msgr2=0x7fd988108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 --2- 172.21.15.16:0/1393623601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd988106b40 0x7fd988108fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd98000b0a0 tx=0x7fd98000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 -- 172.21.15.16:0/1393623601 shutdown_connections 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 --2- 172.21.15.16:0/1393623601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd988106b40 0x7fd988108fb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 --2- 172.21.15.16:0/1393623601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9881018e0 0x7fd988103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 --2- 172.21.15.16:0/1393623601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd988104210 0x7fd988106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 -- 172.21.15.16:0/1393623601 >> 172.21.15.16:0/1393623601 conn(0x7fd9880fb2f0 msgr2=0x7fd9880fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 -- 172.21.15.16:0/1393623601 shutdown_connections 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.174+0000 7fd98d928700 1 -- 172.21.15.16:0/1393623601 wait complete. 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 Processor -- start 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 -- start start 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 0x7fd988196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd988104210 0x7fd9881972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd988106b40 0x7fd98819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98819e8c0 con 0x7fd9881018e0 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98819ea00 con 0x7fd988106b40 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd98d928700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98819eb90 con 0x7fd988104210 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd986ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 0x7fd988196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd986ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 0x7fd988196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45098/0 (socket says 172.21.15.16:45098) 2024-05-08T23:17:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd986ffd700 1 -- 172.21.15.16:0/1065483052 learned_addr learned my addr 172.21.15.16:0/1065483052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd9877fe700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd988106b40 0x7fd98819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.175+0000 7fd97e7fc700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd988104210 0x7fd9881972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd986ffd700 1 -- 172.21.15.16:0/1065483052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd988104210 msgr2=0x7fd9881972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd986ffd700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd988104210 0x7fd9881972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd986ffd700 1 -- 172.21.15.16:0/1065483052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd988106b40 msgr2=0x7fd98819e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd986ffd700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd988106b40 0x7fd98819e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd986ffd700 1 -- 172.21.15.16:0/1065483052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009d00 con 0x7fd9881018e0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd986ffd700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 0x7fd988196d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd97800d950 tx=0x7fd97800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978009940 con 0x7fd9881018e0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd978010460 con 0x7fd9881018e0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98819ee70 con 0x7fd9881018e0 2024-05-08T23:17:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97800f620 con 0x7fd9881018e0 2024-05-08T23:17:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.176+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98819f2a0 con 0x7fd9881018e0 2024-05-08T23:17:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.177+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9880fce70 con 0x7fd9881018e0 2024-05-08T23:17:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.179+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd9780105d0 con 0x7fd9881018e0 2024-05-08T23:17:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.179+0000 7fd98c926700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd9740a11a0 0x7fd9740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.180+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9780cc900 con 0x7fd9881018e0 2024-05-08T23:17:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.180+0000 7fd97e7fc700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd9740a11a0 0x7fd9740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.180+0000 7fd97e7fc700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd9740a11a0 0x7fd9740a3660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd970005950 tx=0x7fd9700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.183+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd978095e40 con 0x7fd9881018e0 2024-05-08T23:17:51.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3181571245' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: pgmap v224: 129 pgs: 14 creating+peering, 90 active+clean, 25 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2910488589' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2910488589' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:51 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2370341892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.402+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd98810c4f0 con 0x7fd9881018e0 2024-05-08T23:17:51.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.404+0000 7fd98c926700 1 -- 172.21.15.16:0/1065483052 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7fd978020090 con 0x7fd9881018e0 2024-05-08T23:17:51.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:51.405 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:51.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd9740a11a0 msgr2=0x7fd9740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd9740a11a0 0x7fd9740a3660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd970005950 tx=0x7fd9700058e0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 msgr2=0x7fd988196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 0x7fd988196d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd97800d950 tx=0x7fd97800dc60 comp rx=0 tx=0).stop 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 shutdown_connections 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd988106b40 0x7fd98819e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9881018e0 0x7fd988196d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd9740a11a0 0x7fd9740a3660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 --2- 172.21.15.16:0/1065483052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd988104210 0x7fd9881972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 >> 172.21.15.16:0/1065483052 conn(0x7fd9880fb2f0 msgr2=0x7fd988102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 shutdown_connections 2024-05-08T23:17:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.407+0000 7fd98d928700 1 -- 172.21.15.16:0/1065483052 wait complete. 2024-05-08T23:17:51.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3181571245' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T23:17:51.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:17:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: pgmap v224: 129 pgs: 14 creating+peering, 90 active+clean, 25 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-05-08T23:17:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2910488589' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:17:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:17:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2910488589' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T23:17:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:51 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2370341892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:51.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3181571245' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: pgmap v224: 129 pgs: 14 creating+peering, 90 active+clean, 25 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2910488589' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2910488589' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:17:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:51 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2370341892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 -- 172.21.15.16:0/2827343086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 msgr2=0x7f0050101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 --2- 172.21.15.16:0/2827343086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050101290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f004c009990 tx=0x7f004c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 -- 172.21.15.16:0/2827343086 shutdown_connections 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 --2- 172.21.15.16:0/2827343086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0050102980 0x7f00501092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 --2- 172.21.15.16:0/2827343086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050101290 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 --2- 172.21.15.16:0/2827343086 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0050101fc0 0x7f0050102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 -- 172.21.15.16:0/2827343086 >> 172.21.15.16:0/2827343086 conn(0x7f00500fc400 msgr2=0x7f00500fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.558+0000 7f005876c700 1 -- 172.21.15.16:0/2827343086 shutdown_connections 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 -- 172.21.15.16:0/2827343086 wait complete. 2024-05-08T23:17:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 Processor -- start 2024-05-08T23:17:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 -- start start 2024-05-08T23:17:51.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0050101fc0 0x7f0050197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0050102980 0x7f005019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005019e810 con 0x7f0050100e70 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005019e980 con 0x7f0050101fc0 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f005876c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005019eb40 con 0x7f0050102980 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f0056508700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f0056508700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45118/0 (socket says 172.21.15.16:45118) 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.559+0000 7f0056508700 1 -- 172.21.15.16:0/2757750335 learned_addr learned my addr 172.21.15.16:0/2757750335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0055d07700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0050101fc0 0x7f0050197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056d09700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0050102980 0x7f005019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056508700 1 -- 172.21.15.16:0/2757750335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0050102980 msgr2=0x7f005019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056508700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0050102980 0x7f005019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056508700 1 -- 172.21.15.16:0/2757750335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0050101fc0 msgr2=0x7f0050197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056508700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0050101fc0 0x7f0050197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056508700 1 -- 172.21.15.16:0/2757750335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0095f0 con 0x7f0050100e70 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f0056508700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050196cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f004c005740 tx=0x7f004c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c01d070 con 0x7f0050100e70 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f004c00be50 con 0x7f0050100e70 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005019edc0 con 0x7f0050100e70 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c021410 con 0x7f0050100e70 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.560+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005019f190 con 0x7f0050100e70 2024-05-08T23:17:51.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.561+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0050108300 con 0x7f0050100e70 2024-05-08T23:17:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.562+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f004c00f6d0 con 0x7f0050100e70 2024-05-08T23:17:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.563+0000 7f00477fe700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f003c0a1120 0x7f003c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.563+0000 7f0055d07700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f003c0a1120 0x7f003c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.563+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f004c0cdf20 con 0x7f0050100e70 2024-05-08T23:17:51.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.564+0000 7f0055d07700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f003c0a1120 0x7f003c0a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00501981a0 tx=0x7f0040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:51.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.567+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f004c097550 con 0x7f0050100e70 2024-05-08T23:17:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.787+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0050067380 con 0x7f0050100e70 2024-05-08T23:17:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.789+0000 7f00477fe700 1 -- 172.21.15.16:0/2757750335 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f004c09ab70 con 0x7f0050100e70 2024-05-08T23:17:51.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:51.790 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f003c0a1120 msgr2=0x7f003c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f003c0a1120 0x7f003c0a35e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f00501981a0 tx=0x7f0040009260 comp rx=0 tx=0).stop 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 msgr2=0x7f0050196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050196cf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f004c005740 tx=0x7f004c005770 comp rx=0 tx=0).stop 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 shutdown_connections 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f003c0a1120 0x7f003c0a35e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0050102980 0x7f005019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0050100e70 0x7f0050196cf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 --2- 172.21.15.16:0/2757750335 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0050101fc0 0x7f0050197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 >> 172.21.15.16:0/2757750335 conn(0x7f00500fc400 msgr2=0x7f0050105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 shutdown_connections 2024-05-08T23:17:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.792+0000 7f005876c700 1 -- 172.21.15.16:0/2757750335 wait complete. 2024-05-08T23:17:51.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:17:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 -- 172.21.15.16:0/1081174377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 msgr2=0x7f07bc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/1081174377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc101290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009990 tx=0x7f07ac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 -- 172.21.15.16:0/1081174377 shutdown_connections 2024-05-08T23:17:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/1081174377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07bc102980 0x7f07bc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/1081174377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc101290 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/1081174377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc101fc0 0x7f07bc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 -- 172.21.15.16:0/1081174377 >> 172.21.15.16:0/1081174377 conn(0x7f07bc0fc400 msgr2=0x7f07bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:51.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 -- 172.21.15.16:0/1081174377 shutdown_connections 2024-05-08T23:17:51.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.961+0000 7f07c1ec6700 1 -- 172.21.15.16:0/1081174377 wait complete. 2024-05-08T23:17:51.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 Processor -- start 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 -- start start 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07bc101fc0 0x7f07bc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc102980 0x7f07bc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc074070 con 0x7f07bc100e70 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc0741e0 con 0x7f07bc102980 2024-05-08T23:17:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07c1ec6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1a0d50 con 0x7f07bc101fc0 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07bb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07bb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45134/0 (socket says 172.21.15.16:45134) 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.962+0000 7f07bb7fe700 1 -- 172.21.15.16:0/3862270235 learned_addr learned my addr 172.21.15.16:0/3862270235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07baffd700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07bc101fc0 0x7f07bc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bbfff700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc102980 0x7f07bc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bb7fe700 1 -- 172.21.15.16:0/3862270235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07bc101fc0 msgr2=0x7f07bc073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bb7fe700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07bc101fc0 0x7f07bc073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bb7fe700 1 -- 172.21.15.16:0/3862270235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc102980 msgr2=0x7f07bc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bb7fe700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc102980 0x7f07bc0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bb7fe700 1 -- 172.21.15.16:0/3862270235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b0009520 con 0x7f07bc100e70 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07bb7fe700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc074d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0059d0 tx=0x7f07ac004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac01d070 con 0x7f07bc100e70 2024-05-08T23:17:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f07ac00bdd0 con 0x7f07bc100e70 2024-05-08T23:17:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac0095f0 con 0x7f07bc100e70 2024-05-08T23:17:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac022d40 con 0x7f07bc100e70 2024-05-08T23:17:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.963+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc1a11c0 con 0x7f07bc100e70 2024-05-08T23:17:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.964+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc108380 con 0x7f07bc100e70 2024-05-08T23:17:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.966+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f07ac022500 con 0x7f07bc100e70 2024-05-08T23:17:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.966+0000 7f07b8ff9700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07a40a11a0 0x7f07a40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.966+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f07ac0ce190 con 0x7f07bc100e70 2024-05-08T23:17:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.966+0000 7f07baffd700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07a40a11a0 0x7f07a40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.967+0000 7f07baffd700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07a40a11a0 0x7f07a40a3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009df0 tx=0x7f07b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:51.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:51.970+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f07ac0977c0 con 0x7f07bc100e70 2024-05-08T23:17:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.240+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07bc10c520 con 0x7f07bc100e70 2024-05-08T23:17:52.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.241+0000 7f07b8ff9700 1 -- 172.21.15.16:0/3862270235 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10816 (secure 0 0 0) 0x7f07ac09ade0 con 0x7f07bc100e70 2024-05-08T23:17:52.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:52.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24517,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:17:50.580857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.243+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07a40a11a0 msgr2=0x7f07a40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.243+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07a40a11a0 0x7f07a40a3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009df0 tx=0x7f07b0009260 comp rx=0 tx=0).stop 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.243+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 msgr2=0x7f07bc074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.243+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc074d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0059d0 tx=0x7f07ac004710 comp rx=0 tx=0).stop 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 shutdown_connections 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07bc102980 0x7f07bc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07bc100e70 0x7f07bc074d00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f07a40a11a0 0x7f07a40a3660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 --2- 172.21.15.16:0/3862270235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07bc101fc0 0x7f07bc073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 >> 172.21.15.16:0/3862270235 conn(0x7f07bc0fc400 msgr2=0x7f07bc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 shutdown_connections 2024-05-08T23:17:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.244+0000 7f07c1ec6700 1 -- 172.21.15.16:0/3862270235 wait complete. 2024-05-08T23:17:52.246 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-05-08T23:17:52.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.395+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/488799365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f981006d0 msgr2=0x7f6f98100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.395+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/488799365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f981006d0 0x7f6f98100b30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009990 tx=0x7f6f8c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/488799365 shutdown_connections 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/488799365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f98101070 0x7f6f98107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/488799365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f980ff4e0 0x7f6f980ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/488799365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f981006d0 0x7f6f98100b30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/488799365 >> 172.21.15.16:0/488799365 conn(0x7f6f980faa70 msgr2=0x7f6f980fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/488799365 shutdown_connections 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/488799365 wait complete. 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 Processor -- start 2024-05-08T23:17:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.396+0000 7f6f9dcd6700 1 -- start start 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f9dcd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 0x7f6f98198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f9dcd6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f981006d0 0x7f6f981993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f9dcd6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f98101070 0x7f6f981a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f9dcd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981a09d0 con 0x7f6f980ff4e0 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f9dcd6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981a0b10 con 0x7f6f981006d0 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f9dcd6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981a0ca0 con 0x7f6f98101070 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 0x7f6f98198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 0x7f6f98198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45154/0 (socket says 172.21.15.16:45154) 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 -- 172.21.15.16:0/265212998 learned_addr learned my addr 172.21.15.16:0/265212998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f97fff700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f98101070 0x7f6f981a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f96ffd700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f981006d0 0x7f6f981993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 -- 172.21.15.16:0/265212998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f98101070 msgr2=0x7f6f981a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f98101070 0x7f6f981a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 -- 172.21.15.16:0/265212998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f981006d0 msgr2=0x7f6f981993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f981006d0 0x7f6f981993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 -- 172.21.15.16:0/265212998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c0095f0 con 0x7f6f980ff4e0 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f96ffd700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f981006d0 0x7f6f981993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.397+0000 7f6f977fe700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 0x7f6f98198ea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c990 tx=0x7f6f8800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.398+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88007b00 con 0x7f6f980ff4e0 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.398+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6f88007c60 con 0x7f6f980ff4e0 2024-05-08T23:17:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.398+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f981a0f80 con 0x7f6f980ff4e0 2024-05-08T23:17:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.398+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88018420 con 0x7f6f980ff4e0 2024-05-08T23:17:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.398+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f981a13a0 con 0x7f6f980ff4e0 2024-05-08T23:17:52.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.399+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f980673f0 con 0x7f6f980ff4e0 2024-05-08T23:17:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.400+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6f8800f450 con 0x7f6f980ff4e0 2024-05-08T23:17:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.400+0000 7f6f94ff9700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6f800a1120 0x7f6f800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.401+0000 7f6f96ffd700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6f800a1120 0x7f6f800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.401+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6f880cca40 con 0x7f6f980ff4e0 2024-05-08T23:17:52.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.401+0000 7f6f96ffd700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6f800a1120 0x7f6f800a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006010 tx=0x7f6f8c00b630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.405+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6f980673f0 con 0x7f6f980ff4e0 2024-05-08T23:17:52.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:52 smithi189 ceph-mon[46919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:17:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:52 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1065483052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:52 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2757750335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:52 smithi155 ceph-mon[47105]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:17:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:52 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1065483052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:52 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2757750335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:52.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:52 smithi016 ceph-mon[42541]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T23:17:52.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:52 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1065483052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:52.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:52 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2757750335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.625+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f9810c5c0 con 0x7f6f980ff4e0 2024-05-08T23:17:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.627+0000 7f6f94ff9700 1 -- 172.21.15.16:0/265212998 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f6f880cee60 con 0x7f6f980ff4e0 2024-05-08T23:17:52.628 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:52.629 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.629+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6f800a1120 msgr2=0x7f6f800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.629+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6f800a1120 0x7f6f800a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006010 tx=0x7f6f8c00b630 comp rx=0 tx=0).stop 2024-05-08T23:17:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.629+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 msgr2=0x7f6f98198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.629+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 0x7f6f98198ea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c990 tx=0x7f6f8800cca0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 shutdown_connections 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6f800a1120 0x7f6f800a35e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f98101070 0x7f6f981a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f980ff4e0 0x7f6f98198ea0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 --2- 172.21.15.16:0/265212998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f981006d0 0x7f6f981993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 >> 172.21.15.16:0/265212998 conn(0x7f6f980faa70 msgr2=0x7f6f98108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 shutdown_connections 2024-05-08T23:17:52.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.630+0000 7f6f9dcd6700 1 -- 172.21.15.16:0/265212998 wait complete. 2024-05-08T23:17:52.645 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T23:17:46.078099+0000', 'modified': '2024-05-08T23:17:50.580857+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6839/773665585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 773665585}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 773665585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-08T23:17:52.645 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T23:17:52.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:17:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.784+0000 7f20773cd700 1 -- 172.21.15.16:0/2559451952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20701019e0 msgr2=0x7f2070103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.784+0000 7f20773cd700 1 --2- 172.21.15.16:0/2559451952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20701019e0 0x7f2070103dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f206c009990 tx=0x7f206c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.784+0000 7f20773cd700 1 -- 172.21.15.16:0/2559451952 shutdown_connections 2024-05-08T23:17:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.784+0000 7f20773cd700 1 --2- 172.21.15.16:0/2559451952 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2070106c40 0x7f20701090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.784+0000 7f20773cd700 1 --2- 172.21.15.16:0/2559451952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20701019e0 0x7f2070103dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.784+0000 7f20773cd700 1 --2- 172.21.15.16:0/2559451952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 0x7f2070106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.785+0000 7f20773cd700 1 -- 172.21.15.16:0/2559451952 >> 172.21.15.16:0/2559451952 conn(0x7f20700fb3f0 msgr2=0x7f20700fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.785+0000 7f20773cd700 1 -- 172.21.15.16:0/2559451952 shutdown_connections 2024-05-08T23:17:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.785+0000 7f20773cd700 1 -- 172.21.15.16:0/2559451952 wait complete. 2024-05-08T23:17:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.785+0000 7f20773cd700 1 Processor -- start 2024-05-08T23:17:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 -- start start 2024-05-08T23:17:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20701019e0 0x7f2070196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 0x7f2070197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 0x7f207019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019e970 con 0x7f2070106c40 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019eab0 con 0x7f20701019e0 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f20773cd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207019ec40 con 0x7f2070104310 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 0x7f207019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 0x7f207019e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45176/0 (socket says 172.21.15.16:45176) 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 -- 172.21.15.16:0/2791803443 learned_addr learned my addr 172.21.15.16:0/2791803443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f2075169700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20701019e0 0x7f2070196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f2074968700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 0x7f2070197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 -- 172.21.15.16:0/2791803443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 msgr2=0x7f2070197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 0x7f2070197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 -- 172.21.15.16:0/2791803443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20701019e0 msgr2=0x7f2070196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20701019e0 0x7f2070196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f207596a700 1 -- 172.21.15.16:0/2791803443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0095f0 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.786+0000 7f2074968700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 0x7f2070197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.787+0000 7f207596a700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 0x7f207019e3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f206800ef30 tx=0x7f206800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.787+0000 7f20667fc700 1 -- 172.21.15.16:0/2791803443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068018070 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.787+0000 7f20667fc700 1 -- 172.21.15.16:0/2791803443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f206800f040 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.787+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207019eec0 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.787+0000 7f20667fc700 1 -- 172.21.15.16:0/2791803443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206800cc70 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.787+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207019f390 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.789+0000 7f20667fc700 1 -- 172.21.15.16:0/2791803443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2068007500 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.790+0000 7f20667fc700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f205c0a11a0 0x7f205c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.790+0000 7f20667fc700 1 -- 172.21.15.16:0/2791803443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f20680cc580 con 0x7f2070106c40 2024-05-08T23:17:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.790+0000 7f2075169700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f205c0a11a0 0x7f205c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.790+0000 7f2075169700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f205c0a11a0 0x7f205c0a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f206c005910 tx=0x7f206c00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.791+0000 7f2083587b80 1 Processor -- start 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.791+0000 7f2083587b80 1 -- start start 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2083587b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a7289b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2083587b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 0x560a7289c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2083587b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289c810 0x560a7289cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2083587b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a727f2100 con 0x560a7289be50 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2083587b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a727efac0 con 0x560a7289c810 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2083587b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a727eb3b0 con 0x560a7289b4f0 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 0x560a7289c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 0x560a7289c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45186/0 (socket says 172.21.15.16:45186) 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 -- 172.21.15.16:0/3457767222 learned_addr learned my addr 172.21.15.16:0/3457767222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2075169700 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a7289b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f207596a700 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289c810 0x560a7289cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 -- 172.21.15.16:0/3457767222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 msgr2=0x560a7289b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a7289b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 -- 172.21.15.16:0/3457767222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289c810 msgr2=0x560a7289cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289c810 0x560a7289cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 -- 172.21.15.16:0/3457767222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a7266c360 con 0x560a7289be50 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f207596a700 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289c810 0x560a7289cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f2074968700 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 0x560a7289c2d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2060009de0 tx=0x7f20600116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c01af43752d98fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.792+0000 7f205b7fe700 1 -- 172.21.15.16:0/3457767222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060011e20 con 0x560a7289be50 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f205b7fe700 1 -- 172.21.15.16:0/3457767222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20600128e0 con 0x560a7289be50 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f205b7fe700 1 -- 172.21.15.16:0/3457767222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206000f7c0 con 0x560a7289be50 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 -- 172.21.15.16:0/3457767222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 msgr2=0x560a7289c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 0x560a7289c2d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2060009de0 tx=0x7f20600116c0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 -- 172.21.15.16:0/3457767222 shutdown_connections 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289c810 0x560a7289cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a7289b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 --2- 172.21.15.16:0/3457767222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289be50 0x560a7289c2d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 -- 172.21.15.16:0/3457767222 >> 172.21.15.16:0/3457767222 conn(0x560a727f91c0 msgr2=0x560a72895d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 -- 172.21.15.16:0/3457767222 shutdown_connections 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.793+0000 7f2083587b80 1 -- 172.21.15.16:0/3457767222 wait complete. 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 -- start start 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 Processor -- start 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289b4f0 0x560a728a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289be50 0x560a728a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a72787510 con 0x560a7289c810 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a727871b0 con 0x560a7289b4f0 2024-05-08T23:17:52.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a7266e650 con 0x560a7289be50 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f2083587b80 10 client.0 _ll_register_callbacks cb 0x560a72790ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f207596a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f207596a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45188/0 (socket says 172.21.15.16:45188) 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.794+0000 7f207596a700 1 -- 172.21.15.16:0/3313731433 learned_addr learned my addr 172.21.15.16:0/3313731433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289b4f0 0x560a728a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2074968700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289be50 0x560a728a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 -- 172.21.15.16:0/3313731433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289be50 msgr2=0x560a728a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289be50 0x560a728a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 -- 172.21.15.16:0/3313731433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 msgr2=0x560a728ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 -- 172.21.15.16:0/3313731433 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2060009a10 con 0x560a7289b4f0 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f207596a700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2075169700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289b4f0 0x560a728a4d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f206c003f60 tx=0x7f206c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2364b75644eea876 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2058ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2058ff9700 1 -- 172.21.15.16:0/3313731433 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c034070 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.795+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a72744220 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2058ff9700 1 -- 172.21.15.16:0/3313731433 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f206c00bd00 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2058ff9700 1 -- 172.21.15.16:0/3313731433 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f206c003d70 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 10 client.24662 fetch_fsmap learned FSMap version 17 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x560a72618b10 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2058ff9700 1 -- 172.21.15.16:0/3313731433 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560a72744220 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2058ff9700 1 -- 172.21.15.16:0/3313731433 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f206c037020 con 0x560a7289b4f0 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 10 client.24662 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 10 client.24662 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 20 client.24662 populate_metadata read hostname 'smithi016' 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a728afff0 0x560a728b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 4 client.24662 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:17:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a7286a7c0 con 0x560a728afff0 2024-05-08T23:17:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.796+0000 7f2074968700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a728afff0 0x560a728b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.797+0000 7f2074968700 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a728afff0 0x560a728b5310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5868f55b466154 server_cookie=632c92ac79f25956 in_seq=0 out_seq=0 2024-05-08T23:17:52.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.797+0000 7f2058ff9700 10 client.24662 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2058ff9700 1 -- 172.21.15.16:0/3313731433 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a7286a7c0 con 0x560a728afff0 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2058ff9700 10 client.24662 handle_command_reply: tid=0 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 client.24662 shutdown 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289b4f0 msgr2=0x560a728a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289b4f0 0x560a728a4d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f206c003f60 tx=0x7f206c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 shutdown_connections 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a728afff0 0x560a728b5310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289b4f0 0x560a728a4d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 --2- 172.21.15.16:0/3313731433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289be50 0x560a728a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2058ff9700 0 client.24662 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:17:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 >> 172.21.15.16:0/3313731433 conn(0x560a727f91c0 msgr2=0x560a72895550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 shutdown_connections 2024-05-08T23:17:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 1 -- 172.21.15.16:0/3313731433 wait complete. 2024-05-08T23:17:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.798+0000 7f2083587b80 20 client.24662 trim_cache size 0 max 16384 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.817+0000 7f2083587b80 1 Processor -- start 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.817+0000 7f2083587b80 1 -- start start 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.817+0000 7f2083587b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 0x560a7289b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.817+0000 7f2083587b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.817+0000 7f2083587b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289c810 0x560a728a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.817+0000 7f2083587b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a727e8010 con 0x560a7289b4f0 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2083587b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a7278a5a0 con 0x560a7289be50 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2083587b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a728ad490 con 0x560a7289c810 2024-05-08T23:17:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 0x560a7289b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 0x560a7289b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45192/0 (socket says 172.21.15.16:45192) 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 -- 172.21.15.16:0/69815328 learned_addr learned my addr 172.21.15.16:0/69815328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f207596a700 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289c810 0x560a728a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 -- 172.21.15.16:0/69815328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289c810 msgr2=0x560a728a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289c810 0x560a728a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 -- 172.21.15.16:0/69815328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 msgr2=0x560a728ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 -- 172.21.15.16:0/69815328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a727b1510 con 0x560a7289b4f0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f2075169700 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 0x560a7289b8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f206c00b9f0 tx=0x7f206c00ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb39f76e4861deae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f205bfff700 1 -- 172.21.15.16:0/69815328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c034070 con 0x560a7289b4f0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f205bfff700 1 -- 172.21.15.16:0/69815328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f206c028040 con 0x560a7289b4f0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.818+0000 7f205bfff700 1 -- 172.21.15.16:0/69815328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c00bd00 con 0x560a7289b4f0 2024-05-08T23:17:52.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 -- 172.21.15.16:0/69815328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 msgr2=0x560a7289b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 0x560a7289b8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f206c00b9f0 tx=0x7f206c00ba20 comp rx=0 tx=0).stop 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 -- 172.21.15.16:0/69815328 shutdown_connections 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289c810 0x560a728a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289b4f0 0x560a7289b8d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 --2- 172.21.15.16:0/69815328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 -- 172.21.15.16:0/69815328 >> 172.21.15.16:0/69815328 conn(0x560a727f91c0 msgr2=0x560a728921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 -- 172.21.15.16:0/69815328 shutdown_connections 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 -- 172.21.15.16:0/69815328 wait complete. 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 -- start start 2024-05-08T23:17:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.819+0000 7f2083587b80 1 Processor -- start 2024-05-08T23:17:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a728a6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a728ab930 con 0x560a7289c810 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a728abaa0 con 0x560a7289be50 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a728abc60 con 0x560a7289b4f0 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2083587b80 10 client.0 _ll_register_callbacks cb 0x560a72790ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45208/0 (socket says 172.21.15.16:45208) 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 -- 172.21.15.16:0/1400130578 learned_addr learned my addr 172.21.15.16:0/1400130578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2075169700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a728a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f2074968700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 -- 172.21.15.16:0/1400130578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 msgr2=0x560a728a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a728a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 -- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 msgr2=0x560a728a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.820+0000 7f207596a700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f207596a700 1 -- 172.21.15.16:0/1400130578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a72618b10 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2074968700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2075169700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a728a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f207596a700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ab3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20680c9e60 tx=0x7f2068099db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd24fd69da61f1f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2058ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2058ff9700 1 -- 172.21.15.16:0/1400130578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068099f70 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20600128e0 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2058ff9700 1 -- 172.21.15.16:0/1400130578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2068099870 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2058ff9700 1 -- 172.21.15.16:0/1400130578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f206809d6a0 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 10 client.14757 fetch_fsmap learned FSMap version 17 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x560a72670c80 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2058ff9700 1 -- 172.21.15.16:0/1400130578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20600128e0 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2058ff9700 1 -- 172.21.15.16:0/1400130578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f20680a1860 con 0x560a7289c810 2024-05-08T23:17:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 20 client.14757 populate_metadata read hostname 'smithi016' 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a72913900 0x560a729102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 4 client.14757 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.821+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a727e3e70 con 0x560a72913900 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.822+0000 7f2075169700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a72913900 0x560a729102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.822+0000 7f2058ff9700 10 client.14757 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.822+0000 7f2075169700 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a72913900 0x560a729102b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcbd00a885fbdd1e server_cookie=495ea7fe2554db47 in_seq=0 out_seq=0 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2058ff9700 1 -- 172.21.15.16:0/1400130578 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x560a727e3e70 con 0x560a72913900 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2058ff9700 10 client.14757 handle_command_reply: tid=0 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 client.14757 shutdown 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 msgr2=0x560a728ab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ab3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20680c9e60 tx=0x7f2068099db0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 shutdown_connections 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a7289c810 0x560a728ab3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a7289b4f0 0x560a728a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560a72913900 0x560a729102b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 --2- 172.21.15.16:0/1400130578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560a7289be50 0x560a728a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2058ff9700 0 client.14757 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 >> 172.21.15.16:0/1400130578 conn(0x560a727f91c0 msgr2=0x560a728a42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 shutdown_connections 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 1 -- 172.21.15.16:0/1400130578 wait complete. 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f2083587b80 20 client.14757 trim_cache size 0 max 16384 2024-05-08T23:17:52.830 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24563, 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.7425152969999997, 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 81, 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 80, 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 12.771598894 2024-05-08T23:17:52.831 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:17:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f205c0a11a0 msgr2=0x7f205c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.823+0000 7f20773cd700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f205c0a11a0 0x7f205c0a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f206c005910 tx=0x7f206c00b610 comp rx=0 tx=0).stop 2024-05-08T23:17:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 msgr2=0x7f207019e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 0x7f207019e3a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f206800ef30 tx=0x7f206800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 shutdown_connections 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2070106c40 0x7f207019e3a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20701019e0 0x7f2070196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f205c0a11a0 0x7f205c0a3660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 --2- 172.21.15.16:0/2791803443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2070104310 0x7f2070197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 >> 172.21.15.16:0/2791803443 conn(0x7f20700fb3f0 msgr2=0x7f2070103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 shutdown_connections 2024-05-08T23:17:52.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:52.824+0000 7f20773cd700 1 -- 172.21.15.16:0/2791803443 wait complete. 2024-05-08T23:17:52.838 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:17:52.838 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:52.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T23:17:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:17:53.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:53 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3862270235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:53.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:53 smithi016 ceph-mon[42541]: pgmap v226: 129 pgs: 7 creating+peering, 111 active+clean, 11 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 687 B/s wr, 2 op/s 2024-05-08T23:17:53.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:53 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/265212998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:53.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:53 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3862270235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:53.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:53 smithi189 ceph-mon[46919]: pgmap v226: 129 pgs: 7 creating+peering, 111 active+clean, 11 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 687 B/s wr, 2 op/s 2024-05-08T23:17:53.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:53 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/265212998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:53 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3862270235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:17:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:53 smithi155 ceph-mon[47105]: pgmap v226: 129 pgs: 7 creating+peering, 111 active+clean, 11 unknown; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 687 B/s wr, 2 op/s 2024-05-08T23:17:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:53 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/265212998' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 -- 172.21.15.16:0/3726136391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd680068950 msgr2=0x7fd680068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 --2- 172.21.15.16:0/3726136391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd680068950 0x7fd680068db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd674009990 tx=0x7fd674009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 -- 172.21.15.16:0/3726136391 shutdown_connections 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 --2- 172.21.15.16:0/3726136391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6800692f0 0x7fd68010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 --2- 172.21.15.16:0/3726136391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680107020 0x7fd680107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 --2- 172.21.15.16:0/3726136391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd680068950 0x7fd680068db0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 -- 172.21.15.16:0/3726136391 >> 172.21.15.16:0/3726136391 conn(0x7fd680075f80 msgr2=0x7fd680076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 -- 172.21.15.16:0/3726136391 shutdown_connections 2024-05-08T23:17:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 -- 172.21.15.16:0/3726136391 wait complete. 2024-05-08T23:17:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.111+0000 7fd685fa6700 1 Processor -- start 2024-05-08T23:17:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 -- start start 2024-05-08T23:17:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680068950 0x7fd680199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 0x7fd680199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd680107020 0x7fd68019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68019e4d0 con 0x7fd6800692f0 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68019e610 con 0x7fd680107020 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd685fa6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68019e7d0 con 0x7fd680068950 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd67effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 0x7fd680199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd67f7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680068950 0x7fd680199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd67effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 0x7fd680199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45232/0 (socket says 172.21.15.16:45232) 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd67effd700 1 -- 172.21.15.16:0/563739881 learned_addr learned my addr 172.21.15.16:0/563739881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.112+0000 7fd67ffff700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd680107020 0x7fd68019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67effd700 1 -- 172.21.15.16:0/563739881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680068950 msgr2=0x7fd680199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67effd700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680068950 0x7fd680199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67effd700 1 -- 172.21.15.16:0/563739881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd680107020 msgr2=0x7fd68019dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67effd700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd680107020 0x7fd68019dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67effd700 1 -- 172.21.15.16:0/563739881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740095f0 con 0x7fd6800692f0 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67f7fe700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680068950 0x7fd680199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67effd700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 0x7fd680199bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd674004730 tx=0x7fd674004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67401d070 con 0x7fd6800692f0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd67400be50 con 0x7fd6800692f0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68019ea50 con 0x7fd6800692f0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674022e00 con 0x7fd6800692f0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.113+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6801a6340 con 0x7fd6800692f0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.115+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6801026d0 con 0x7fd6800692f0 2024-05-08T23:17:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.116+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd67400fac0 con 0x7fd6800692f0 2024-05-08T23:17:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.116+0000 7fd67cff9700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd6680a11a0 0x7fd6680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.116+0000 7fd67f7fe700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd6680a11a0 0x7fd6680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.116+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd6740cddc0 con 0x7fd6800692f0 2024-05-08T23:17:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.117+0000 7fd67f7fe700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd6680a11a0 0x7fd6680a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fd0 tx=0x7fd670009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:54.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.121+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd674097510 con 0x7fd6800692f0 2024-05-08T23:17:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.383+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd68019a8e0 con 0x7fd6800692f0 2024-05-08T23:17:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.383+0000 7fd67cff9700 1 -- 172.21.15.16:0/563739881 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd674027070 con 0x7fd6800692f0 2024-05-08T23:17:54.384 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T23:17:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd6680a11a0 msgr2=0x7fd6680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd6680a11a0 0x7fd6680a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd670009fd0 tx=0x7fd670009260 comp rx=0 tx=0).stop 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 msgr2=0x7fd680199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 0x7fd680199bb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd674004730 tx=0x7fd674004760 comp rx=0 tx=0).stop 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 shutdown_connections 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd680107020 0x7fd68019dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd6680a11a0 0x7fd6680a3660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd680068950 0x7fd680199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 --2- 172.21.15.16:0/563739881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6800692f0 0x7fd680199bb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 >> 172.21.15.16:0/563739881 conn(0x7fd680075f80 msgr2=0x7fd6800feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 shutdown_connections 2024-05-08T23:17:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:54.386+0000 7fd685fa6700 1 -- 172.21.15.16:0/563739881 wait complete. 2024-05-08T23:17:54.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:54 smithi016 ceph-mon[42541]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:17:54.408 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:54 smithi016 ceph-mon[42541]: Cluster is now healthy 2024-05-08T23:17:54.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:54 smithi189 ceph-mon[46919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:17:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:54 smithi189 ceph-mon[46919]: Cluster is now healthy 2024-05-08T23:17:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:54 smithi155 ceph-mon[47105]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T23:17:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:54 smithi155 ceph-mon[47105]: Cluster is now healthy 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:17:54.760 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T23:17:54.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:54.760 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-05-08T23:17:54.783 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:17:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:17:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:17:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:17:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-05-08T23:17:54.783 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41782sec preferred_lft 41782sec 2024-05-08T23:17:54.784 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-05-08T23:17:54.785 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:17:54.785 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:17:54.785 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:17:54.785 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-05-08T23:17:54.786 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:17:54.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:54.786 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:17:54.786 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:17:54.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:17:54.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-05-08T23:17:54.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-05-08T23:17:54.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:17:54.787 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:17:54.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:54.980 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:17:55.014 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-05-08T23:17:55.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.016 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-05-08T23:17:55.095 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-05-08T23:17:55.096 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:17:55.096 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:17:55.096 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:17:55.096 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:17:55.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.096 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:17:55.097 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:17:55.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:17:55.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:17:55.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:17:55.097 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:17:55.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:55 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/563739881' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:17:55.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:55 smithi016 ceph-mon[42541]: pgmap v227: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-08T23:17:55.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.338 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-08T23:17:55.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.359 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-05-08T23:17:55.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:55 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/563739881' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:17:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:55 smithi189 ceph-mon[46919]: pgmap v227: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-08T23:17:55.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.421 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:17:55.422 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:17:55.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T23:17:55.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T23:17:55.422 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:17:55.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:55 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/563739881' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:17:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:55 smithi155 ceph-mon[47105]: pgmap v227: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-08T23:17:55.534 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T23:17:55.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.534 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:17:55.534 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:17:55.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T23:17:55.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:17:55.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T23:17:55.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T23:17:55.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T23:17:55.535 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:17:55.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:55.765 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:17:55.765 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:17:55.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-05-08T23:17:55.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:17:55.765 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:17:55.880 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T23:17:55.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:17:55.880 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:55.900 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T23:17:55.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:17:55.900 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:55.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:17:56.037 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:17:56.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:17:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:17:56.115 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:17:56.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:56.116 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:17:56.178 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:17:56.178 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T23:17:56.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:17:56.259 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:17:56.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:17:56.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-05-08T23:17:56.328+0000 7efc479625c0 -1 init, newargv = 0x558646cea8d0 newargc=15 2024-05-08T23:17:56.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[98366]: starting ceph client 2024-05-08T23:17:56.332 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:17:56.333 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:17:56.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:56.334 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:17:56.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[98366]: starting fuse 2024-05-08T23:17:56.357 INFO:teuthology.orchestra.run.smithi016.stdout:124 2024-05-08T23:17:56.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-08T23:17:56.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:17:56.357 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-05-08T23:17:56.357 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-05-08T23:17:56.357 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-05-08T23:17:56.357 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-05-08T23:17:56.357 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:17:56.358 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-05-08T23:17:56.359 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-05-08T23:17:56.359 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-05-08T23:17:56.359 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:17:56.359 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:17:56.359 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-05-08T23:17:56.359 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-05-08T23:17:56.496 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.98366.asok 2024-05-08T23:17:56.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.98366.asok 2024-05-08T23:17:56.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:56.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.98366.asok status 2024-05-08T23:17:56.856 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "98366", 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24674, 2024-05-08T23:17:56.857 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 24674 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2013627759 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:17:56.858 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2013627759 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.24674 192.168.0.1:0/2013627759", 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2013627759", 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 17, 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 81, 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-05-08T23:17:56.859 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:17:56.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:56.871 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-05-08T23:17:56.871 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:56.892 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-05-08T23:17:56.892 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:56.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:56.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:17:56.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:17:56.967 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T23:17:57.115 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-05-08T23:17:57.115 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T23:17:57.115 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:57.115 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:17:57.115 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:17:57.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 -- 172.21.15.16:0/3880654548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c102990 msgr2=0x7f1d9c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 --2- 172.21.15.16:0/3880654548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c102990 0x7f1d9c1092b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b0a0 tx=0x7f1d9400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 -- 172.21.15.16:0/3880654548 shutdown_connections 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 --2- 172.21.15.16:0/3880654548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c102990 0x7f1d9c1092b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 --2- 172.21.15.16:0/3880654548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c100e80 0x7f1d9c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 --2- 172.21.15.16:0/3880654548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d9c101fd0 0x7f1d9c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 -- 172.21.15.16:0/3880654548 >> 172.21.15.16:0/3880654548 conn(0x7f1d9c0fc410 msgr2=0x7f1d9c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:57.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 -- 172.21.15.16:0/3880654548 shutdown_connections 2024-05-08T23:17:57.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.326+0000 7f1da0e39700 1 -- 172.21.15.16:0/3880654548 wait complete. 2024-05-08T23:17:57.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 Processor -- start 2024-05-08T23:17:57.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 -- start start 2024-05-08T23:17:57.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 0x7f1d9c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d9c101fd0 0x7f1d9c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c102990 0x7f1d9c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c074070 con 0x7f1d9c100e80 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0741e0 con 0x7f1d9c101fd0 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1da0e39700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c1a0e20 con 0x7f1d9c102990 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1d9a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 0x7f1d9c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1d9a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 0x7f1d9c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52804/0 (socket says 172.21.15.16:52804) 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1d9a59c700 1 -- 172.21.15.16:0/52223494 learned_addr learned my addr 172.21.15.16:0/52223494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1d99d9b700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d9c101fd0 0x7f1d9c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.327+0000 7f1d9ad9d700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c102990 0x7f1d9c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9a59c700 1 -- 172.21.15.16:0/52223494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c102990 msgr2=0x7f1d9c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9a59c700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c102990 0x7f1d9c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9a59c700 1 -- 172.21.15.16:0/52223494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d9c101fd0 msgr2=0x7f1d9c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9a59c700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d9c101fd0 0x7f1d9c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9a59c700 1 -- 172.21.15.16:0/52223494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94009d00 con 0x7f1d9c100e80 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9ad9d700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c102990 0x7f1d9c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:57.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d9a59c700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 0x7f1d9c074d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e9a0 tx=0x7f1d8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84009790 con 0x7f1d9c100e80 2024-05-08T23:17:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c1a10b0 con 0x7f1d9c100e80 2024-05-08T23:17:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d84010430 con 0x7f1d9c100e80 2024-05-08T23:17:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84018680 con 0x7f1d9c100e80 2024-05-08T23:17:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.328+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c1a15c0 con 0x7f1d9c100e80 2024-05-08T23:17:57.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.329+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c1082f0 con 0x7f1d9c100e80 2024-05-08T23:17:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.330+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1d840105a0 con 0x7f1d9c100e80 2024-05-08T23:17:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.331+0000 7f1d937fe700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1d880a1120 0x7f1d880a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.332+0000 7f1d99d9b700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1d880a1120 0x7f1d880a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.332+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d84014070 con 0x7f1d9c100e80 2024-05-08T23:17:57.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.332+0000 7f1d99d9b700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1d880a1120 0x7f1d880a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009c60 tx=0x7f1d8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:57.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.335+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1d84096520 con 0x7f1d9c100e80 2024-05-08T23:17:57.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.553+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d9c10c530 con 0x7f1d9c100e80 2024-05-08T23:17:57.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.554+0000 7f1d937fe700 1 -- 172.21.15.16:0/52223494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f1d84026090 con 0x7f1d9c100e80 2024-05-08T23:17:57.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:57.555 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:57.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1d880a1120 msgr2=0x7f1d880a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1d880a1120 0x7f1d880a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009c60 tx=0x7f1d8c009260 comp rx=0 tx=0).stop 2024-05-08T23:17:57.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 msgr2=0x7f1d9c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 0x7f1d9c074d00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e9a0 tx=0x7f1d8400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 shutdown_connections 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1d880a1120 0x7f1d880a35e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d9c102990 0x7f1d9c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d9c100e80 0x7f1d9c074d00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 --2- 172.21.15.16:0/52223494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d9c101fd0 0x7f1d9c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 >> 172.21.15.16:0/52223494 conn(0x7f1d9c0fc410 msgr2=0x7f1d9c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 shutdown_connections 2024-05-08T23:17:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.557+0000 7f1da0e39700 1 -- 172.21.15.16:0/52223494 wait complete. 2024-05-08T23:17:57.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:17:57.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:17:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 -- 172.21.15.16:0/1309210636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102b10 msgr2=0x7fe6541093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 --2- 172.21.15.16:0/1309210636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102b10 0x7fe6541093c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe64c00b0d0 tx=0x7fe64c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 -- 172.21.15.16:0/1309210636 shutdown_connections 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 --2- 172.21.15.16:0/1309210636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102b10 0x7fe6541093c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 --2- 172.21.15.16:0/1309210636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe654100f80 0x7fe6541013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 --2- 172.21.15.16:0/1309210636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102170 0x7fe6541025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.713+0000 7fe659441700 1 -- 172.21.15.16:0/1309210636 >> 172.21.15.16:0/1309210636 conn(0x7fe6540fc4f0 msgr2=0x7fe6540fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.714+0000 7fe659441700 1 -- 172.21.15.16:0/1309210636 shutdown_connections 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.714+0000 7fe659441700 1 -- 172.21.15.16:0/1309210636 wait complete. 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.714+0000 7fe659441700 1 Processor -- start 2024-05-08T23:17:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.714+0000 7fe659441700 1 -- start start 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe654100f80 0x7fe654196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 0x7fe654197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102b10 0x7fe65419e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65419ea40 con 0x7fe654102170 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65419eb80 con 0x7fe654102b10 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65419ed10 con 0x7fe654100f80 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 0x7fe654197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 0x7fe654197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52824/0 (socket says 172.21.15.16:52824) 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 -- 172.21.15.16:0/363200485 learned_addr learned my addr 172.21.15.16:0/363200485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe652ffd700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe654100f80 0x7fe654196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6537fe700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102b10 0x7fe65419e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 -- 172.21.15.16:0/363200485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe654100f80 msgr2=0x7fe654196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe654100f80 0x7fe654196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 -- 172.21.15.16:0/363200485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102b10 msgr2=0x7fe65419e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102b10 0x7fe65419e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 -- 172.21.15.16:0/363200485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c009d00 con 0x7fe654102170 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6537fe700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102b10 0x7fe65419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe6527fc700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 0x7fe654197450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c920 tx=0x7fe64400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644007b00 con 0x7fe654102170 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe64400ce70 con 0x7fe654102170 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644005440 con 0x7fe654102170 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.715+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65419eff0 con 0x7fe654102170 2024-05-08T23:17:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.716+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65419f420 con 0x7fe654102170 2024-05-08T23:17:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.717+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6541083b0 con 0x7fe654102170 2024-05-08T23:17:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.718+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe6440055a0 con 0x7fe654102170 2024-05-08T23:17:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.718+0000 7fe64bfff700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe6400a1120 0x7fe6400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.719+0000 7fe652ffd700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe6400a1120 0x7fe6400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.719+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe6440cc6a0 con 0x7fe654102170 2024-05-08T23:17:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.719+0000 7fe652ffd700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe6400a1120 0x7fe6400a35e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe654101fd0 tx=0x7fe63c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.723+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe644095c40 con 0x7fe654102170 2024-05-08T23:17:57.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:57 smithi016 ceph-mon[42541]: pgmap v228: 129 pgs: 129 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 137 B/s wr, 0 op/s 2024-05-08T23:17:57.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:57 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/52223494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:57.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:57 smithi189 ceph-mon[46919]: pgmap v228: 129 pgs: 129 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 137 B/s wr, 0 op/s 2024-05-08T23:17:57.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:57 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/52223494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.943+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe65402ce40 con 0x7fe654102170 2024-05-08T23:17:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.944+0000 7fe64bfff700 1 -- 172.21.15.16:0/363200485 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe644099260 con 0x7fe654102170 2024-05-08T23:17:57.944 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe6400a1120 msgr2=0x7fe6400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe6400a1120 0x7fe6400a35e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe654101fd0 tx=0x7fe63c009260 comp rx=0 tx=0).stop 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 msgr2=0x7fe654197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 0x7fe654197450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c920 tx=0x7fe64400cc30 comp rx=0 tx=0).stop 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 shutdown_connections 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe654102b10 0x7fe65419e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe6400a1120 0x7fe6400a35e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe654100f80 0x7fe654196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 --2- 172.21.15.16:0/363200485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe654102170 0x7fe654197450 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 >> 172.21.15.16:0/363200485 conn(0x7fe6540fc4f0 msgr2=0x7fe6541054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 shutdown_connections 2024-05-08T23:17:57.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:57.946+0000 7fe659441700 1 -- 172.21.15.16:0/363200485 wait complete. 2024-05-08T23:17:57.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:17:57.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:57 smithi155 ceph-mon[47105]: pgmap v228: 129 pgs: 129 active+clean; 579 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 137 B/s wr, 0 op/s 2024-05-08T23:17:57.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:57 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/52223494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:17:58.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 -- 172.21.15.16:0/1525292001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a4100fb0 msgr2=0x7ff1a4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/1525292001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a4100fb0 0x7ff1a4101430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff198009990 tx=0x7ff198009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 -- 172.21.15.16:0/1525292001 shutdown_connections 2024-05-08T23:17:58.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/1525292001 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 0x7ff1a4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/1525292001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/1525292001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a4100fb0 0x7ff1a4101430 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 -- 172.21.15.16:0/1525292001 >> 172.21.15.16:0/1525292001 conn(0x7ff1a40fb3f0 msgr2=0x7ff1a40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 -- 172.21.15.16:0/1525292001 shutdown_connections 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.101+0000 7ff1aa73a700 1 -- 172.21.15.16:0/1525292001 wait complete. 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 Processor -- start 2024-05-08T23:17:58.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 -- start start 2024-05-08T23:17:58.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1a4100fb0 0x7ff1a4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 0x7ff1a41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41a0b30 con 0x7ff1a40ffe60 2024-05-08T23:17:58.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41a0c70 con 0x7ff1a4101970 2024-05-08T23:17:58.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1aa73a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a41a0e00 con 0x7ff1a4100fb0 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52832/0 (socket says 172.21.15.16:52832) 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 -- 172.21.15.16:0/2745852987 learned_addr learned my addr 172.21.15.16:0/2745852987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a37fe700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1a4100fb0 0x7ff1a4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a8cd7700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 0x7ff1a41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 -- 172.21.15.16:0/2745852987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1a4100fb0 msgr2=0x7ff1a4199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1a4100fb0 0x7ff1a4199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 -- 172.21.15.16:0/2745852987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 msgr2=0x7ff1a41a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 0x7ff1a41a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.102+0000 7ff1a3fff700 1 -- 172.21.15.16:0/2745852987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980095f0 con 0x7ff1a40ffe60 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.103+0000 7ff1a8cd7700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 0x7ff1a41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:17:58.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.103+0000 7ff1a3fff700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4199000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff19400e9a0 tx=0x7ff19400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.103+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194009790 con 0x7ff1a40ffe60 2024-05-08T23:17:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.103+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a41a10e0 con 0x7ff1a40ffe60 2024-05-08T23:17:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.103+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff19400cbd0 con 0x7ff1a40ffe60 2024-05-08T23:17:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.103+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a41a1640 con 0x7ff1a40ffe60 2024-05-08T23:17:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.104+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194018620 con 0x7ff1a40ffe60 2024-05-08T23:17:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.104+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4067cf0 con 0x7ff1a40ffe60 2024-05-08T23:17:58.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.105+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff19400cd40 con 0x7ff1a40ffe60 2024-05-08T23:17:58.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.106+0000 7ff1a17fa700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff18c0a1120 0x7ff18c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.106+0000 7ff1a37fe700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff18c0a1120 0x7ff18c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.106+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff194014070 con 0x7ff1a40ffe60 2024-05-08T23:17:58.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.106+0000 7ff1a37fe700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff18c0a1120 0x7ff18c0a35e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff19800b610 tx=0x7ff19800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:58.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.109+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff194096630 con 0x7ff1a40ffe60 2024-05-08T23:17:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.330+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1a410b780 con 0x7ff1a40ffe60 2024-05-08T23:17:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.331+0000 7ff1a17fa700 1 -- 172.21.15.16:0/2745852987 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff194099c50 con 0x7ff1a40ffe60 2024-05-08T23:17:58.331 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:17:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff18c0a1120 msgr2=0x7ff18c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff18c0a1120 0x7ff18c0a35e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff19800b610 tx=0x7ff19800b260 comp rx=0 tx=0).stop 2024-05-08T23:17:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 msgr2=0x7ff1a4199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4199000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff19400e9a0 tx=0x7ff19400ed60 comp rx=0 tx=0).stop 2024-05-08T23:17:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 shutdown_connections 2024-05-08T23:17:58.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1a4101970 0x7ff1a41a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1a40ffe60 0x7ff1a4199000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff18c0a1120 0x7ff18c0a35e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 --2- 172.21.15.16:0/2745852987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1a4100fb0 0x7ff1a4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 >> 172.21.15.16:0/2745852987 conn(0x7ff1a40fb3f0 msgr2=0x7ff1a40680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 shutdown_connections 2024-05-08T23:17:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.333+0000 7ff1aa73a700 1 -- 172.21.15.16:0/2745852987 wait complete. 2024-05-08T23:17:58.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:17:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 -- 172.21.15.16:0/1450283914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 msgr2=0x7f66fc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 --2- 172.21.15.16:0/1450283914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc1082b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f66f400b0a0 tx=0x7f66f400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 -- 172.21.15.16:0/1450283914 shutdown_connections 2024-05-08T23:17:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 --2- 172.21.15.16:0/1450283914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc1082b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 --2- 172.21.15.16:0/1450283914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc0ffe80 0x7f66fc1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 --2- 172.21.15.16:0/1450283914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc100fd0 0x7f66fc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 -- 172.21.15.16:0/1450283914 >> 172.21.15.16:0/1450283914 conn(0x7f66fc0fb3f0 msgr2=0x7f66fc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 -- 172.21.15.16:0/1450283914 shutdown_connections 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.488+0000 7f67036b7700 1 -- 172.21.15.16:0/1450283914 wait complete. 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 Processor -- start 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 -- start start 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc0ffe80 0x7f66fc196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc100fd0 0x7f66fc197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc19e9f0 con 0x7f66fc101990 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc19eb30 con 0x7f66fc100fd0 2024-05-08T23:17:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f67036b7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc19ecc0 con 0x7f66fc0ffe80 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52852/0 (socket says 172.21.15.16:52852) 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 -- 172.21.15.16:0/1242559661 learned_addr learned my addr 172.21.15.16:0/1242559661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6700c52700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc100fd0 0x7f66fc197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701453700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc0ffe80 0x7f66fc196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 -- 172.21.15.16:0/1242559661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc0ffe80 msgr2=0x7f66fc196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc0ffe80 0x7f66fc196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 -- 172.21.15.16:0/1242559661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc100fd0 msgr2=0x7f66fc197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc100fd0 0x7f66fc197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6701c54700 1 -- 172.21.15.16:0/1242559661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4009d00 con 0x7f66fc101990 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.489+0000 7f6700c52700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc100fd0 0x7f66fc197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.490+0000 7f6701c54700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc19e420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009570 tx=0x7f66f40095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.490+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400e040 con 0x7f66fc101990 2024-05-08T23:17:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.490+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66f401d470 con 0x7f66fc101990 2024-05-08T23:17:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.490+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc19ef40 con 0x7f66fc101990 2024-05-08T23:17:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.490+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40052a0 con 0x7f66fc101990 2024-05-08T23:17:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.490+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc19f370 con 0x7f66fc101990 2024-05-08T23:17:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.491+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc1072f0 con 0x7f66fc101990 2024-05-08T23:17:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.492+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f66f401d5e0 con 0x7f66fc101990 2024-05-08T23:17:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.492+0000 7f66f27fc700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f66e80a11a0 0x7f66e80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.493+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f66f40ce680 con 0x7f66fc101990 2024-05-08T23:17:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.493+0000 7f6701453700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f66e80a11a0 0x7f66e80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.493+0000 7f6701453700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f66e80a11a0 0x7f66e80a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009c60 tx=0x7f66f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.496+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f66f4097cb0 con 0x7f66fc101990 2024-05-08T23:17:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.720+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66fc10c680 con 0x7f66fc101990 2024-05-08T23:17:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.721+0000 7f66f27fc700 1 -- 172.21.15.16:0/1242559661 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66f409b2d0 con 0x7f66fc101990 2024-05-08T23:17:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f66e80a11a0 msgr2=0x7f66e80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f66e80a11a0 0x7f66e80a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009c60 tx=0x7f66f8009260 comp rx=0 tx=0).stop 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 msgr2=0x7f66fc19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc19e420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009570 tx=0x7f66f40095a0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 shutdown_connections 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc101990 0x7f66fc19e420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f66e80a11a0 0x7f66e80a3660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66fc0ffe80 0x7f66fc196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 --2- 172.21.15.16:0/1242559661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66fc100fd0 0x7f66fc197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 >> 172.21.15.16:0/1242559661 conn(0x7f66fc0fb3f0 msgr2=0x7f66fc104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 shutdown_connections 2024-05-08T23:17:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.723+0000 7f67036b7700 1 -- 172.21.15.16:0/1242559661 wait complete. 2024-05-08T23:17:58.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1345509350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc100fb0 msgr2=0x7f98bc101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1345509350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc100fb0 0x7f98bc101430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009990 tx=0x7f98a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1345509350 shutdown_connections 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1345509350 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc101970 0x7f98bc108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1345509350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98bc0ffe60 0x7f98bc100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1345509350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc100fb0 0x7f98bc101430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1345509350 >> 172.21.15.16:0/1345509350 conn(0x7f98bc0fb3f0 msgr2=0x7f98bc0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1345509350 shutdown_connections 2024-05-08T23:17:58.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.878+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1345509350 wait complete. 2024-05-08T23:17:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 Processor -- start 2024-05-08T23:17:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 -- start start 2024-05-08T23:17:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc0ffe60 0x7f98bc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98bc100fb0 0x7f98bc19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 0x7f98bc19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc19e890 con 0x7f98bc101970 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc19ea00 con 0x7f98bc0ffe60 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c2b9e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc19ebc0 con 0x7f98bc100fb0 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c113b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 0x7f98bc19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c113b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 0x7f98bc19e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52870/0 (socket says 172.21.15.16:52870) 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.879+0000 7f98c113b700 1 -- 172.21.15.16:0/1409530465 learned_addr learned my addr 172.21.15.16:0/1409530465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c093a700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc0ffe60 0x7f98bc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98bbfff700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98bc100fb0 0x7f98bc19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c113b700 1 -- 172.21.15.16:0/1409530465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98bc100fb0 msgr2=0x7f98bc19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c113b700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98bc100fb0 0x7f98bc19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c113b700 1 -- 172.21.15.16:0/1409530465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc0ffe60 msgr2=0x7f98bc068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c113b700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc0ffe60 0x7f98bc068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c113b700 1 -- 172.21.15.16:0/1409530465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a80095f0 con 0x7f98bc101970 2024-05-08T23:17:58.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c113b700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 0x7f98bc19e0f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98b400ef30 tx=0x7f98b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4018070 con 0x7f98bc101970 2024-05-08T23:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98b400f040 con 0x7f98bc101970 2024-05-08T23:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc19ee40 con 0x7f98bc101970 2024-05-08T23:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.880+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b400cbc0 con 0x7f98bc101970 2024-05-08T23:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.881+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc19f270 con 0x7f98bc101970 2024-05-08T23:17:58.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.882+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc107320 con 0x7f98bc101970 2024-05-08T23:17:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.883+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f98b4007500 con 0x7f98bc101970 2024-05-08T23:17:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.883+0000 7f98b9ffb700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f98ac0a1120 0x7f98ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.884+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f98b40cc580 con 0x7f98bc101970 2024-05-08T23:17:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.884+0000 7f98c093a700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f98ac0a1120 0x7f98ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:58.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.884+0000 7f98c093a700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f98ac0a1120 0x7f98ac0a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009c60 tx=0x7f98b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:58.888+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f98b4095bb0 con 0x7f98bc101970 2024-05-08T23:17:58.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:58 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/363200485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:58.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:58 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2745852987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:17:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:58 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/363200485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:58.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:58 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2745852987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.107+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f98bc02cdf0 con 0x7f98bc101970 2024-05-08T23:17:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.108+0000 7f98b9ffb700 1 -- 172.21.15.16:0/1409530465 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v81) v1 ==== 103+0+11 (secure 0 0 0) 0x7f98b40991d0 con 0x7f98bc101970 2024-05-08T23:17:59.108 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:17:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f98ac0a1120 msgr2=0x7f98ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f98ac0a1120 0x7f98ac0a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009c60 tx=0x7f98b0009260 comp rx=0 tx=0).stop 2024-05-08T23:17:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 msgr2=0x7f98bc19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 0x7f98bc19e0f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f98b400ef30 tx=0x7f98b400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 shutdown_connections 2024-05-08T23:17:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc101970 0x7f98bc19e0f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98bc0ffe60 0x7f98bc068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f98ac0a1120 0x7f98ac0a35e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 --2- 172.21.15.16:0/1409530465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98bc100fb0 0x7f98bc19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 >> 172.21.15.16:0/1409530465 conn(0x7f98bc0fb3f0 msgr2=0x7f98bc104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 shutdown_connections 2024-05-08T23:17:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.110+0000 7f98c2b9e700 1 -- 172.21.15.16:0/1409530465 wait complete. 2024-05-08T23:17:59.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:58 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/363200485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:58 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2745852987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:17:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 -- 172.21.15.16:0/4091070490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 msgr2=0x7f65a81093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 --2- 172.21.15.16:0/4091070490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a81093d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b0a0 tx=0x7f65a000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 -- 172.21.15.16:0/4091070490 shutdown_connections 2024-05-08T23:17:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 --2- 172.21.15.16:0/4091070490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a81093d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 --2- 172.21.15.16:0/4091070490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8100f20 0x7f65a8101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 --2- 172.21.15.16:0/4091070490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 0x7f65a8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 -- 172.21.15.16:0/4091070490 >> 172.21.15.16:0/4091070490 conn(0x7f65a80fc4e0 msgr2=0x7f65a80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 -- 172.21.15.16:0/4091070490 shutdown_connections 2024-05-08T23:17:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.262+0000 7f65affc6700 1 -- 172.21.15.16:0/4091070490 wait complete. 2024-05-08T23:17:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 Processor -- start 2024-05-08T23:17:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 -- start start 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8100f20 0x7f65a8196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 0x7f65a8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819ea50 con 0x7f65a8102ab0 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819eb90 con 0x7f65a8100f20 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65affc6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819ed20 con 0x7f65a81020f0 2024-05-08T23:17:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65ae563700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65ae563700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a819e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52898/0 (socket says 172.21.15.16:52898) 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65ae563700 1 -- 172.21.15.16:0/1447833225 learned_addr learned my addr 172.21.15.16:0/1447833225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65ad561700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 0x7f65a8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.263+0000 7f65add62700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8100f20 0x7f65a8196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ae563700 1 -- 172.21.15.16:0/1447833225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 msgr2=0x7f65a8197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ae563700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 0x7f65a8197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ae563700 1 -- 172.21.15.16:0/1447833225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8100f20 msgr2=0x7f65a8196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ae563700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8100f20 0x7f65a8196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ae563700 1 -- 172.21.15.16:0/1447833225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0009d00 con 0x7f65a8102ab0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ad561700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 0x7f65a8197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65ae563700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a819e480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f65a0007950 tx=0x7f65a0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000e040 con 0x7f65a8102ab0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65a0004c50 con 0x7f65a8102ab0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a819efa0 con 0x7f65a8102ab0 2024-05-08T23:17:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0005040 con 0x7f65a8102ab0 2024-05-08T23:17:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.264+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a819f3d0 con 0x7f65a8102ab0 2024-05-08T23:17:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.265+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a81083e0 con 0x7f65a8102ab0 2024-05-08T23:17:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.266+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f65a001d470 con 0x7f65a8102ab0 2024-05-08T23:17:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.267+0000 7f659effd700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65940a11a0 0x7f65940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.267+0000 7f65add62700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65940a11a0 0x7f65940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.267+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f65a00ce640 con 0x7f65a8102ab0 2024-05-08T23:17:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.268+0000 7f65add62700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65940a11a0 0x7f65940a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f65a8100c40 tx=0x7f65a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.271+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f65a0097e20 con 0x7f65a8102ab0 2024-05-08T23:17:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.511+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65a8067460 con 0x7f65a8102ab0 2024-05-08T23:17:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.511+0000 7f659effd700 1 -- 172.21.15.16:0/1447833225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) v1 ==== 81+0+589 (secure 0 0 0) 0x7f65a0021080 con 0x7f65a8102ab0 2024-05-08T23:17:59.512 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4190536936 2024-05-09T23:12:42.499707+0000 2024-05-08T23:17:59.512 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/30115990 2024-05-09T23:12:42.499707+0000 2024-05-08T23:17:59.512 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/972303159 2024-05-09T23:12:42.499707+0000 2024-05-08T23:17:59.512 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:17:59.513 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1837768303 2024-05-09T23:12:19.352750+0000 2024-05-08T23:17:59.513 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:17:59.513 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4028430980 2024-05-09T23:12:19.352750+0000 2024-05-08T23:17:59.513 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4149284949 2024-05-09T23:12:19.352750+0000 2024-05-08T23:17:59.513 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:17:59.513 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:17:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.513+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65940a11a0 msgr2=0x7f65940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.513+0000 7f65affc6700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65940a11a0 0x7f65940a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f65a8100c40 tx=0x7f65a4009260 comp rx=0 tx=0).stop 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.513+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 msgr2=0x7f65a819e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.513+0000 7f65affc6700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a819e480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f65a0007950 tx=0x7f65a0007870 comp rx=0 tx=0).stop 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 shutdown_connections 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8102ab0 0x7f65a819e480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8100f20 0x7f65a8196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65940a11a0 0x7f65940a3660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 --2- 172.21.15.16:0/1447833225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a81020f0 0x7f65a8197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 >> 172.21.15.16:0/1447833225 conn(0x7f65a80fc4e0 msgr2=0x7f65a8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 shutdown_connections 2024-05-08T23:17:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.514+0000 7f65affc6700 1 -- 172.21.15.16:0/1447833225 wait complete. 2024-05-08T23:17:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-05-08T23:17:59.528 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:17:59.528 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:17:59.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:17:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 -- 172.21.15.16:0/3981510696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a01018e0 msgr2=0x7f43a0103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 --2- 172.21.15.16:0/3981510696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a01018e0 0x7f43a0103cd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4390009990 tx=0x7f4390009ca0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 -- 172.21.15.16:0/3981510696 shutdown_connections 2024-05-08T23:17:59.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 --2- 172.21.15.16:0/3981510696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43a0106b40 0x7f43a0108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 --2- 172.21.15.16:0/3981510696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a01018e0 0x7f43a0103cd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 --2- 172.21.15.16:0/3981510696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a0104210 0x7f43a0106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 -- 172.21.15.16:0/3981510696 >> 172.21.15.16:0/3981510696 conn(0x7f43a00fb2f0 msgr2=0x7f43a00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:59.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 -- 172.21.15.16:0/3981510696 shutdown_connections 2024-05-08T23:17:59.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.728+0000 7f43a4838700 1 -- 172.21.15.16:0/3981510696 wait complete. 2024-05-08T23:17:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 Processor -- start 2024-05-08T23:17:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 -- start start 2024-05-08T23:17:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01018e0 0x7f43a0196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43a0104210 0x7f43a01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 0x7f43a019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019e850 con 0x7f43a0106b40 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019e9c0 con 0x7f43a01018e0 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.729+0000 7f43a4838700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019eb80 con 0x7f43a0104210 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 0x7f43a019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 0x7f43a019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52910/0 (socket says 172.21.15.16:52910) 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 -- 172.21.15.16:0/4262355046 learned_addr learned my addr 172.21.15.16:0/4262355046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f4397fff700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43a0104210 0x7f43a01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439e59c700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01018e0 0x7f43a0196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 -- 172.21.15.16:0/4262355046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43a0104210 msgr2=0x7f43a01972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43a0104210 0x7f43a01972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 -- 172.21.15.16:0/4262355046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01018e0 msgr2=0x7f43a0196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01018e0 0x7f43a0196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 -- 172.21.15.16:0/4262355046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43900095f0 con 0x7f43a0106b40 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f439ed9d700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 0x7f43a019e280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f439800ef30 tx=0x7f439800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398018070 con 0x7f43a0106b40 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f439800f040 con 0x7f43a0106b40 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a019ee00 con 0x7f43a0106b40 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439800cc70 con 0x7f43a0106b40 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.730+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a019f230 con 0x7f43a0106b40 2024-05-08T23:17:59.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.732+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a00fce70 con 0x7f43a0106b40 2024-05-08T23:17:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.733+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4398003a00 con 0x7f43a0106b40 2024-05-08T23:17:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.733+0000 7f43977fe700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f438c0a1120 0x7f438c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:17:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.734+0000 7f439e59c700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f438c0a1120 0x7f438c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:17:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.734+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f43980ccd00 con 0x7f43a0106b40 2024-05-08T23:17:59.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.734+0000 7f439e59c700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f438c0a1120 0x7f438c0a35e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43a01981e0 tx=0x7f439000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:17:59.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.737+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f43980962d0 con 0x7f43a0106b40 2024-05-08T23:17:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:59 smithi016 ceph-mon[42541]: pgmap v229: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2024-05-08T23:17:59.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:59 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1242559661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:59 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1409530465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:17:59 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1447833225' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:17:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:59 smithi189 ceph-mon[46919]: pgmap v229: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2024-05-08T23:17:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:59 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1242559661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:59 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1409530465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:17:59 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1447833225' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:17:59.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.956+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43a0067c10 con 0x7f43a0106b40 2024-05-08T23:17:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.958+0000 7f43977fe700 1 -- 172.21.15.16:0/4262355046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f43980998f0 con 0x7f43a0106b40 2024-05-08T23:17:59.959 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:17:59.959 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:17:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f438c0a1120 msgr2=0x7f438c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f438c0a1120 0x7f438c0a35e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43a01981e0 tx=0x7f439000b610 comp rx=0 tx=0).stop 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 msgr2=0x7f43a019e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 0x7f43a019e280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f439800ef30 tx=0x7f439800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 shutdown_connections 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a0106b40 0x7f43a019e280 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43a01018e0 0x7f43a0196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f438c0a1120 0x7f438c0a35e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 --2- 172.21.15.16:0/4262355046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43a0104210 0x7f43a01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 >> 172.21.15.16:0/4262355046 conn(0x7f43a00fb2f0 msgr2=0x7f43a0103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 shutdown_connections 2024-05-08T23:17:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:17:59.961+0000 7f43a4838700 1 -- 172.21.15.16:0/4262355046 wait complete. 2024-05-08T23:17:59.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:59 smithi155 ceph-mon[47105]: pgmap v229: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2024-05-08T23:17:59.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:59 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1242559661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:59 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1409530465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:17:59.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:17:59 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1447833225' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:17:59.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:17:59.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 -- 172.21.15.16:0/3118314490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda90101980 msgr2=0x7fda901082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 --2- 172.21.15.16:0/3118314490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda90101980 0x7fda901082a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda8800b0a0 tx=0x7fda8800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 -- 172.21.15.16:0/3118314490 shutdown_connections 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 --2- 172.21.15.16:0/3118314490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda90101980 0x7fda901082a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 --2- 172.21.15.16:0/3118314490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda900ffe70 0x7fda90100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 --2- 172.21.15.16:0/3118314490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90100fc0 0x7fda90101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 -- 172.21.15.16:0/3118314490 >> 172.21.15.16:0/3118314490 conn(0x7fda900fb400 msgr2=0x7fda900fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 -- 172.21.15.16:0/3118314490 shutdown_connections 2024-05-08T23:18:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 -- 172.21.15.16:0/3118314490 wait complete. 2024-05-08T23:18:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 Processor -- start 2024-05-08T23:18:00.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.115+0000 7fda96390700 1 -- start start 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda96390700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 0x7fda90196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda96390700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda90100fc0 0x7fda90197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda96390700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90101980 0x7fda9019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda96390700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9019ea50 con 0x7fda900ffe70 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda96390700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9019eb90 con 0x7fda90101980 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda96390700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9019ed20 con 0x7fda90100fc0 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 0x7fda90196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 0x7fda90196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52920/0 (socket says 172.21.15.16:52920) 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 -- 172.21.15.16:0/1310463630 learned_addr learned my addr 172.21.15.16:0/1310463630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:00.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda9492d700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90101980 0x7fda9019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8f7fe700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda90100fc0 0x7fda90197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 -- 172.21.15.16:0/1310463630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda90100fc0 msgr2=0x7fda90197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda90100fc0 0x7fda90197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 -- 172.21.15.16:0/1310463630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90101980 msgr2=0x7fda9019e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90101980 0x7fda9019e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.116+0000 7fda8ffff700 1 -- 172.21.15.16:0/1310463630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88009d00 con 0x7fda900ffe70 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.117+0000 7fda8ffff700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 0x7fda90196f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9a0 tx=0x7fda8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.117+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80009790 con 0x7fda900ffe70 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.117+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda80010430 con 0x7fda900ffe70 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.117+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9019efa0 con 0x7fda900ffe70 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.117+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8000cea0 con 0x7fda900ffe70 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.117+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9019f420 con 0x7fda900ffe70 2024-05-08T23:18:00.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.118+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda90068140 con 0x7fda900ffe70 2024-05-08T23:18:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.119+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fda800105a0 con 0x7fda900ffe70 2024-05-08T23:18:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.120+0000 7fda8d7fa700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda780a1120 0x7fda780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.120+0000 7fda8f7fe700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda780a1120 0x7fda780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.120+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda80014070 con 0x7fda900ffe70 2024-05-08T23:18:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.120+0000 7fda8f7fe700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda780a1120 0x7fda780a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda84009c60 tx=0x7fda84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.124+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fda80052d60 con 0x7fda900ffe70 2024-05-08T23:18:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.341+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda9010b520 con 0x7fda900ffe70 2024-05-08T23:18:00.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.343+0000 7fda8d7fa700 1 -- 172.21.15.16:0/1310463630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda80096240 con 0x7fda900ffe70 2024-05-08T23:18:00.343 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda780a1120 msgr2=0x7fda780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda780a1120 0x7fda780a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda84009c60 tx=0x7fda84009260 comp rx=0 tx=0).stop 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 msgr2=0x7fda90196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 0x7fda90196f20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9a0 tx=0x7fda8000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 shutdown_connections 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90101980 0x7fda9019e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda900ffe70 0x7fda90196f20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fda780a1120 0x7fda780a35e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 --2- 172.21.15.16:0/1310463630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda90100fc0 0x7fda90197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 >> 172.21.15.16:0/1310463630 conn(0x7fda900fb400 msgr2=0x7fda90069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 shutdown_connections 2024-05-08T23:18:00.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.344+0000 7fda96390700 1 -- 172.21.15.16:0/1310463630 wait complete. 2024-05-08T23:18:00.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:18:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 -- 172.21.15.16:0/3766674182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 msgr2=0x7f068c1093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 --2- 172.21.15.16:0/3766674182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c1093d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f068400b0a0 tx=0x7f068400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 -- 172.21.15.16:0/3766674182 shutdown_connections 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 --2- 172.21.15.16:0/3766674182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c1093d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 --2- 172.21.15.16:0/3766674182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f068c100f20 0x7f068c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 --2- 172.21.15.16:0/3766674182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f068c1020f0 0x7f068c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 -- 172.21.15.16:0/3766674182 >> 172.21.15.16:0/3766674182 conn(0x7f068c0fc4e0 msgr2=0x7f068c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 -- 172.21.15.16:0/3766674182 shutdown_connections 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.500+0000 7f069275c700 1 -- 172.21.15.16:0/3766674182 wait complete. 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 Processor -- start 2024-05-08T23:18:00.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 -- start start 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f068c100f20 0x7f068c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f068c1020f0 0x7f068c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c19ea50 con 0x7f068c102ab0 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c19eb90 con 0x7f068c1020f0 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f069275c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c19ed20 con 0x7f068c100f20 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f0690cf9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f0690cf9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52930/0 (socket says 172.21.15.16:52930) 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f0690cf9700 1 -- 172.21.15.16:0/1298338495 learned_addr learned my addr 172.21.15.16:0/1298338495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f068b7fe700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f068c1020f0 0x7f068c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.501+0000 7f068bfff700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f068c100f20 0x7f068c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f0690cf9700 1 -- 172.21.15.16:0/1298338495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f068c100f20 msgr2=0x7f068c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f0690cf9700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f068c100f20 0x7f068c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f0690cf9700 1 -- 172.21.15.16:0/1298338495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f068c1020f0 msgr2=0x7f068c197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f0690cf9700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f068c1020f0 0x7f068c197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f0690cf9700 1 -- 172.21.15.16:0/1298338495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0684009d00 con 0x7f068c102ab0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f0690cf9700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c19e480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0684007950 tx=0x7f068400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068400e040 con 0x7f068c102ab0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0684004c50 con 0x7f068c102ab0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c19efa0 con 0x7f068c102ab0 2024-05-08T23:18:00.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840052a0 con 0x7f068c102ab0 2024-05-08T23:18:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.502+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c19f3c0 con 0x7f068c102ab0 2024-05-08T23:18:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.503+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c1083e0 con 0x7f068c102ab0 2024-05-08T23:18:00.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.505+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f068401d470 con 0x7f068c102ab0 2024-05-08T23:18:00.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.505+0000 7f06897fa700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f06740a11a0 0x7f06740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.506+0000 7f068bfff700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f06740a11a0 0x7f06740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.506+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06840ce690 con 0x7f068c102ab0 2024-05-08T23:18:00.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.506+0000 7f068bfff700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f06740a11a0 0x7f06740a3660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f068c100c40 tx=0x7f067c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:00.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.509+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0684097cc0 con 0x7f068c102ab0 2024-05-08T23:18:00.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.730+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f068c10c7a0 con 0x7f068c102ab0 2024-05-08T23:18:00.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.730+0000 7f06897fa700 1 -- 172.21.15.16:0/1298338495 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f068409b2e0 con 0x7f068c102ab0 2024-05-08T23:18:00.730 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:18:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f06740a11a0 msgr2=0x7f06740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f06740a11a0 0x7f06740a3660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f068c100c40 tx=0x7f067c007400 comp rx=0 tx=0).stop 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 msgr2=0x7f068c19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c19e480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0684007950 tx=0x7f068400ba50 comp rx=0 tx=0).stop 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 shutdown_connections 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f068c102ab0 0x7f068c19e480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f06740a11a0 0x7f06740a3660 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f068c100f20 0x7f068c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 --2- 172.21.15.16:0/1298338495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f068c1020f0 0x7f068c197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 >> 172.21.15.16:0/1298338495 conn(0x7f068c0fc4e0 msgr2=0x7f068c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 shutdown_connections 2024-05-08T23:18:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.733+0000 7f069275c700 1 -- 172.21.15.16:0/1298338495 wait complete. 2024-05-08T23:18:00.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:18:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 -- 172.21.15.16:0/3032355860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 msgr2=0x7fc734100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 --2- 172.21.15.16:0/3032355860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734100280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc730009990 tx=0x7fc730009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 -- 172.21.15.16:0/3032355860 shutdown_connections 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 --2- 172.21.15.16:0/3032355860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 0x7fc734108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 --2- 172.21.15.16:0/3032355860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734100280 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 --2- 172.21.15.16:0/3032355860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc734100fb0 0x7fc734101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 -- 172.21.15.16:0/3032355860 >> 172.21.15.16:0/3032355860 conn(0x7fc7340fb3f0 msgr2=0x7fc7340fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 -- 172.21.15.16:0/3032355860 shutdown_connections 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.890+0000 7fc73c38b700 1 -- 172.21.15.16:0/3032355860 wait complete. 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 Processor -- start 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 -- start start 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc734100fb0 0x7fc734197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 0x7fc73419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419e8e0 con 0x7fc7340ffe60 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419ea50 con 0x7fc734100fb0 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73c38b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73419ec10 con 0x7fc734101970 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52946/0 (socket says 172.21.15.16:52946) 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 -- 172.21.15.16:0/726882643 learned_addr learned my addr 172.21.15.16:0/726882643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a928700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 0x7fc73419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc739926700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc734100fb0 0x7fc734197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 -- 172.21.15.16:0/726882643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 msgr2=0x7fc73419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 0x7fc73419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 -- 172.21.15.16:0/726882643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc734100fb0 msgr2=0x7fc734197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc734100fb0 0x7fc734197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a127700 1 -- 172.21.15.16:0/726882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300095f0 con 0x7fc7340ffe60 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.891+0000 7fc73a928700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 0x7fc73419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.892+0000 7fc73a127700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734196df0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc73000b5c0 tx=0x7fc730005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.892+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73001d070 con 0x7fc7340ffe60 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.892+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc730004080 con 0x7fc7340ffe60 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.892+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730022c90 con 0x7fc7340ffe60 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.892+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73419ee90 con 0x7fc7340ffe60 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.892+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73419f200 con 0x7fc7340ffe60 2024-05-08T23:18:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.893+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734067cf0 con 0x7fc7340ffe60 2024-05-08T23:18:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.894+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc730022470 con 0x7fc7340ffe60 2024-05-08T23:18:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.895+0000 7fc72b7fe700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7200a11a0 0x7fc7200a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.895+0000 7fc739926700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7200a11a0 0x7fc7200a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.895+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc7300ce220 con 0x7fc7340ffe60 2024-05-08T23:18:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.896+0000 7fc739926700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7200a11a0 0x7fc7200a3660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc724005950 tx=0x7fc7240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:00.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:00.899+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc7300977f0 con 0x7fc7340ffe60 2024-05-08T23:18:00.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:00 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4262355046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:00.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:00 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1310463630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:18:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:00 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4262355046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:00.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:00 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1310463630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:00 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4262355046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:01.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:00 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1310463630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.118+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc73410b6a0 con 0x7fc7340ffe60 2024-05-08T23:18:01.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.118+0000 7fc72b7fe700 1 -- 172.21.15.16:0/726882643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7300041f0 con 0x7fc7340ffe60 2024-05-08T23:18:01.118 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7200a11a0 msgr2=0x7fc7200a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7200a11a0 0x7fc7200a3660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc724005950 tx=0x7fc7240058e0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 msgr2=0x7fc734196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734196df0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc73000b5c0 tx=0x7fc730005ef0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 shutdown_connections 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7200a11a0 0x7fc7200a3660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc734101970 0x7fc73419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7340ffe60 0x7fc734196df0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 --2- 172.21.15.16:0/726882643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc734100fb0 0x7fc734197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 >> 172.21.15.16:0/726882643 conn(0x7fc7340fb3f0 msgr2=0x7fc7340680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 shutdown_connections 2024-05-08T23:18:01.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.121+0000 7fc73c38b700 1 -- 172.21.15.16:0/726882643 wait complete. 2024-05-08T23:18:01.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 -- 172.21.15.16:0/4158223448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c100fd0 msgr2=0x7f427c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 --2- 172.21.15.16:0/4158223448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c100fd0 0x7f427c101450 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f426c009990 tx=0x7f426c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 -- 172.21.15.16:0/4158223448 shutdown_connections 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 --2- 172.21.15.16:0/4158223448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f427c101990 0x7f427c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 --2- 172.21.15.16:0/4158223448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f427c0ffe80 0x7f427c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 --2- 172.21.15.16:0/4158223448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c100fd0 0x7f427c101450 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 -- 172.21.15.16:0/4158223448 >> 172.21.15.16:0/4158223448 conn(0x7f427c0fb3f0 msgr2=0x7f427c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:01.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 -- 172.21.15.16:0/4158223448 shutdown_connections 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.274+0000 7f42837cd700 1 -- 172.21.15.16:0/4158223448 wait complete. 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 Processor -- start 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 -- start start 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f427c0ffe80 0x7f427c196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f427c100fd0 0x7f427c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 0x7f427c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c19e950 con 0x7f427c101990 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c19ea90 con 0x7f427c100fd0 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f42837cd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c19ec20 con 0x7f427c0ffe80 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f4281d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 0x7f427c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f4281d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 0x7f427c19e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52958/0 (socket says 172.21.15.16:52958) 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f4281d6a700 1 -- 172.21.15.16:0/4022931482 learned_addr learned my addr 172.21.15.16:0/4022931482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f4281569700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f427c0ffe80 0x7f427c196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.275+0000 7f4280d68700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f427c100fd0 0x7f427c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f4281d6a700 1 -- 172.21.15.16:0/4022931482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f427c0ffe80 msgr2=0x7f427c196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f4281d6a700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f427c0ffe80 0x7f427c196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f4281d6a700 1 -- 172.21.15.16:0/4022931482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f427c100fd0 msgr2=0x7f427c1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f4281d6a700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f427c100fd0 0x7f427c1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f4281d6a700 1 -- 172.21.15.16:0/4022931482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0095f0 con 0x7f427c101990 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f4281d6a700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 0x7f427c19e380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f427400ef30 tx=0x7f427400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4274018070 con 0x7f427c101990 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427400f040 con 0x7f427c101990 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c19eea0 con 0x7f427c101990 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427400cbc0 con 0x7f427c101990 2024-05-08T23:18:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.276+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c19f2d0 con 0x7f427c101990 2024-05-08T23:18:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.277+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c067460 con 0x7f427c101990 2024-05-08T23:18:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.278+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f427400cd60 con 0x7f427c101990 2024-05-08T23:18:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.279+0000 7f42727fc700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f42680a1120 0x7f42680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.279+0000 7f4281569700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f42680a1120 0x7f42680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.279+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f42740ccb20 con 0x7f427c101990 2024-05-08T23:18:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.280+0000 7f4281569700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f42680a1120 0x7f42680a35e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4278007c60 tx=0x7f42780073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:01.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.283+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f42740960f0 con 0x7f427c101990 2024-05-08T23:18:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.501+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f427c19f610 con 0x7f427c101990 2024-05-08T23:18:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.502+0000 7f42727fc700 1 -- 172.21.15.16:0/4022931482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v81) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4274099710 con 0x7f427c101990 2024-05-08T23:18:01.502 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:18:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f42680a1120 msgr2=0x7f42680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f42680a1120 0x7f42680a35e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4278007c60 tx=0x7f42780073d0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 msgr2=0x7f427c19e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 0x7f427c19e380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f427400ef30 tx=0x7f427400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 shutdown_connections 2024-05-08T23:18:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f427c101990 0x7f427c19e380 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f42680a1120 0x7f42680a35e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f427c0ffe80 0x7f427c196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 --2- 172.21.15.16:0/4022931482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f427c100fd0 0x7f427c1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 >> 172.21.15.16:0/4022931482 conn(0x7f427c0fb3f0 msgr2=0x7f427c069220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.504+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 shutdown_connections 2024-05-08T23:18:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.505+0000 7f42837cd700 1 -- 172.21.15.16:0/4022931482 wait complete. 2024-05-08T23:18:01.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 -- 172.21.15.16:0/3790855902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 msgr2=0x7f62241082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 --2- 172.21.15.16:0/3790855902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f62241082a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f621c00b0a0 tx=0x7f621c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 -- 172.21.15.16:0/3790855902 shutdown_connections 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 --2- 172.21.15.16:0/3790855902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f62241082a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 --2- 172.21.15.16:0/3790855902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62240ffe70 0x7f6224100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 --2- 172.21.15.16:0/3790855902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6224100fc0 0x7f6224101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 -- 172.21.15.16:0/3790855902 >> 172.21.15.16:0/3790855902 conn(0x7f62240fb400 msgr2=0x7f62240fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:01.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 -- 172.21.15.16:0/3790855902 shutdown_connections 2024-05-08T23:18:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.659+0000 7f622c7d1700 1 -- 172.21.15.16:0/3790855902 wait complete. 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 Processor -- start 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 -- start start 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62240ffe70 0x7f6224196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6224100fc0 0x7f6224197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f622419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622419ea30 con 0x7f6224101980 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622419eb70 con 0x7f62240ffe70 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622c7d1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622419ed00 con 0x7f6224100fc0 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622ad6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f622419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622a56d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62240ffe70 0x7f6224196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622ad6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f622419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52982/0 (socket says 172.21.15.16:52982) 2024-05-08T23:18:01.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f622ad6e700 1 -- 172.21.15.16:0/1437955237 learned_addr learned my addr 172.21.15.16:0/1437955237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.660+0000 7f6229d6c700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6224100fc0 0x7f6224197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622ad6e700 1 -- 172.21.15.16:0/1437955237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6224100fc0 msgr2=0x7f6224197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622ad6e700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6224100fc0 0x7f6224197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622ad6e700 1 -- 172.21.15.16:0/1437955237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62240ffe70 msgr2=0x7f6224196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622ad6e700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62240ffe70 0x7f6224196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622ad6e700 1 -- 172.21.15.16:0/1437955237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009d00 con 0x7f6224101980 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622ad6e700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f622419e460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f621c009840 tx=0x7f621c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00e040 con 0x7f6224101980 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f621c004c50 con 0x7f6224101980 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622419ef80 con 0x7f6224101980 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0052a0 con 0x7f6224101980 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.661+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622419f3a0 con 0x7f6224101980 2024-05-08T23:18:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.662+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62241072e0 con 0x7f6224101980 2024-05-08T23:18:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.663+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f621c01d470 con 0x7f6224101980 2024-05-08T23:18:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.664+0000 7f621b7fe700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f62100a1120 0x7f62100a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.664+0000 7f622a56d700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f62100a1120 0x7f62100a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.664+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f621c0cea80 con 0x7f6224101980 2024-05-08T23:18:01.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.665+0000 7f622a56d700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f62100a1120 0x7f62100a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6220009c60 tx=0x7f6220009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.668+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f621c098170 con 0x7f6224101980 2024-05-08T23:18:01.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:01 smithi016 ceph-mon[42541]: pgmap v230: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-08T23:18:01.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:01 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1298338495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:01 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/726882643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:01 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4022931482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.901+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6224068140 con 0x7f6224101980 2024-05-08T23:18:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.902+0000 7f621b7fe700 1 -- 172.21.15.16:0/1437955237 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v81) v1 ==== 81+0+589 (secure 0 0 0) 0x7f621c021260 con 0x7f6224101980 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4190536936 2024-05-09T23:12:42.499707+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/30115990 2024-05-09T23:12:42.499707+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/972303159 2024-05-09T23:12:42.499707+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1837768303 2024-05-09T23:12:19.352750+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4028430980 2024-05-09T23:12:19.352750+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4149284949 2024-05-09T23:12:19.352750+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:18:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:18:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f62100a1120 msgr2=0x7f62100a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f62100a1120 0x7f62100a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6220009c60 tx=0x7f6220009260 comp rx=0 tx=0).stop 2024-05-08T23:18:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 msgr2=0x7f622419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f622419e460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f621c009840 tx=0x7f621c00ba50 comp rx=0 tx=0).stop 2024-05-08T23:18:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 shutdown_connections 2024-05-08T23:18:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6224101980 0x7f622419e460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62240ffe70 0x7f6224196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f62100a1120 0x7f62100a35e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 --2- 172.21.15.16:0/1437955237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6224100fc0 0x7f6224197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 >> 172.21.15.16:0/1437955237 conn(0x7f62240fb400 msgr2=0x7f6224104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 shutdown_connections 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:01.904+0000 7f622c7d1700 1 -- 172.21.15.16:0/1437955237 wait complete. 2024-05-08T23:18:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-05-08T23:18:01.918 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-05-08T23:18:01.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:18:01.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T23:18:01.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:01 smithi189 ceph-mon[46919]: pgmap v230: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-08T23:18:01.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:01 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1298338495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:01 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/726882643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:01 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4022931482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[98366]: fuse finished with error 0 and tester_r 0 2024-05-08T23:18:01.957 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:18:01.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:01 smithi155 ceph-mon[47105]: pgmap v230: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-05-08T23:18:01.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:01 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1298338495' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:01 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/726882643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:18:01.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:01 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4022931482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:18:02.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:02 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1437955237' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:18:02.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:02 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1437955237' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:18:03.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:02 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1437955237' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:18:03.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:03 smithi189 ceph-mon[46919]: pgmap v231: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 256 B/s wr, 0 op/s 2024-05-08T23:18:03.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:03 smithi155 ceph-mon[47105]: pgmap v231: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 256 B/s wr, 0 op/s 2024-05-08T23:18:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:03 smithi016 ceph-mon[42541]: pgmap v231: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 256 B/s wr, 0 op/s 2024-05-08T23:18:05.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:05 smithi189 ceph-mon[46919]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:05.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:05 smithi155 ceph-mon[47105]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:05 smithi016 ceph-mon[42541]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:07.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:07 smithi189 ceph-mon[46919]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:07.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:07 smithi155 ceph-mon[47105]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:08.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:18:08.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:18:08.059 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T23:18:08.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:18:08.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:07 smithi016 ceph-mon[42541]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:08.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 -- 172.21.15.16:0/320996486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 msgr2=0x7f73781092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 --2- 172.21.15.16:0/320996486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 0x7f73781092a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f737000b0a0 tx=0x7f737000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 -- 172.21.15.16:0/320996486 shutdown_connections 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 --2- 172.21.15.16:0/320996486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 0x7f73781092a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 --2- 172.21.15.16:0/320996486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7378100e70 0x7f7378101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 --2- 172.21.15.16:0/320996486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 -- 172.21.15.16:0/320996486 >> 172.21.15.16:0/320996486 conn(0x7f73780fc400 msgr2=0x7f73780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 -- 172.21.15.16:0/320996486 shutdown_connections 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.262+0000 7f737d8a9700 1 -- 172.21.15.16:0/320996486 wait complete. 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 Processor -- start 2024-05-08T23:18:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 -- start start 2024-05-08T23:18:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7378100e70 0x7f7378196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 0x7f737819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737819e8e0 con 0x7f7378101fc0 2024-05-08T23:18:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737819ea50 con 0x7f7378102980 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f737d8a9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737819ec10 con 0x7f7378100e70 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f73767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.263+0000 7f73767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34484/0 (socket says 172.21.15.16:34484) 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 -- 172.21.15.16:0/1918612667 learned_addr learned my addr 172.21.15.16:0/1918612667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f7376ffd700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7378100e70 0x7f7378196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73777fe700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 0x7f737819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 -- 172.21.15.16:0/1918612667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7378100e70 msgr2=0x7f7378196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7378100e70 0x7f7378196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 -- 172.21.15.16:0/1918612667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 msgr2=0x7f737819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 0x7f737819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 -- 172.21.15.16:0/1918612667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370009d00 con 0x7f7378101fc0 2024-05-08T23:18:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f73767fc700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378197330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9a0 tx=0x7f736000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360009790 con 0x7f7378101fc0 2024-05-08T23:18:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737819eef0 con 0x7f7378101fc0 2024-05-08T23:18:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7360010430 con 0x7f7378101fc0 2024-05-08T23:18:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000cea0 con 0x7f7378101fc0 2024-05-08T23:18:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.264+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737819f310 con 0x7f7378101fc0 2024-05-08T23:18:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.265+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7378108300 con 0x7f7378101fc0 2024-05-08T23:18:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.266+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f73600105a0 con 0x7f7378101fc0 2024-05-08T23:18:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.267+0000 7f737c8a7700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f73640a1120 0x7f73640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.267+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7360014070 con 0x7f7378101fc0 2024-05-08T23:18:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.267+0000 7f7376ffd700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f73640a1120 0x7f73640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.268+0000 7f7376ffd700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f73640a1120 0x7f73640a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7378198270 tx=0x7f7368009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.272+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7360052d60 con 0x7f7378101fc0 2024-05-08T23:18:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.489+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7378068060 con 0x7f7378101fc0 2024-05-08T23:18:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.490+0000 7f737c8a7700 1 -- 172.21.15.16:0/1918612667 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f7360096250 con 0x7f7378101fc0 2024-05-08T23:18:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:08.491 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:18:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f73640a1120 msgr2=0x7f73640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f73640a1120 0x7f73640a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7378198270 tx=0x7f7368009260 comp rx=0 tx=0).stop 2024-05-08T23:18:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 msgr2=0x7f7378197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378197330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f736000e9a0 tx=0x7f736000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 shutdown_connections 2024-05-08T23:18:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7378102980 0x7f737819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f73640a1120 0x7f73640a35e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7378100e70 0x7f7378196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 --2- 172.21.15.16:0/1918612667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7378101fc0 0x7f7378197330 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 >> 172.21.15.16:0/1918612667 conn(0x7f73780fc400 msgr2=0x7f7378105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 shutdown_connections 2024-05-08T23:18:08.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.493+0000 7f737d8a9700 1 -- 172.21.15.16:0/1918612667 wait complete. 2024-05-08T23:18:08.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 -- 172.21.15.16:0/2885222248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 msgr2=0x7f4a3c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2885222248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c1081a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400b0d0 tx=0x7f4a3400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 -- 172.21.15.16:0/2885222248 shutdown_connections 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2885222248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c1081a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2885222248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c0ffd60 0x7f4a3c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2885222248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c100f50 0x7f4a3c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 -- 172.21.15.16:0/2885222248 >> 172.21.15.16:0/2885222248 conn(0x7f4a3c0fb2f0 msgr2=0x7f4a3c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 -- 172.21.15.16:0/2885222248 shutdown_connections 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.645+0000 7f4a432aa700 1 -- 172.21.15.16:0/2885222248 wait complete. 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 Processor -- start 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 -- start start 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c0ffd60 0x7f4a3c068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c100f50 0x7f4a3c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19e920 con 0x7f4a3c1018f0 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19ea90 con 0x7f4a3c0ffd60 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a432aa700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19ec50 con 0x7f4a3c100f50 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a41847700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a41847700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c19e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34510/0 (socket says 172.21.15.16:34510) 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a41046700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c0ffd60 0x7f4a3c068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.646+0000 7f4a41847700 1 -- 172.21.15.16:0/2275078982 learned_addr learned my addr 172.21.15.16:0/2275078982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a40845700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c100f50 0x7f4a3c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a41847700 1 -- 172.21.15.16:0/2275078982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c100f50 msgr2=0x7f4a3c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a41847700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c100f50 0x7f4a3c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a41847700 1 -- 172.21.15.16:0/2275078982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c0ffd60 msgr2=0x7f4a3c068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a41847700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c0ffd60 0x7f4a3c068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a41847700 1 -- 172.21.15.16:0/2275078982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a34009d00 con 0x7f4a3c1018f0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a40845700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c100f50 0x7f4a3c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a41847700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c19e130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a34007af0 tx=0x7f4a34007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400e040 con 0x7f4a3c1018f0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a3400be30 con 0x7f4a3c1018f0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c19ee70 con 0x7f4a3c1018f0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3401b930 con 0x7f4a3c1018f0 2024-05-08T23:18:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.647+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c19f2f0 con 0x7f4a3c1018f0 2024-05-08T23:18:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.648+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c107180 con 0x7f4a3c1018f0 2024-05-08T23:18:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.649+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4a3401ba90 con 0x7f4a3c1018f0 2024-05-08T23:18:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.650+0000 7f4a327fc700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4a280a11a0 0x7f4a280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.650+0000 7f4a41046700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4a280a11a0 0x7f4a280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.650+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4a340ce2e0 con 0x7f4a3c1018f0 2024-05-08T23:18:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.650+0000 7f4a41046700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4a280a11a0 0x7f4a280a3660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c100db0 tx=0x7f4a38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.654+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4a34097910 con 0x7f4a3c1018f0 2024-05-08T23:18:08.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1918612667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.877+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a3c067c10 con 0x7f4a3c1018f0 2024-05-08T23:18:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.878+0000 7f4a327fc700 1 -- 172.21.15.16:0/2275078982 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f4a3401bd40 con 0x7f4a3c1018f0 2024-05-08T23:18:08.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:08.878 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:18:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4a280a11a0 msgr2=0x7f4a280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4a280a11a0 0x7f4a280a3660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c100db0 tx=0x7f4a38009260 comp rx=0 tx=0).stop 2024-05-08T23:18:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 msgr2=0x7f4a3c19e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:08.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c19e130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a34007af0 tx=0x7f4a34007870 comp rx=0 tx=0).stop 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 shutdown_connections 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c1018f0 0x7f4a3c19e130 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a3c0ffd60 0x7f4a3c068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4a280a11a0 0x7f4a280a3660 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.880+0000 7f4a432aa700 1 --2- 172.21.15.16:0/2275078982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a3c100f50 0x7f4a3c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.881+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 >> 172.21.15.16:0/2275078982 conn(0x7f4a3c0fb2f0 msgr2=0x7f4a3c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.881+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 shutdown_connections 2024-05-08T23:18:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:08.881+0000 7f4a432aa700 1 -- 172.21.15.16:0/2275078982 wait complete. 2024-05-08T23:18:08.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T23:18:08.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:08 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1918612667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:08.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:08 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1918612667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 -- 172.21.15.16:0/579232155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 msgr2=0x7f22c01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 --2- 172.21.15.16:0/579232155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c01025d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f22b00099c0 tx=0x7f22b0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 -- 172.21.15.16:0/579232155 shutdown_connections 2024-05-08T23:18:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 --2- 172.21.15.16:0/579232155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c0102b10 0x7f22c01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 --2- 172.21.15.16:0/579232155 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c0100f80 0x7f22c01013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 --2- 172.21.15.16:0/579232155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c01025d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 -- 172.21.15.16:0/579232155 >> 172.21.15.16:0/579232155 conn(0x7f22c00fc4f0 msgr2=0x7f22c00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 -- 172.21.15.16:0/579232155 shutdown_connections 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.035+0000 7f22c8026700 1 -- 172.21.15.16:0/579232155 wait complete. 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 Processor -- start 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 -- start start 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c0100f80 0x7f22c0196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c0197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c0102b10 0x7f22c019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c019e960 con 0x7f22c0102170 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c019eaa0 con 0x7f22c0100f80 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c8026700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c019ec30 con 0x7f22c0102b10 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c55c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c0197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c55c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c0197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34524/0 (socket says 172.21.15.16:34524) 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c55c1700 1 -- 172.21.15.16:0/749242242 learned_addr learned my addr 172.21.15.16:0/749242242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.036+0000 7f22c5dc2700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c0100f80 0x7f22c0196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c65c3700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c0102b10 0x7f22c019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c55c1700 1 -- 172.21.15.16:0/749242242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c0102b10 msgr2=0x7f22c019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c55c1700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c0102b10 0x7f22c019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c55c1700 1 -- 172.21.15.16:0/749242242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c0100f80 msgr2=0x7f22c0196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c55c1700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c0100f80 0x7f22c0196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c55c1700 1 -- 172.21.15.16:0/749242242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00095f0 con 0x7f22c0102170 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c55c1700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c0197370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22b0000c00 tx=0x7f22b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b001d070 con 0x7f22c0102170 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22b0005030 con 0x7f22c0102170 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c019eeb0 con 0x7f22c0102170 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004b10 con 0x7f22c0102170 2024-05-08T23:18:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.037+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c019f2d0 con 0x7f22c0102170 2024-05-08T23:18:09.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.038+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c01083a0 con 0x7f22c0102170 2024-05-08T23:18:09.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.040+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f22b00051a0 con 0x7f22c0102170 2024-05-08T23:18:09.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.040+0000 7f22b6ffd700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22ac0a11a0 0x7f22ac0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.040+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f22b00cda90 con 0x7f22c0102170 2024-05-08T23:18:09.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.040+0000 7f22c5dc2700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22ac0a11a0 0x7f22ac0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.041+0000 7f22c5dc2700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22ac0a11a0 0x7f22ac0a3660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f22c0101fd0 tx=0x7f22bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.044+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22b00970c0 con 0x7f22c0102170 2024-05-08T23:18:09.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.314+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f22c0067cf0 con 0x7f22c0102170 2024-05-08T23:18:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.667+0000 7f22b6ffd700 1 -- 172.21.15.16:0/749242242 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f22b009a6e0 con 0x7f22c0102170 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22ac0a11a0 msgr2=0x7f22ac0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22ac0a11a0 0x7f22ac0a3660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f22c0101fd0 tx=0x7f22bc009260 comp rx=0 tx=0).stop 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 msgr2=0x7f22c0197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c0197370 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f22b0000c00 tx=0x7f22b000f690 comp rx=0 tx=0).stop 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 shutdown_connections 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22ac0a11a0 0x7f22ac0a3660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22c0102b10 0x7f22c019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c0100f80 0x7f22c0196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 --2- 172.21.15.16:0/749242242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22c0102170 0x7f22c0197370 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 >> 172.21.15.16:0/749242242 conn(0x7f22c00fc4f0 msgr2=0x7f22c01054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 shutdown_connections 2024-05-08T23:18:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.670+0000 7f22c8026700 1 -- 172.21.15.16:0/749242242 wait complete. 2024-05-08T23:18:09.686 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-08T23:18:09.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-08T23:18:09.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/895259334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0100f20 msgr2=0x7f8ee0101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/895259334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0100f20 0x7f8ee0101360 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009990 tx=0x7f8ed0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/895259334 shutdown_connections 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/895259334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102ab0 0x7f8ee01093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/895259334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0100f20 0x7f8ee0101360 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/895259334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ee01020f0 0x7f8ee0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.834+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/895259334 >> 172.21.15.16:0/895259334 conn(0x7f8ee00fc4e0 msgr2=0x7f8ee00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/895259334 shutdown_connections 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/895259334 wait complete. 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 Processor -- start 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 -- start start 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0100f20 0x7f8ee0196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ee01020f0 0x7f8ee0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 0x7f8ee019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee019e9f0 con 0x7f8ee0102ab0 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee019eb30 con 0x7f8ee01020f0 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee7e7d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee019ecc0 con 0x7f8ee0100f20 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.835+0000 7f8ee641a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 0x7f8ee019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 0x7f8ee019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2024-05-08T23:18:09.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 -- 172.21.15.16:0/4179078504 learned_addr learned my addr 172.21.15.16:0/4179078504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee5c19700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0100f20 0x7f8ee0196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee5418700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ee01020f0 0x7f8ee0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 -- 172.21.15.16:0/4179078504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0100f20 msgr2=0x7f8ee0196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0100f20 0x7f8ee0196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 -- 172.21.15.16:0/4179078504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ee01020f0 msgr2=0x7f8ee0197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ee01020f0 0x7f8ee0197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 -- 172.21.15.16:0/4179078504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00095f0 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee641a700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 0x7f8ee019e420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8007ee0 tx=0x7f8ed800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8017070 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ed800f040 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee019efa0 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8005710 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.836+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee019f420 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.837+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee0108480 con 0x7f8ee0102ab0 2024-05-08T23:18:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.839+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8ed8005870 con 0x7f8ee0102ab0 2024-05-08T23:18:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.839+0000 7f8ed6ffd700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8ecc0a1120 0x7f8ecc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.839+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8ed80cc840 con 0x7f8ee0102ab0 2024-05-08T23:18:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.839+0000 7f8ee5c19700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8ecc0a1120 0x7f8ecc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.840+0000 7f8ee5c19700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8ecc0a1120 0x7f8ecc0a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000b5c0 tx=0x7f8ed0003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:09.843+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8ed8095de0 con 0x7f8ee0102ab0 2024-05-08T23:18:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:09 smithi189 ceph-mon[46919]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:09 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2275078982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:09.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:09 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/749242242' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:18:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:09 smithi155 ceph-mon[47105]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:09 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2275078982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:09.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:09 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/749242242' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:18:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.113+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f8ee010c680 con 0x7f8ee0102ab0 2024-05-08T23:18:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:09 smithi016 ceph-mon[42541]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T23:18:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:09 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2275078982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:09 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/749242242' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T23:18:10.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.674+0000 7f8ed6ffd700 1 -- 172.21.15.16:0/4179078504 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8ed8099400 con 0x7f8ee0102ab0 2024-05-08T23:18:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8ecc0a1120 msgr2=0x7f8ecc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8ecc0a1120 0x7f8ecc0a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000b5c0 tx=0x7f8ed0003720 comp rx=0 tx=0).stop 2024-05-08T23:18:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 msgr2=0x7f8ee019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 0x7f8ee019e420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8007ee0 tx=0x7f8ed800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 shutdown_connections 2024-05-08T23:18:10.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee0102ab0 0x7f8ee019e420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8ecc0a1120 0x7f8ecc0a35e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0100f20 0x7f8ee0196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 --2- 172.21.15.16:0/4179078504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ee01020f0 0x7f8ee0197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 >> 172.21.15.16:0/4179078504 conn(0x7f8ee00fc4e0 msgr2=0x7f8ee01055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 shutdown_connections 2024-05-08T23:18:10.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.677+0000 7f8ee7e7d700 1 -- 172.21.15.16:0/4179078504 wait complete. 2024-05-08T23:18:10.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T23:18:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 -- 172.21.15.16:0/3511274744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328104380 msgr2=0x7fe328106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/3511274744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328104380 0x7fe328106770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe310009990 tx=0x7fe310009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 -- 172.21.15.16:0/3511274744 shutdown_connections 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/3511274744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe328106cb0 0x7fe3281090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/3511274744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328101a50 0x7fe328103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/3511274744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328104380 0x7fe328106770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 -- 172.21.15.16:0/3511274744 >> 172.21.15.16:0/3511274744 conn(0x7fe3280fb400 msgr2=0x7fe3280fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 -- 172.21.15.16:0/3511274744 shutdown_connections 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.840+0000 7fe32e8f7700 1 -- 172.21.15.16:0/3511274744 wait complete. 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 Processor -- start 2024-05-08T23:18:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 -- start start 2024-05-08T23:18:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 0x7fe328101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328104380 0x7fe3280ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe328106cb0 0x7fe3280fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281006e0 con 0x7fe328101a50 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe328100850 con 0x7fe328106cb0 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe32e8f7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281a0e40 con 0x7fe328104380 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.841+0000 7fe327fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 0x7fe328101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 0x7fe328101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34550/0 (socket says 172.21.15.16:34550) 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 -- 172.21.15.16:0/4270457741 learned_addr learned my addr 172.21.15.16:0/4270457741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe3277fe700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328104380 0x7fe3280ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe32ce94700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe328106cb0 0x7fe3280fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 -- 172.21.15.16:0/4270457741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328104380 msgr2=0x7fe3280ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328104380 0x7fe3280ffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 -- 172.21.15.16:0/4270457741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe328106cb0 msgr2=0x7fe3280fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe328106cb0 0x7fe3280fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 -- 172.21.15.16:0/4270457741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100095f0 con 0x7fe328101a50 2024-05-08T23:18:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe327fff700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 0x7fe328101370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9a0 tx=0x7fe31800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318009790 con 0x7fe328101a50 2024-05-08T23:18:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe318010430 con 0x7fe328101a50 2024-05-08T23:18:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3281a10d0 con 0x7fe328101a50 2024-05-08T23:18:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318018680 con 0x7fe328101a50 2024-05-08T23:18:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.842+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3281a15e0 con 0x7fe328101a50 2024-05-08T23:18:10.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.843+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3280fcf80 con 0x7fe328101a50 2024-05-08T23:18:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.844+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe3180105a0 con 0x7fe328101a50 2024-05-08T23:18:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.845+0000 7fe3257fa700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.845+0000 7fe3277fe700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.846+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe318014070 con 0x7fe328101a50 2024-05-08T23:18:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.846+0000 7fe3277fe700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3140a1120 0x7fe3140a35e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe310006010 tx=0x7fe31000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:10.850+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe318052d60 con 0x7fe328101a50 2024-05-08T23:18:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:10 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/749242242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:18:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:10 smithi155 ceph-mon[47105]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:18:10.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:10 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4179078504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T23:18:11.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/749242242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:18:11.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:10 smithi016 ceph-mon[42541]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:18:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4179078504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T23:18:11.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.123+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe32810c8e0 con 0x7fe328101a50 2024-05-08T23:18:11.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:10 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/749242242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T23:18:11.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:10 smithi189 ceph-mon[46919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:18:11.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:10 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4179078504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-08T23:18:11.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.681+0000 7fe3257fa700 1 -- 172.21.15.16:0/4270457741 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe3180962d0 con 0x7fe328101a50 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3140a1120 msgr2=0x7fe3140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3140a1120 0x7fe3140a35e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe310006010 tx=0x7fe31000b610 comp rx=0 tx=0).stop 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 msgr2=0x7fe328101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 0x7fe328101370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9a0 tx=0x7fe31800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 shutdown_connections 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe328106cb0 0x7fe3280fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe328101a50 0x7fe328101370 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 --2- 172.21.15.16:0/4270457741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe328104380 0x7fe3280ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 >> 172.21.15.16:0/4270457741 conn(0x7fe3280fb400 msgr2=0x7fe328102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 shutdown_connections 2024-05-08T23:18:11.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.684+0000 7fe32e8f7700 1 -- 172.21.15.16:0/4270457741 wait complete. 2024-05-08T23:18:11.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T23:18:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 -- 172.21.15.16:0/1045231232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044100f20 msgr2=0x7ff044101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/1045231232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044100f20 0x7ff044101360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff038009990 tx=0x7ff038009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 -- 172.21.15.16:0/1045231232 shutdown_connections 2024-05-08T23:18:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/1045231232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff044102ab0 0x7ff0441093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/1045231232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044100f20 0x7ff044101360 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/1045231232 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0441020f0 0x7ff044102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 -- 172.21.15.16:0/1045231232 >> 172.21.15.16:0/1045231232 conn(0x7ff0440fc4e0 msgr2=0x7ff0440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 -- 172.21.15.16:0/1045231232 shutdown_connections 2024-05-08T23:18:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.849+0000 7ff04c3ef700 1 -- 172.21.15.16:0/1045231232 wait complete. 2024-05-08T23:18:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 Processor -- start 2024-05-08T23:18:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 -- start start 2024-05-08T23:18:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff044100f20 0x7ff044196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0441020f0 0x7ff044197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 0x7ff04419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04419e8f0 con 0x7ff044102ab0 2024-05-08T23:18:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04419ea60 con 0x7ff044100f20 2024-05-08T23:18:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04c3ef700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04419ec20 con 0x7ff0441020f0 2024-05-08T23:18:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04a98c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 0x7ff04419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04a98c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 0x7ff04419e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34578/0 (socket says 172.21.15.16:34578) 2024-05-08T23:18:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04a98c700 1 -- 172.21.15.16:0/671004632 learned_addr learned my addr 172.21.15.16:0/671004632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04a18b700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff044100f20 0x7ff044196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.850+0000 7ff04998a700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0441020f0 0x7ff044197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04a98c700 1 -- 172.21.15.16:0/671004632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0441020f0 msgr2=0x7ff044197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04a98c700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0441020f0 0x7ff044197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04a98c700 1 -- 172.21.15.16:0/671004632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff044100f20 msgr2=0x7ff044196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:11.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04a98c700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff044100f20 0x7ff044196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:11.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04a98c700 1 -- 172.21.15.16:0/671004632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380095f0 con 0x7ff044102ab0 2024-05-08T23:18:11.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04a98c700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 0x7ff04419e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff03c007d70 tx=0x7ff03c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00ad40 con 0x7ff044102ab0 2024-05-08T23:18:11.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04419ef00 con 0x7ff044102ab0 2024-05-08T23:18:11.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff03c0128b0 con 0x7ff044102ab0 2024-05-08T23:18:11.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0115a0 con 0x7ff044102ab0 2024-05-08T23:18:11.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.851+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04419f320 con 0x7ff044102ab0 2024-05-08T23:18:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.852+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff044108480 con 0x7ff044102ab0 2024-05-08T23:18:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.853+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff03c012a20 con 0x7ff044102ab0 2024-05-08T23:18:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.854+0000 7ff0377fe700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff0300a1120 0x7ff0300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.854+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff03c0cd170 con 0x7ff044102ab0 2024-05-08T23:18:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.854+0000 7ff04a18b700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff0300a1120 0x7ff0300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.855+0000 7ff04a18b700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff0300a1120 0x7ff0300a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff03800b5c0 tx=0x7ff038003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:11.858+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff03c0966b0 con 0x7ff044102ab0 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4179078504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T23:18:11.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:11 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4270457741' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:18:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T23:18:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:18:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4179078504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T23:18:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:18:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:18:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T23:18:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T23:18:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:11 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4270457741' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:18:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.130+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ff044067460 con 0x7ff044102ab0 2024-05-08T23:18:12.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4179078504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-08T23:18:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:11 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4270457741' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T23:18:12.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.692+0000 7ff0377fe700 1 -- 172.21.15.16:0/671004632 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff03c099cd0 con 0x7ff044102ab0 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff0300a1120 msgr2=0x7ff0300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff0300a1120 0x7ff0300a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff03800b5c0 tx=0x7ff038003720 comp rx=0 tx=0).stop 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 msgr2=0x7ff04419e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 0x7ff04419e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff03c007d70 tx=0x7ff03c00a340 comp rx=0 tx=0).stop 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 shutdown_connections 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff044102ab0 0x7ff04419e320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff044100f20 0x7ff044196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff0300a1120 0x7ff0300a35e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 --2- 172.21.15.16:0/671004632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0441020f0 0x7ff044197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 >> 172.21.15.16:0/671004632 conn(0x7ff0440fc4e0 msgr2=0x7ff0441055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 shutdown_connections 2024-05-08T23:18:12.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.695+0000 7ff04c3ef700 1 -- 172.21.15.16:0/671004632 wait complete. 2024-05-08T23:18:12.711 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-08T23:18:12.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T23:18:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 -- 172.21.15.16:0/1860693032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 msgr2=0x7f57b4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/1860693032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b4101450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f579c009990 tx=0x7f579c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 -- 172.21.15.16:0/1860693032 shutdown_connections 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/1860693032 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4101990 0x7f57b41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/1860693032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b40ffe80 0x7f57b41002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/1860693032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b4101450 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.856+0000 7f57b8bc4700 1 -- 172.21.15.16:0/1860693032 >> 172.21.15.16:0/1860693032 conn(0x7f57b40fb3f0 msgr2=0x7f57b40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.857+0000 7f57b8bc4700 1 -- 172.21.15.16:0/1860693032 shutdown_connections 2024-05-08T23:18:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.857+0000 7f57b8bc4700 1 -- 172.21.15.16:0/1860693032 wait complete. 2024-05-08T23:18:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.857+0000 7f57b8bc4700 1 Processor -- start 2024-05-08T23:18:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 -- start start 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b40ffe80 0x7f57b4196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4101990 0x7f57b419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419e900 con 0x7f57b4100fd0 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419ea70 con 0x7f57b4101990 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b8bc4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b419ec30 con 0x7f57b40ffe80 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34586/0 (socket says 172.21.15.16:34586) 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 -- 172.21.15.16:0/2435481728 learned_addr learned my addr 172.21.15.16:0/2435481728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b2d9d700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4101990 0x7f57b419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57b259c700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b40ffe80 0x7f57b4196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 -- 172.21.15.16:0/2435481728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b40ffe80 msgr2=0x7f57b4196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b40ffe80 0x7f57b4196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 -- 172.21.15.16:0/2435481728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4101990 msgr2=0x7f57b419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4101990 0x7f57b419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 -- 172.21.15.16:0/2435481728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0095f0 con 0x7f57b4100fd0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.858+0000 7f57a9d9b700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b41973a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f579c0059d0 tx=0x7f579c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.859+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c01d070 con 0x7f57b4100fd0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.859+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f579c00be50 con 0x7f57b4100fd0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.859+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b419eeb0 con 0x7f57b4100fd0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.859+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c022e10 con 0x7f57b4100fd0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.859+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b419f220 con 0x7f57b4100fd0 2024-05-08T23:18:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.860+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b4067460 con 0x7f57b4100fd0 2024-05-08T23:18:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.861+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f579c022570 con 0x7f57b4100fd0 2024-05-08T23:18:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.862+0000 7f57abfff700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f57a00a1120 0x7f57a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.862+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f579c0ce600 con 0x7f57b4100fd0 2024-05-08T23:18:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.862+0000 7f57b259c700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f57a00a1120 0x7f57a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.862+0000 7f57b259c700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f57a00a1120 0x7f57a00a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57b4198290 tx=0x7f57a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:12.865+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f579c097c30 con 0x7f57b4100fd0 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] up:active 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4270457741' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: Cluster is now healthy 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T23:18:12.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:12 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/671004632' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:18:13.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] up:active 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4270457741' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: Cluster is now healthy 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T23:18:13.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/671004632' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:18:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.117+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b40698c0 con 0x7f57a00a1120 2024-05-08T23:18:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.138+0000 7f57abfff700 1 -- 172.21.15.16:0/2435481728 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f57b40698c0 con 0x7f57a00a1120 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f57a00a1120 msgr2=0x7f57a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f57a00a1120 0x7f57a00a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57b4198290 tx=0x7f57a4009260 comp rx=0 tx=0).stop 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 msgr2=0x7f57b41973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b41973a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f579c0059d0 tx=0x7f579c004710 comp rx=0 tx=0).stop 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 shutdown_connections 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57b4101990 0x7f57b419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f57a00a1120 0x7f57a00a35e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57b40ffe80 0x7f57b4196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 --2- 172.21.15.16:0/2435481728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57b4100fd0 0x7f57b41973a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 >> 172.21.15.16:0/2435481728 conn(0x7f57b40fb3f0 msgr2=0x7f57b40691b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 shutdown_connections 2024-05-08T23:18:13.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.141+0000 7f57b8bc4700 1 -- 172.21.15.16:0/2435481728 wait complete. 2024-05-08T23:18:13.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T23:18:13.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] up:active 2024-05-08T23:18:13.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4270457741' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: Cluster is now healthy 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T23:18:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:12 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/671004632' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T23:18:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 -- 172.21.15.16:0/2346919094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 msgr2=0x7fed54100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 --2- 172.21.15.16:0/2346919094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54100280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fed44009990 tx=0x7fed44009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 -- 172.21.15.16:0/2346919094 shutdown_connections 2024-05-08T23:18:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 --2- 172.21.15.16:0/2346919094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed54101970 0x7fed54108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 --2- 172.21.15.16:0/2346919094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54100280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 --2- 172.21.15.16:0/2346919094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54100fb0 0x7fed54101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 -- 172.21.15.16:0/2346919094 >> 172.21.15.16:0/2346919094 conn(0x7fed540fb3f0 msgr2=0x7fed540fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 -- 172.21.15.16:0/2346919094 shutdown_connections 2024-05-08T23:18:13.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.300+0000 7fed58a4a700 1 -- 172.21.15.16:0/2346919094 wait complete. 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 Processor -- start 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 -- start start 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed54100fb0 0x7fed541995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54101970 0x7fed541a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a0b90 con 0x7fed540ffe60 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a0cd0 con 0x7fed54100fb0 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed58a4a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541a0e60 con 0x7fed54101970 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34622/0 (socket says 172.21.15.16:34622) 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 -- 172.21.15.16:0/3503602932 learned_addr learned my addr 172.21.15.16:0/3503602932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed52d9d700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54101970 0x7fed541a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed51d9b700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed54100fb0 0x7fed541995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 -- 172.21.15.16:0/3503602932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54101970 msgr2=0x7fed541a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54101970 0x7fed541a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 -- 172.21.15.16:0/3503602932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed54100fb0 msgr2=0x7fed541995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.301+0000 7fed5259c700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed54100fb0 0x7fed541995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed5259c700 1 -- 172.21.15.16:0/3503602932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed440095f0 con 0x7fed540ffe60 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed52d9d700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54101970 0x7fed541a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed5259c700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54199060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fed4400b5c0 tx=0x7fed44003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4401d070 con 0x7fed540ffe60 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed541a10e0 con 0x7fed540ffe60 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed4400bd10 con 0x7fed540ffe60 2024-05-08T23:18:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44022dc0 con 0x7fed540ffe60 2024-05-08T23:18:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.302+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed541a1510 con 0x7fed540ffe60 2024-05-08T23:18:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.303+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed54067cf0 con 0x7fed540ffe60 2024-05-08T23:18:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.304+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fed44003ff0 con 0x7fed540ffe60 2024-05-08T23:18:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.305+0000 7fed437fe700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed3c0a1120 0x7fed3c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.305+0000 7fed51d9b700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed3c0a1120 0x7fed3c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.305+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fed440cdb70 con 0x7fed540ffe60 2024-05-08T23:18:13.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.305+0000 7fed51d9b700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed3c0a1120 0x7fed3c0a35e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed48005950 tx=0x7fed480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.309+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fed44097140 con 0x7fed540ffe60 2024-05-08T23:18:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.569+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fed540687b0 con 0x7fed3c0a1120 2024-05-08T23:18:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.591+0000 7fed437fe700 1 -- 172.21.15.16:0/3503602932 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fed540687b0 con 0x7fed3c0a1120 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed3c0a1120 msgr2=0x7fed3c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed3c0a1120 0x7fed3c0a35e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed48005950 tx=0x7fed480058e0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 msgr2=0x7fed54199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54199060 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fed4400b5c0 tx=0x7fed44003b00 comp rx=0 tx=0).stop 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 shutdown_connections 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed3c0a1120 0x7fed3c0a35e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed54101970 0x7fed541a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540ffe60 0x7fed54199060 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 --2- 172.21.15.16:0/3503602932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed54100fb0 0x7fed541995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 >> 172.21.15.16:0/3503602932 conn(0x7fed540fb3f0 msgr2=0x7fed540680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 shutdown_connections 2024-05-08T23:18:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.594+0000 7fed58a4a700 1 -- 172.21.15.16:0/3503602932 wait complete. 2024-05-08T23:18:13.606 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:18:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:18:13.122+0000 7fe4d1b29700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:18:13.607 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:18:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:18:13.122+0000 7fe4d1b29700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:18:13.607 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:18:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:18:13.122+0000 7fe4d1b29700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:18:13.607 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:18:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:18:13.122+0000 7fe4d1b29700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:18:13.607 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:18:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:18:13.122+0000 7fe4d1b29700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:18:13.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T23:18:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 -- 172.21.15.16:0/194412273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 msgr2=0x7f2288101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 --2- 172.21.15.16:0/194412273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288101450 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2278009990 tx=0x7f2278009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 -- 172.21.15.16:0/194412273 shutdown_connections 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 --2- 172.21.15.16:0/194412273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2288101990 0x7f22881082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 --2- 172.21.15.16:0/194412273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22880ffe80 0x7f22881002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 --2- 172.21.15.16:0/194412273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288101450 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 -- 172.21.15.16:0/194412273 >> 172.21.15.16:0/194412273 conn(0x7f22880fb3f0 msgr2=0x7f22880fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 -- 172.21.15.16:0/194412273 shutdown_connections 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 -- 172.21.15.16:0/194412273 wait complete. 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.753+0000 7f228f881700 1 Processor -- start 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 -- start start 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22880ffe80 0x7f2288196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2288101990 0x7f228819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228819e970 con 0x7f2288100fd0 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228819eab0 con 0x7f2288101990 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228f881700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228819ec40 con 0x7f22880ffe80 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34640/0 (socket says 172.21.15.16:34640) 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 -- 172.21.15.16:0/3484894681 learned_addr learned my addr 172.21.15.16:0/3484894681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228d61d700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22880ffe80 0x7f2288196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228de1e700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2288101990 0x7f228819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 -- 172.21.15.16:0/3484894681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22880ffe80 msgr2=0x7f2288196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22880ffe80 0x7f2288196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 -- 172.21.15.16:0/3484894681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2288101990 msgr2=0x7f228819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2288101990 0x7f228819e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 -- 172.21.15.16:0/3484894681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780095f0 con 0x7f2288100fd0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.754+0000 7f228ce1c700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288197380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f227800b5c0 tx=0x7f2278003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.755+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227801d070 con 0x7f2288100fd0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.755+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f227800bd10 con 0x7f2288100fd0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.755+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228819eec0 con 0x7f2288100fd0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.755+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278022d20 con 0x7f2288100fd0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.755+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228819f280 con 0x7f2288100fd0 2024-05-08T23:18:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.756+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2288067460 con 0x7f2288100fd0 2024-05-08T23:18:13.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.757+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2278003f40 con 0x7f2288100fd0 2024-05-08T23:18:13.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.757+0000 7f227e7fc700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22740a1120 0x7f22740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.758+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f22780cde90 con 0x7f2288100fd0 2024-05-08T23:18:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.758+0000 7f228d61d700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22740a1120 0x7f22740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.758+0000 7f228d61d700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22740a1120 0x7f22740a35e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2284005950 tx=0x7f228400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:13.761+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22780974c0 con 0x7f2288100fd0 2024-05-08T23:18:13.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:13 smithi016 ceph-mon[42541]: pgmap v236: 129 pgs: 129 active+clean; 582 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-05-08T23:18:13.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:13 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] up:active 2024-05-08T23:18:13.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:13 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/671004632' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:18:13.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:13 smithi016 ceph-mon[42541]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-08T23:18:13.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:13 smithi016 ceph-mon[42541]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:18:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:13 smithi155 ceph-mon[47105]: pgmap v236: 129 pgs: 129 active+clean; 582 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-05-08T23:18:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:13 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] up:active 2024-05-08T23:18:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:13 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/671004632' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:18:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:13 smithi155 ceph-mon[47105]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-08T23:18:13.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:13 smithi155 ceph-mon[47105]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:18:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.019+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f22880698d0 con 0x7f22740a1120 2024-05-08T23:18:14.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.023+0000 7f227e7fc700 1 -- 172.21.15.16:0/3484894681 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f22880698d0 con 0x7f22740a1120 2024-05-08T23:18:14.023 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/9ee67809-27ca-4adc-9b10-dce1a6e369ab 2024-05-08T23:18:14.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22740a1120 msgr2=0x7f22740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22740a1120 0x7f22740a35e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2284005950 tx=0x7f228400b410 comp rx=0 tx=0).stop 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 msgr2=0x7f2288197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288197380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f227800b5c0 tx=0x7f2278003a50 comp rx=0 tx=0).stop 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 shutdown_connections 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2288101990 0x7f228819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f22740a1120 0x7f22740a35e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22880ffe80 0x7f2288196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 --2- 172.21.15.16:0/3484894681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2288100fd0 0x7f2288197380 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 >> 172.21.15.16:0/3484894681 conn(0x7f22880fb3f0 msgr2=0x7f22880691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 shutdown_connections 2024-05-08T23:18:14.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.026+0000 7f228f881700 1 -- 172.21.15.16:0/3484894681 wait complete. 2024-05-08T23:18:14.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T23:18:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:13 smithi189 ceph-mon[46919]: pgmap v236: 129 pgs: 129 active+clean; 582 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-05-08T23:18:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:13 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] up:active 2024-05-08T23:18:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:13 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/671004632' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T23:18:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:13 smithi189 ceph-mon[46919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-08T23:18:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:13 smithi189 ceph-mon[46919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 -- 172.21.15.16:0/886685498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102b10 msgr2=0x7f2e401093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 --2- 172.21.15.16:0/886685498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102b10 0x7f2e401093c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800b0d0 tx=0x7f2e3800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 -- 172.21.15.16:0/886685498 shutdown_connections 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 --2- 172.21.15.16:0/886685498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102b10 0x7f2e401093c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 --2- 172.21.15.16:0/886685498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40100f80 0x7f2e401013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 --2- 172.21.15.16:0/886685498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e40102170 0x7f2e401025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 -- 172.21.15.16:0/886685498 >> 172.21.15.16:0/886685498 conn(0x7f2e400fc4f0 msgr2=0x7f2e400fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 -- 172.21.15.16:0/886685498 shutdown_connections 2024-05-08T23:18:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.185+0000 7f2e4797a700 1 -- 172.21.15.16:0/886685498 wait complete. 2024-05-08T23:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 Processor -- start 2024-05-08T23:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 -- start start 2024-05-08T23:18:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e40100f80 0x7f2e40074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 0x7f2e40073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40102b10 0x7f2e400738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40074070 con 0x7f2e40102170 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400741e0 con 0x7f2e40102b10 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e4797a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401a0f40 con 0x7f2e40100f80 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e44f15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 0x7f2e40073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e44f15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 0x7f2e40073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34646/0 (socket says 172.21.15.16:34646) 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e44f15700 1 -- 172.21.15.16:0/1768407652 learned_addr learned my addr 172.21.15.16:0/1768407652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.186+0000 7f2e45716700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e40100f80 0x7f2e40074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e45f17700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40102b10 0x7f2e400738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e44f15700 1 -- 172.21.15.16:0/1768407652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e40100f80 msgr2=0x7f2e40074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e44f15700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e40100f80 0x7f2e40074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e44f15700 1 -- 172.21.15.16:0/1768407652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40102b10 msgr2=0x7f2e400738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e44f15700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40102b10 0x7f2e400738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e44f15700 1 -- 172.21.15.16:0/1768407652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009d00 con 0x7f2e40102170 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e45f17700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40102b10 0x7f2e400738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e44f15700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 0x7f2e40073390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00d950 tx=0x7f2e3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0098e0 con 0x7f2e40102170 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e3c010460 con 0x7f2e40102170 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e401a11d0 con 0x7f2e40102170 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00f620 con 0x7f2e40102170 2024-05-08T23:18:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.187+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e401a1730 con 0x7f2e40102170 2024-05-08T23:18:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.188+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e401083d0 con 0x7f2e40102170 2024-05-08T23:18:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.189+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2e3c0105d0 con 0x7f2e40102170 2024-05-08T23:18:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.190+0000 7f2e327fc700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2e2c0a11a0 0x7f2e2c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.190+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2e3c0ccbb0 con 0x7f2e40102170 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.190+0000 7f2e45716700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2e2c0a11a0 0x7f2e2c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.191+0000 7f2e45716700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2e2c0a11a0 0x7f2e2c0a3660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2e40101fd0 tx=0x7f2e3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.195+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2e3c096150 con 0x7f2e40102170 2024-05-08T23:18:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.452+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e400613e0 con 0x7f2e2c0a11a0 2024-05-08T23:18:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.476+0000 7f2e327fc700 1 -- 172.21.15.16:0/1768407652 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2e400613e0 con 0x7f2e2c0a11a0 2024-05-08T23:18:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2e2c0a11a0 msgr2=0x7f2e2c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2e2c0a11a0 0x7f2e2c0a3660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2e40101fd0 tx=0x7f2e3400b410 comp rx=0 tx=0).stop 2024-05-08T23:18:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 msgr2=0x7f2e40073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 0x7f2e40073390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00d950 tx=0x7f2e3c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 shutdown_connections 2024-05-08T23:18:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e40102b10 0x7f2e400738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2e2c0a11a0 0x7f2e2c0a3660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e40100f80 0x7f2e40074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 --2- 172.21.15.16:0/1768407652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e40102170 0x7f2e40073390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 >> 172.21.15.16:0/1768407652 conn(0x7f2e400fc4f0 msgr2=0x7f2e40105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 shutdown_connections 2024-05-08T23:18:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.478+0000 7f2e4797a700 1 -- 172.21.15.16:0/1768407652 wait complete. 2024-05-08T23:18:14.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 -- 172.21.15.16:0/1644189673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 msgr2=0x7fd79c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/1644189673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c101430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd784009990 tx=0x7fd784009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 -- 172.21.15.16:0/1644189673 shutdown_connections 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/1644189673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79c101970 0x7fd79c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/1644189673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c0ffe60 0x7fd79c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/1644189673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c101430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 -- 172.21.15.16:0/1644189673 >> 172.21.15.16:0/1644189673 conn(0x7fd79c0fb3f0 msgr2=0x7fd79c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 -- 172.21.15.16:0/1644189673 shutdown_connections 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 -- 172.21.15.16:0/1644189673 wait complete. 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.643+0000 7fd7a23d9700 1 Processor -- start 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 -- start start 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79c0ffe60 0x7fd79c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c101970 0x7fd79c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c19e8a0 con 0x7fd79c100fb0 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c19ea10 con 0x7fd79c0ffe60 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a23d9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c19ebd0 con 0x7fd79c101970 2024-05-08T23:18:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79c0ffe60 0x7fd79c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34664/0 (socket says 172.21.15.16:34664) 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 -- 172.21.15.16:0/130691785 learned_addr learned my addr 172.21.15.16:0/130691785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd7a0976700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c101970 0x7fd79c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 -- 172.21.15.16:0/130691785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c101970 msgr2=0x7fd79c19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c101970 0x7fd79c19e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 -- 172.21.15.16:0/130691785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79c0ffe60 msgr2=0x7fd79c068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79c0ffe60 0x7fd79c068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.644+0000 7fd79b7fe700 1 -- 172.21.15.16:0/130691785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840095f0 con 0x7fd79c100fb0 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.645+0000 7fd79b7fe700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c19bb10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd784006010 tx=0x7fd784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.645+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78401d070 con 0x7fd79c100fb0 2024-05-08T23:18:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.645+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd784003f80 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.645+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c19ee50 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.645+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784022bc0 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.645+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c19f270 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.646+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c107320 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.647+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd784003af0 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.648+0000 7fd7997fa700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7880a11a0 0x7fd7880a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.648+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd7840ce720 con 0x7fd79c100fb0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.648+0000 7fd79bfff700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7880a11a0 0x7fd7880a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.648+0000 7fd79bfff700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7880a11a0 0x7fd7880a3660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005950 tx=0x7fd78c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.651+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd784097d50 con 0x7fd79c100fb0 2024-05-08T23:18:14.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.710+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd784098120 con 0x7fd79c100fb0 2024-05-08T23:18:14.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:14 smithi016 ceph-mon[42541]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:14.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:14 smithi016 ceph-mon[42541]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.920+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd79c067cc0 con 0x7fd7880a11a0 2024-05-08T23:18:14.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.925+0000 7fd7997fa700 1 -- 172.21.15.16:0/130691785 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fd79c067cc0 con 0x7fd7880a11a0 2024-05-08T23:18:14.925 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/0b0377b7-27cb-4389-b74d-b796e175604f 2024-05-08T23:18:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.927+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7880a11a0 msgr2=0x7fd7880a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.927+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7880a11a0 0x7fd7880a3660 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd78c005950 tx=0x7fd78c00b410 comp rx=0 tx=0).stop 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.927+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 msgr2=0x7fd79c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.927+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c19bb10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd784006010 tx=0x7fd784004730 comp rx=0 tx=0).stop 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 shutdown_connections 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7880a11a0 0x7fd7880a3660 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c101970 0x7fd79c19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79c0ffe60 0x7fd79c068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 --2- 172.21.15.16:0/130691785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd79c100fb0 0x7fd79c19bb10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 >> 172.21.15.16:0/130691785 conn(0x7fd79c0fb3f0 msgr2=0x7fd79c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 shutdown_connections 2024-05-08T23:18:14.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:14.928+0000 7fd7a23d9700 1 -- 172.21.15.16:0/130691785 wait complete. 2024-05-08T23:18:14.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:18:14.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:14 smithi155 ceph-mon[47105]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:14.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:14 smithi155 ceph-mon[47105]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 -- 172.21.15.16:0/4106439258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70381018e0 msgr2=0x7f7038103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 --2- 172.21.15.16:0/4106439258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70381018e0 0x7f7038103cd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7028009990 tx=0x7f7028009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 -- 172.21.15.16:0/4106439258 shutdown_connections 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 --2- 172.21.15.16:0/4106439258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7038106b40 0x7f7038108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 --2- 172.21.15.16:0/4106439258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70381018e0 0x7f7038103cd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 --2- 172.21.15.16:0/4106439258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7038104210 0x7f7038106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 -- 172.21.15.16:0/4106439258 >> 172.21.15.16:0/4106439258 conn(0x7f70380fb2f0 msgr2=0x7f70380fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 -- 172.21.15.16:0/4106439258 shutdown_connections 2024-05-08T23:18:15.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.081+0000 7f703e81d700 1 -- 172.21.15.16:0/4106439258 wait complete. 2024-05-08T23:18:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.082+0000 7f703e81d700 1 Processor -- start 2024-05-08T23:18:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.082+0000 7f703e81d700 1 -- start start 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703e81d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70381018e0 0x7f7038196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703e81d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7038104210 0x7f7038197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703e81d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 0x7f703819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703e81d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703819e800 con 0x7f7038106b40 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703e81d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703819e970 con 0x7f70381018e0 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703e81d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703819eb30 con 0x7f7038104210 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 0x7f703819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 0x7f703819e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34674/0 (socket says 172.21.15.16:34674) 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 -- 172.21.15.16:0/3420945515 learned_addr learned my addr 172.21.15.16:0/3420945515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f70377fe700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7038104210 0x7f7038197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f7037fff700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70381018e0 0x7f7038196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 -- 172.21.15.16:0/3420945515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7038104210 msgr2=0x7f7038197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7038104210 0x7f7038197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 -- 172.21.15.16:0/3420945515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70381018e0 msgr2=0x7f7038196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.083+0000 7f703cdba700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70381018e0 0x7f7038196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f703cdba700 1 -- 172.21.15.16:0/3420945515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70280095f0 con 0x7f7038106b40 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f703cdba700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 0x7f703819e230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7030007ee0 tx=0x7f703000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030017070 con 0x7f7038106b40 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f703000f040 con 0x7f7038106b40 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703819ee10 con 0x7f7038106b40 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030005570 con 0x7f7038106b40 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.084+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703819f290 con 0x7f7038106b40 2024-05-08T23:18:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.085+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70380fce70 con 0x7f7038106b40 2024-05-08T23:18:15.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.086+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f703000db20 con 0x7f7038106b40 2024-05-08T23:18:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.087+0000 7f70357fa700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f70200a1230 0x7f70200a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.087+0000 7f7037fff700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f70200a1230 0x7f70200a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.087+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f70300cc2d0 con 0x7f7038106b40 2024-05-08T23:18:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.087+0000 7f7037fff700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f70200a1230 0x7f70200a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f7028005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.091+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7030095710 con 0x7f7038106b40 2024-05-08T23:18:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:14 smithi189 ceph-mon[46919]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:14 smithi189 ceph-mon[46919]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.362+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7038067c10 con 0x7f7038106b40 2024-05-08T23:18:15.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.363+0000 7f70357fa700 1 -- 172.21.15.16:0/3420945515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f7030098d30 con 0x7f7038106b40 2024-05-08T23:18:15.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:15.363 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:18:12.692850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:18:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f70200a1230 msgr2=0x7f70200a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f70200a1230 0x7f70200a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f7028005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 msgr2=0x7f703819e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 0x7f703819e230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7030007ee0 tx=0x7f703000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 shutdown_connections 2024-05-08T23:18:15.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7038106b40 0x7f703819e230 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70381018e0 0x7f7038196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f70200a1230 0x7f70200a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 --2- 172.21.15.16:0/3420945515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7038104210 0x7f7038197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 >> 172.21.15.16:0/3420945515 conn(0x7f70380fb2f0 msgr2=0x7f70381037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.365+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 shutdown_connections 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.366+0000 7f703e81d700 1 -- 172.21.15.16:0/3420945515 wait complete. 2024-05-08T23:18:15.366 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:18:15.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 -- 172.21.15.16:0/3565220830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb41029a0 msgr2=0x7f0cb41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 --2- 172.21.15.16:0/3565220830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb41029a0 0x7f0cb41092c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00b0a0 tx=0x7f0cac00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 -- 172.21.15.16:0/3565220830 shutdown_connections 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 --2- 172.21.15.16:0/3565220830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb41029a0 0x7f0cb41092c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 --2- 172.21.15.16:0/3565220830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb4100e90 0x7f0cb41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 --2- 172.21.15.16:0/3565220830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 0x7f0cb4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 -- 172.21.15.16:0/3565220830 >> 172.21.15.16:0/3565220830 conn(0x7f0cb40fc400 msgr2=0x7f0cb40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.518+0000 7f0cb9860700 1 -- 172.21.15.16:0/3565220830 shutdown_connections 2024-05-08T23:18:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.519+0000 7f0cb9860700 1 -- 172.21.15.16:0/3565220830 wait complete. 2024-05-08T23:18:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.519+0000 7f0cb9860700 1 Processor -- start 2024-05-08T23:18:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.519+0000 7f0cb9860700 1 -- start start 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 0x7f0cb4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 0x7f0cb4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb41029a0 0x7f0cb40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4074070 con 0x7f0cb4100e90 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40741e0 con 0x7f0cb41029a0 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41a0e30 con 0x7f0cb4101fe0 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 0x7f0cb4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 0x7f0cb4074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34702/0 (socket says 172.21.15.16:34702) 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 -- 172.21.15.16:0/2364987466 learned_addr learned my addr 172.21.15.16:0/2364987466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb27fc700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 0x7f0cb4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb37fe700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb41029a0 0x7f0cb40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 -- 172.21.15.16:0/2364987466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 msgr2=0x7f0cb4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 0x7f0cb4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 -- 172.21.15.16:0/2364987466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb41029a0 msgr2=0x7f0cb40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb41029a0 0x7f0cb40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 -- 172.21.15.16:0/2364987466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca8009520 con 0x7f0cb4100e90 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb37fe700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb41029a0 0x7f0cb40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb27fc700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 0x7f0cb4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb2ffd700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 0x7f0cb4074d00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400d980 tx=0x7f0ca400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4004220 con 0x7f0cb4100e90 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cac009d00 con 0x7f0cb4100e90 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ca4007640 con 0x7f0cb4100e90 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.520+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40055e0 con 0x7f0cb4100e90 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.521+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb41a1440 con 0x7f0cb4100e90 2024-05-08T23:18:15.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.521+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb4108300 con 0x7f0cb4100e90 2024-05-08T23:18:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.523+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0ca4004750 con 0x7f0cb4100e90 2024-05-08T23:18:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.523+0000 7f0cb885e700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c9c0a11b0 0x7f0c9c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.524+0000 7f0cb27fc700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c9c0a11b0 0x7f0c9c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.524+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0ca40ccfd0 con 0x7f0cb4100e90 2024-05-08T23:18:15.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.524+0000 7f0cb27fc700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c9c0a11b0 0x7f0c9c0a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8011630 tx=0x7f0ca8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:15.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.527+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0ca40963b0 con 0x7f0cb4100e90 2024-05-08T23:18:15.701 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: 2024-05-08T23:18:15.449+0000 7f9a4cd7c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T23:18:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.752+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cb4067c10 con 0x7f0cb4100e90 2024-05-08T23:18:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.754+0000 7f0cb885e700 1 -- 172.21.15.16:0/2364987466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f0ca4016020 con 0x7f0cb4100e90 2024-05-08T23:18:15.754 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:15.754 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:18:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c9c0a11b0 msgr2=0x7f0c9c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c9c0a11b0 0x7f0c9c0a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8011630 tx=0x7f0ca8009260 comp rx=0 tx=0).stop 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 msgr2=0x7f0cb4074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 0x7f0cb4074d00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400d980 tx=0x7f0ca400dc90 comp rx=0 tx=0).stop 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 shutdown_connections 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cb41029a0 0x7f0cb40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb4100e90 0x7f0cb4074d00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c9c0a11b0 0x7f0c9c0a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 --2- 172.21.15.16:0/2364987466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cb4101fe0 0x7f0cb4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 >> 172.21.15.16:0/2364987466 conn(0x7f0cb40fc400 msgr2=0x7f0cb4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 shutdown_connections 2024-05-08T23:18:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.757+0000 7f0cb9860700 1 -- 172.21.15.16:0/2364987466 wait complete. 2024-05-08T23:18:15.771 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T23:18:15.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:18:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 -- 172.21.15.16:0/1871624346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a080ffd60 msgr2=0x7f2a08100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/1871624346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a080ffd60 0x7f2a08100180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0099c0 tx=0x7f29fc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 -- 172.21.15.16:0/1871624346 shutdown_connections 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/1871624346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a081018f0 0x7f2a081081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/1871624346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a080ffd60 0x7f2a08100180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/1871624346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a08100f50 0x7f2a081013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.909+0000 7f2a0ee45700 1 -- 172.21.15.16:0/1871624346 >> 172.21.15.16:0/1871624346 conn(0x7f2a080fb2f0 msgr2=0x7f2a080fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 -- 172.21.15.16:0/1871624346 shutdown_connections 2024-05-08T23:18:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 -- 172.21.15.16:0/1871624346 wait complete. 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 Processor -- start 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 -- start start 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a080ffd60 0x7f2a08196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 0x7f2a08197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a081018f0 0x7f2a0819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e830 con 0x7f2a08100f50 2024-05-08T23:18:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e9a0 con 0x7f2a080ffd60 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.910+0000 7f2a0ee45700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819eb60 con 0x7f2a081018f0 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 0x7f2a08197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 0x7f2a08197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34718/0 (socket says 172.21.15.16:34718) 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 -- 172.21.15.16:0/2064095490 learned_addr learned my addr 172.21.15.16:0/2064095490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a0cbe1700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a080ffd60 0x7f2a08196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a0d3e2700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a081018f0 0x7f2a0819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 -- 172.21.15.16:0/2064095490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a081018f0 msgr2=0x7f2a0819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a081018f0 0x7f2a0819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 -- 172.21.15.16:0/2064095490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a080ffd60 msgr2=0x7f2a08196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a080ffd60 0x7f2a08196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 -- 172.21.15.16:0/2064095490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc0095f0 con 0x7f2a08100f50 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a07fff700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 0x7f2a08197250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29f400d980 tx=0x7f29f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004220 con 0x7f2a08100f50 2024-05-08T23:18:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29f4007640 con 0x7f2a08100f50 2024-05-08T23:18:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0819ee40 con 0x7f2a08100f50 2024-05-08T23:18:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.911+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40055e0 con 0x7f2a08100f50 2024-05-08T23:18:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.912+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0819f270 con 0x7f2a08100f50 2024-05-08T23:18:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.912+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08107280 con 0x7f2a08100f50 2024-05-08T23:18:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.914+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f29f4004750 con 0x7f2a08100f50 2024-05-08T23:18:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.914+0000 7f2a05ffb700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29f80a11b0 0x7f29f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.914+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f29f40ccf70 con 0x7f2a08100f50 2024-05-08T23:18:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.914+0000 7f2a0cbe1700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29f80a11b0 0x7f29f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.915+0000 7f2a0cbe1700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29f80a11b0 0x7f29f80a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29fc000c00 tx=0x7f29fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:15.918+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f29f40963b0 con 0x7f2a08100f50 2024-05-08T23:18:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-mon[47105]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-mon[47105]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-mon[47105]: pgmap v237: 129 pgs: 129 active+clean; 584 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-05-08T23:18:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-mon[47105]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T23:18:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-mon[47105]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:15 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3420945515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:16.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:15 smithi016 ceph-mon[42541]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:15 smithi016 ceph-mon[42541]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:15 smithi016 ceph-mon[42541]: pgmap v237: 129 pgs: 129 active+clean; 584 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-05-08T23:18:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:15 smithi016 ceph-mon[42541]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T23:18:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:15 smithi016 ceph-mon[42541]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:15 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3420945515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:16.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.134+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a0802ceb0 con 0x7f29f80a11b0 2024-05-08T23:18:16.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.148+0000 7f2a05ffb700 1 -- 172.21.15.16:0/2064095490 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418791 (secure 0 0 0) 0x7f2a0802ceb0 con 0x7f29f80a11b0 2024-05-08T23:18:16.148 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29f80a11b0 msgr2=0x7f29f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29f80a11b0 0x7f29f80a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29fc000c00 tx=0x7f29fc01a040 comp rx=0 tx=0).stop 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 msgr2=0x7f2a08197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 0x7f2a08197250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29f400d980 tx=0x7f29f400dc90 comp rx=0 tx=0).stop 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 shutdown_connections 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f29f80a11b0 0x7f29f80a3670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a081018f0 0x7f2a0819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2a080ffd60 0x7f2a08196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 --2- 172.21.15.16:0/2064095490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a08100f50 0x7f2a08197250 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 >> 172.21.15.16:0/2064095490 conn(0x7f2a080fb2f0 msgr2=0x7f2a081043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 shutdown_connections 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.152+0000 7f2a0ee45700 1 -- 172.21.15.16:0/2064095490 wait complete. 2024-05-08T23:18:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:18:16.166 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-05-08T23:18:14.572043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597705,"num_objects":55,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":293,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389860,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":377325,"kb_avail":1124551964,"statfs":{"total":1151940427776,"available":1151541211136,"internally_reserved":0,"allocated":12689408,"data_stored":7067208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":386381128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2884,"num_objects":31,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":65,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.112156+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:17:50.112156+0000","last_peered":"2024-05-08T23:17:50.112156+0000","last_clean":"2024-05-08T23:17:50.112156+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:17:50.112156+0000","last_undegraded":"2024-05-08T23:17:50.112156+0000","last_fullsized":"2024-05-08T23:17:50.112156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116529+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:17:50.116529+0000","last_peered":"2024-05-08T23:17:50.116529+0000","last_clean":"2024-05-08T23:17:50.116529+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:17:50.116529+0000","last_undegraded":"2024-05-08T23:17:50.116529+0000","last_fullsized":"2024-05-08T23:17:50.116529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117104+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:17:50.117104+0000","last_peered":"2024-05-08T23:17:50.117104+0000","last_clean":"2024-05-08T23:17:50.117104+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:17:50.117104+0000","last_undegraded":"2024-05-08T23:17:50.117104+0000","last_fullsized":"2024-05-08T23:17:50.117104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118325+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:17:50.118325+0000","last_peered":"2024-05-08T23:17:50.118325+0000","last_clean":"2024-05-08T23:17:50.118325+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:17:50.118325+0000","last_undegraded":"2024-05-08T23:17:50.118325+0000","last_fullsized":"2024-05-08T23:17:50.118325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115828+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:17:50.115828+0000","last_peered":"2024-05-08T23:17:50.115828+0000","last_clean":"2024-05-08T23:17:50.115828+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:17:50.115828+0000","last_undegraded":"2024-05-08T23:17:50.115828+0000","last_fullsized":"2024-05-08T23:17:50.115828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.242862+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:17:50.242862+0000","last_peered":"2024-05-08T23:17:50.242862+0000","last_clean":"2024-05-08T23:17:50.242862+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:17:50.242862+0000","last_undegraded":"2024-05-08T23:17:50.242862+0000","last_fullsized":"2024-05-08T23:17:50.242862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120386+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:17:50.120386+0000","last_peered":"2024-05-08T23:17:50.120386+0000","last_clean":"2024-05-08T23:17:50.120386+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:17:50.120386+0000","last_undegraded":"2024-05-08T23:17:50.120386+0000","last_fullsized":"2024-05-08T23:17:50.120386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118752+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:17:50.118752+0000","last_peered":"2024-05-08T23:17:50.118752+0000","last_clean":"2024-05-08T23:17:50.118752+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:17:50.118752+0000","last_undegraded":"2024-05-08T23:17:50.118752+0000","last_fullsized":"2024-05-08T23:17:50.118752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114352+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:17:50.114352+0000","last_peered":"2024-05-08T23:17:50.114352+0000","last_clean":"2024-05-08T23:17:50.114352+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:17:50.114352+0000","last_undegraded":"2024-05-08T23:17:50.114352+0000","last_fullsized":"2024-05-08T23:17:50.114352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.243355+0000","last_change":"2024-05-08T23:17:46.107981+0000","last_active":"2024-05-08T23:17:50.243355+0000","last_peered":"2024-05-08T23:17:50.243355+0000","last_clean":"2024-05-08T23:17:50.243355+0000","last_became_active":"2024-05-08T23:17:46.107334+0000","last_became_peered":"2024-05-08T23:17:46.107334+0000","last_unstale":"2024-05-08T23:17:50.243355+0000","last_undegraded":"2024-05-08T23:17:50.243355+0000","last_fullsized":"2024-05-08T23:17:50.243355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.409548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120911+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:17:50.120911+0000","last_peered":"2024-05-08T23:17:50.120911+0000","last_clean":"2024-05-08T23:17:50.120911+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:17:50.120911+0000","last_undegraded":"2024-05-08T23:17:50.120911+0000","last_fullsized":"2024-05-08T23:17:50.120911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118236+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:17:50.118236+0000","last_peered":"2024-05-08T23:17:50.118236+0000","last_clean":"2024-05-08T23:17:50.118236+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:17:50.118236+0000","last_undegraded":"2024-05-08T23:17:50.118236+0000","last_fullsized":"2024-05-08T23:17:50.118236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.127869+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:17:50.127869+0000","last_peered":"2024-05-08T23:17:50.127869+0000","last_clean":"2024-05-08T23:17:50.127869+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:17:50.127869+0000","last_undegraded":"2024-05-08T23:17:50.127869+0000","last_fullsized":"2024-05-08T23:17:50.127869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116213+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:17:50.116213+0000","last_peered":"2024-05-08T23:17:50.116213+0000","last_clean":"2024-05-08T23:17:50.116213+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:17:50.116213+0000","last_undegraded":"2024-05-08T23:17:50.116213+0000","last_fullsized":"2024-05-08T23:17:50.116213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116083+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:17:50.116083+0000","last_peered":"2024-05-08T23:17:50.116083+0000","last_clean":"2024-05-08T23:17:50.116083+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:17:50.116083+0000","last_undegraded":"2024-05-08T23:17:50.116083+0000","last_fullsized":"2024-05-08T23:17:50.116083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114209+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:17:50.114209+0000","last_peered":"2024-05-08T23:17:50.114209+0000","last_clean":"2024-05-08T23:17:50.114209+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:17:50.114209+0000","last_undegraded":"2024-05-08T23:17:50.114209+0000","last_fullsized":"2024-05-08T23:17:50.114209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.112035+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:17:50.112035+0000","last_peered":"2024-05-08T23:17:50.112035+0000","last_clean":"2024-05-08T23:17:50.112035+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:17:50.112035+0000","last_undegraded":"2024-05-08T23:17:50.112035+0000","last_fullsized":"2024-05-08T23:17:50.112035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111038+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:17:50.111038+0000","last_peered":"2024-05-08T23:17:50.111038+0000","last_clean":"2024-05-08T23:17:50.111038+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:17:50.111038+0000","last_undegraded":"2024-05-08T23:17:50.111038+0000","last_fullsized":"2024-05-08T23:17:50.111038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120611+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:17:50.120611+0000","last_peered":"2024-05-08T23:17:50.120611+0000","last_clean":"2024-05-08T23:17:50.120611+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:17:50.120611+0000","last_undegraded":"2024-05-08T23:17:50.120611+0000","last_fullsized":"2024-05-08T23:17:50.120611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679948+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:17:50.679948+0000","last_peered":"2024-05-08T23:17:50.679948+0000","last_clean":"2024-05-08T23:17:50.679948+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:17:50.679948+0000","last_undegraded":"2024-05-08T23:17:50.679948+0000","last_fullsized":"2024-05-08T23:17:50.679948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111847+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:17:50.111847+0000","last_peered":"2024-05-08T23:17:50.111847+0000","last_clean":"2024-05-08T23:17:50.111847+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:17:50.111847+0000","last_undegraded":"2024-05-08T23:17:50.111847+0000","last_fullsized":"2024-05-08T23:17:50.111847+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118270+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:17:50.118270+0000","last_peered":"2024-05-08T23:17:50.118270+0000","last_clean":"2024-05-08T23:17:50.118270+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:17:50.118270+0000","last_undegraded":"2024-05-08T23:17:50.118270+0000","last_fullsized":"2024-05-08T23:17:50.118270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680723+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:17:50.680723+0000","last_peered":"2024-05-08T23:17:50.680723+0000","last_clean":"2024-05-08T23:17:50.680723+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:17:50.680723+0000","last_undegraded":"2024-05-08T23:17:50.680723+0000","last_fullsized":"2024-05-08T23:17:50.680723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119459+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:17:50.119459+0000","last_peered":"2024-05-08T23:17:50.119459+0000","last_clean":"2024-05-08T23:17:50.119459+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:17:50.119459+0000","last_undegraded":"2024-05-08T23:17:50.119459+0000","last_fullsized":"2024-05-08T23:17:50.119459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116279+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:17:50.116279+0000","last_peered":"2024-05-08T23:17:50.116279+0000","last_clean":"2024-05-08T23:17:50.116279+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:17:50.116279+0000","last_undegraded":"2024-05-08T23:17:50.116279+0000","last_fullsized":"2024-05-08T23:17:50.116279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680954+0000","last_change":"2024-05-08T23:17:50.137037+0000","last_active":"2024-05-08T23:17:50.680954+0000","last_peered":"2024-05-08T23:17:50.680954+0000","last_clean":"2024-05-08T23:17:50.680954+0000","last_became_active":"2024-05-08T23:17:50.136745+0000","last_became_peered":"2024-05-08T23:17:50.136745+0000","last_unstale":"2024-05-08T23:17:50.680954+0000","last_undegraded":"2024-05-08T23:17:50.680954+0000","last_fullsized":"2024-05-08T23:17:50.680954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:27.433598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.695010+0000","last_change":"2024-05-08T23:17:45.619378+0000","last_active":"2024-05-08T23:18:11.695010+0000","last_peered":"2024-05-08T23:18:11.695010+0000","last_clean":"2024-05-08T23:18:11.695010+0000","last_became_active":"2024-05-08T23:17:45.619061+0000","last_became_peered":"2024-05-08T23:17:45.619061+0000","last_unstale":"2024-05-08T23:18:11.695010+0000","last_undegraded":"2024-05-08T23:18:11.695010+0000","last_fullsized":"2024-05-08T23:18:11.695010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:29.421309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114620+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:17:50.114620+0000","last_peered":"2024-05-08T23:17:50.114620+0000","last_clean":"2024-05-08T23:17:50.114620+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:17:50.114620+0000","last_undegraded":"2024-05-08T23:17:50.114620+0000","last_fullsized":"2024-05-08T23:17:50.114620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680416+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:17:50.680416+0000","last_peered":"2024-05-08T23:17:50.680416+0000","last_clean":"2024-05-08T23:17:50.680416+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:17:50.680416+0000","last_undegraded":"2024-05-08T23:17:50.680416+0000","last_fullsized":"2024-05-08T23:17:50.680416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116514+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:17:50.116514+0000","last_peered":"2024-05-08T23:17:50.116514+0000","last_clean":"2024-05-08T23:17:50.116514+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:17:50.116514+0000","last_undegraded":"2024-05-08T23:17:50.116514+0000","last_fullsized":"2024-05-08T23:17:50.116514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117513+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:17:50.117513+0000","last_peered":"2024-05-08T23:17:50.117513+0000","last_clean":"2024-05-08T23:17:50.117513+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:17:50.117513+0000","last_undegraded":"2024-05-08T23:17:50.117513+0000","last_fullsized":"2024-05-08T23:17:50.117513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680078+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:17:50.680078+0000","last_peered":"2024-05-08T23:17:50.680078+0000","last_clean":"2024-05-08T23:17:50.680078+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:17:50.680078+0000","last_undegraded":"2024-05-08T23:17:50.680078+0000","last_fullsized":"2024-05-08T23:17:50.680078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:12.703163+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:12.703163+0000","last_peered":"2024-05-08T23:18:12.703163+0000","last_clean":"2024-05-08T23:18:12.703163+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:12.703163+0000","last_undegraded":"2024-05-08T23:18:12.703163+0000","last_fullsized":"2024-05-08T23:18:12.703163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116641+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:17:50.116641+0000","last_peered":"2024-05-08T23:17:50.116641+0000","last_clean":"2024-05-08T23:17:50.116641+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:17:50.116641+0000","last_undegraded":"2024-05-08T23:17:50.116641+0000","last_fullsized":"2024-05-08T23:17:50.116641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680065+0000","last_change":"2024-05-08T23:17:50.131565+0000","last_active":"2024-05-08T23:17:50.680065+0000","last_peered":"2024-05-08T23:17:50.680065+0000","last_clean":"2024-05-08T23:17:50.680065+0000","last_became_active":"2024-05-08T23:17:50.131136+0000","last_became_peered":"2024-05-08T23:17:50.131136+0000","last_unstale":"2024-05-08T23:17:50.680065+0000","last_undegraded":"2024-05-08T23:17:50.680065+0000","last_fullsized":"2024-05-08T23:17:50.680065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:27.930499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.696381+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:11.696381+0000","last_peered":"2024-05-08T23:18:11.696381+0000","last_clean":"2024-05-08T23:18:11.696381+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:11.696381+0000","last_undegraded":"2024-05-08T23:18:11.696381+0000","last_fullsized":"2024-05-08T23:18:11.696381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116903+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:17:50.116903+0000","last_peered":"2024-05-08T23:17:50.116903+0000","last_clean":"2024-05-08T23:17:50.116903+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:17:50.116903+0000","last_undegraded":"2024-05-08T23:17:50.116903+0000","last_fullsized":"2024-05-08T23:17:50.116903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.587483+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:17:50.587483+0000","last_peered":"2024-05-08T23:17:50.587483+0000","last_clean":"2024-05-08T23:17:50.587483+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:17:50.587483+0000","last_undegraded":"2024-05-08T23:17:50.587483+0000","last_fullsized":"2024-05-08T23:17:50.587483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694311+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:11.694311+0000","last_peered":"2024-05-08T23:18:11.694311+0000","last_clean":"2024-05-08T23:18:11.694311+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:11.694311+0000","last_undegraded":"2024-05-08T23:18:11.694311+0000","last_fullsized":"2024-05-08T23:18:11.694311+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119368+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:17:50.119368+0000","last_peered":"2024-05-08T23:17:50.119368+0000","last_clean":"2024-05-08T23:17:50.119368+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:17:50.119368+0000","last_undegraded":"2024-05-08T23:17:50.119368+0000","last_fullsized":"2024-05-08T23:17:50.119368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680422+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:17:50.680422+0000","last_peered":"2024-05-08T23:17:50.680422+0000","last_clean":"2024-05-08T23:17:50.680422+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:17:50.680422+0000","last_undegraded":"2024-05-08T23:17:50.680422+0000","last_fullsized":"2024-05-08T23:17:50.680422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681190+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:17:50.681190+0000","last_peered":"2024-05-08T23:17:50.681190+0000","last_clean":"2024-05-08T23:17:50.681190+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:17:50.681190+0000","last_undegraded":"2024-05-08T23:17:50.681190+0000","last_fullsized":"2024-05-08T23:17:50.681190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.110894+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:17:50.110894+0000","last_peered":"2024-05-08T23:17:50.110894+0000","last_clean":"2024-05-08T23:17:50.110894+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:17:50.110894+0000","last_undegraded":"2024-05-08T23:17:50.110894+0000","last_fullsized":"2024-05-08T23:17:50.110894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.126952+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:17:50.126952+0000","last_peered":"2024-05-08T23:17:50.126952+0000","last_clean":"2024-05-08T23:17:50.126952+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:17:50.126952+0000","last_undegraded":"2024-05-08T23:17:50.126952+0000","last_fullsized":"2024-05-08T23:17:50.126952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.688796+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:10.688796+0000","last_peered":"2024-05-08T23:18:10.688796+0000","last_clean":"2024-05-08T23:18:10.688796+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:10.688796+0000","last_undegraded":"2024-05-08T23:18:10.688796+0000","last_fullsized":"2024-05-08T23:18:10.688796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114857+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:17:50.114857+0000","last_peered":"2024-05-08T23:17:50.114857+0000","last_clean":"2024-05-08T23:17:50.114857+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:17:50.114857+0000","last_undegraded":"2024-05-08T23:17:50.114857+0000","last_fullsized":"2024-05-08T23:17:50.114857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.135909+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:17:50.135909+0000","last_peered":"2024-05-08T23:17:50.135909+0000","last_clean":"2024-05-08T23:17:50.135909+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:17:50.135909+0000","last_undegraded":"2024-05-08T23:17:50.135909+0000","last_fullsized":"2024-05-08T23:17:50.135909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.687601+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:10.687601+0000","last_peered":"2024-05-08T23:18:10.687601+0000","last_clean":"2024-05-08T23:18:10.687601+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:10.687601+0000","last_undegraded":"2024-05-08T23:18:10.687601+0000","last_fullsized":"2024-05-08T23:18:10.687601+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116479+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:17:50.116479+0000","last_peered":"2024-05-08T23:17:50.116479+0000","last_clean":"2024-05-08T23:17:50.116479+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:17:50.116479+0000","last_undegraded":"2024-05-08T23:17:50.116479+0000","last_fullsized":"2024-05-08T23:17:50.116479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681134+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:17:50.681134+0000","last_peered":"2024-05-08T23:17:50.681134+0000","last_clean":"2024-05-08T23:17:50.681134+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:17:50.681134+0000","last_undegraded":"2024-05-08T23:17:50.681134+0000","last_fullsized":"2024-05-08T23:17:50.681134+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.688096+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:10.688096+0000","last_peered":"2024-05-08T23:18:10.688096+0000","last_clean":"2024-05-08T23:18:10.688096+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:10.688096+0000","last_undegraded":"2024-05-08T23:18:10.688096+0000","last_fullsized":"2024-05-08T23:18:10.688096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114712+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:17:50.114712+0000","last_peered":"2024-05-08T23:17:50.114712+0000","last_clean":"2024-05-08T23:17:50.114712+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:17:50.114712+0000","last_undegraded":"2024-05-08T23:17:50.114712+0000","last_fullsized":"2024-05-08T23:17:50.114712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.130450+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:17:50.130450+0000","last_peered":"2024-05-08T23:17:50.130450+0000","last_clean":"2024-05-08T23:17:50.130450+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:17:50.130450+0000","last_undegraded":"2024-05-08T23:17:50.130450+0000","last_fullsized":"2024-05-08T23:17:50.130450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'12","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.687274+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:10.687274+0000","last_peered":"2024-05-08T23:18:10.687274+0000","last_clean":"2024-05-08T23:18:10.687274+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:10.687274+0000","last_undegraded":"2024-05-08T23:18:10.687274+0000","last_fullsized":"2024-05-08T23:18:10.687274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116280+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:17:50.116280+0000","last_peered":"2024-05-08T23:17:50.116280+0000","last_clean":"2024-05-08T23:17:50.116280+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:17:50.116280+0000","last_undegraded":"2024-05-08T23:17:50.116280+0000","last_fullsized":"2024-05-08T23:17:50.116280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680882+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:17:50.680882+0000","last_peered":"2024-05-08T23:17:50.680882+0000","last_clean":"2024-05-08T23:17:50.680882+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:17:50.680882+0000","last_undegraded":"2024-05-08T23:17:50.680882+0000","last_fullsized":"2024-05-08T23:17:50.680882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694496+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:11.694496+0000","last_peered":"2024-05-08T23:18:11.694496+0000","last_clean":"2024-05-08T23:18:11.694496+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:11.694496+0000","last_undegraded":"2024-05-08T23:18:11.694496+0000","last_fullsized":"2024-05-08T23:18:11.694496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.123579+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:17:50.123579+0000","last_peered":"2024-05-08T23:17:50.123579+0000","last_clean":"2024-05-08T23:17:50.123579+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:17:50.123579+0000","last_undegraded":"2024-05-08T23:17:50.123579+0000","last_fullsized":"2024-05-08T23:17:50.123579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.587349+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:17:50.587349+0000","last_peered":"2024-05-08T23:17:50.587349+0000","last_clean":"2024-05-08T23:17:50.587349+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:17:50.587349+0000","last_undegraded":"2024-05-08T23:17:50.587349+0000","last_fullsized":"2024-05-08T23:17:50.587349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689176+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:10.689176+0000","last_peered":"2024-05-08T23:18:10.689176+0000","last_clean":"2024-05-08T23:18:10.689176+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:10.689176+0000","last_undegraded":"2024-05-08T23:18:10.689176+0000","last_fullsized":"2024-05-08T23:18:10.689176+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120699+0000","last_change":"2024-05-08T23:17:46.113407+0000","last_active":"2024-05-08T23:17:50.120699+0000","last_peered":"2024-05-08T23:17:50.120699+0000","last_clean":"2024-05-08T23:17:50.120699+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:17:50.120699+0000","last_undegraded":"2024-05-08T23:17:50.120699+0000","last_fullsized":"2024-05-08T23:17:50.120699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:24.660375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.130530+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:17:50.130530+0000","last_peered":"2024-05-08T23:17:50.130530+0000","last_clean":"2024-05-08T23:17:50.130530+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:17:50.130530+0000","last_undegraded":"2024-05-08T23:17:50.130530+0000","last_fullsized":"2024-05-08T23:17:50.130530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.693293+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:11.693293+0000","last_peered":"2024-05-08T23:18:11.693293+0000","last_clean":"2024-05-08T23:18:11.693293+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:11.693293+0000","last_undegraded":"2024-05-08T23:18:11.693293+0000","last_fullsized":"2024-05-08T23:18:11.693293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120457+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:17:50.120457+0000","last_peered":"2024-05-08T23:17:50.120457+0000","last_clean":"2024-05-08T23:17:50.120457+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:17:50.120457+0000","last_undegraded":"2024-05-08T23:17:50.120457+0000","last_fullsized":"2024-05-08T23:17:50.120457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.787938+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:17:50.787938+0000","last_peered":"2024-05-08T23:17:50.787938+0000","last_clean":"2024-05-08T23:17:50.787938+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:17:50.787938+0000","last_undegraded":"2024-05-08T23:17:50.787938+0000","last_fullsized":"2024-05-08T23:17:50.787938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.695844+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:11.695844+0000","last_peered":"2024-05-08T23:18:11.695844+0000","last_clean":"2024-05-08T23:18:11.695844+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:11.695844+0000","last_undegraded":"2024-05-08T23:18:11.695844+0000","last_fullsized":"2024-05-08T23:18:11.695844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116027+0000","last_change":"2024-05-08T23:17:46.107292+0000","last_active":"2024-05-08T23:17:50.116027+0000","last_peered":"2024-05-08T23:17:50.116027+0000","last_clean":"2024-05-08T23:17:50.116027+0000","last_became_active":"2024-05-08T23:17:46.106950+0000","last_became_peered":"2024-05-08T23:17:46.106950+0000","last_unstale":"2024-05-08T23:17:50.116027+0000","last_undegraded":"2024-05-08T23:17:50.116027+0000","last_fullsized":"2024-05-08T23:17:50.116027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:36.714482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681308+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:17:50.681308+0000","last_peered":"2024-05-08T23:17:50.681308+0000","last_clean":"2024-05-08T23:17:50.681308+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:17:50.681308+0000","last_undegraded":"2024-05-08T23:17:50.681308+0000","last_fullsized":"2024-05-08T23:17:50.681308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116111+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:17:50.116111+0000","last_peered":"2024-05-08T23:17:50.116111+0000","last_clean":"2024-05-08T23:17:50.116111+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:17:50.116111+0000","last_undegraded":"2024-05-08T23:17:50.116111+0000","last_fullsized":"2024-05-08T23:17:50.116111+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111309+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:17:50.111309+0000","last_peered":"2024-05-08T23:17:50.111309+0000","last_clean":"2024-05-08T23:17:50.111309+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:17:50.111309+0000","last_undegraded":"2024-05-08T23:17:50.111309+0000","last_fullsized":"2024-05-08T23:17:50.111309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680407+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:17:50.680407+0000","last_peered":"2024-05-08T23:17:50.680407+0000","last_clean":"2024-05-08T23:17:50.680407+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:17:50.680407+0000","last_undegraded":"2024-05-08T23:17:50.680407+0000","last_fullsized":"2024-05-08T23:17:50.680407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.110829+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:17:50.110829+0000","last_peered":"2024-05-08T23:17:50.110829+0000","last_clean":"2024-05-08T23:17:50.110829+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:17:50.110829+0000","last_undegraded":"2024-05-08T23:17:50.110829+0000","last_fullsized":"2024-05-08T23:17:50.110829+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.243362+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:17:50.243362+0000","last_peered":"2024-05-08T23:17:50.243362+0000","last_clean":"2024-05-08T23:17:50.243362+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:17:50.243362+0000","last_undegraded":"2024-05-08T23:17:50.243362+0000","last_fullsized":"2024-05-08T23:17:50.243362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.787736+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:17:50.787736+0000","last_peered":"2024-05-08T23:17:50.787736+0000","last_clean":"2024-05-08T23:17:50.787736+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:17:50.787736+0000","last_undegraded":"2024-05-08T23:17:50.787736+0000","last_fullsized":"2024-05-08T23:17:50.787736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679548+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:17:50.679548+0000","last_peered":"2024-05-08T23:17:50.679548+0000","last_clean":"2024-05-08T23:17:50.679548+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:17:50.679548+0000","last_undegraded":"2024-05-08T23:17:50.679548+0000","last_fullsized":"2024-05-08T23:17:50.679548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.242917+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:17:50.242917+0000","last_peered":"2024-05-08T23:17:50.242917+0000","last_clean":"2024-05-08T23:17:50.242917+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:17:50.242917+0000","last_undegraded":"2024-05-08T23:17:50.242917+0000","last_fullsized":"2024-05-08T23:17:50.242917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.787867+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:17:50.787867+0000","last_peered":"2024-05-08T23:17:50.787867+0000","last_clean":"2024-05-08T23:17:50.787867+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:17:50.787867+0000","last_undegraded":"2024-05-08T23:17:50.787867+0000","last_fullsized":"2024-05-08T23:17:50.787867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115384+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:17:50.115384+0000","last_peered":"2024-05-08T23:17:50.115384+0000","last_clean":"2024-05-08T23:17:50.115384+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:17:50.115384+0000","last_undegraded":"2024-05-08T23:17:50.115384+0000","last_fullsized":"2024-05-08T23:17:50.115384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.696251+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:11.696251+0000","last_peered":"2024-05-08T23:18:11.696251+0000","last_clean":"2024-05-08T23:18:11.696251+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:11.696251+0000","last_undegraded":"2024-05-08T23:18:11.696251+0000","last_fullsized":"2024-05-08T23:18:11.696251+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.627189+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:17:53.627189+0000","last_peered":"2024-05-08T23:17:53.627189+0000","last_clean":"2024-05-08T23:17:53.627189+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:17:53.627189+0000","last_undegraded":"2024-05-08T23:17:53.627189+0000","last_fullsized":"2024-05-08T23:17:53.627189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117394+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:17:50.117394+0000","last_peered":"2024-05-08T23:17:50.117394+0000","last_clean":"2024-05-08T23:17:50.117394+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:17:50.117394+0000","last_undegraded":"2024-05-08T23:17:50.117394+0000","last_fullsized":"2024-05-08T23:17:50.117394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689040+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:10.689040+0000","last_peered":"2024-05-08T23:18:10.689040+0000","last_clean":"2024-05-08T23:18:10.689040+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:10.689040+0000","last_undegraded":"2024-05-08T23:18:10.689040+0000","last_fullsized":"2024-05-08T23:18:10.689040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":333,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679333+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:17:50.679333+0000","last_peered":"2024-05-08T23:17:50.679333+0000","last_clean":"2024-05-08T23:17:50.679333+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:17:50.679333+0000","last_undegraded":"2024-05-08T23:17:50.679333+0000","last_fullsized":"2024-05-08T23:17:50.679333+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680116+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:17:50.680116+0000","last_peered":"2024-05-08T23:17:50.680116+0000","last_clean":"2024-05-08T23:17:50.680116+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:17:50.680116+0000","last_undegraded":"2024-05-08T23:17:50.680116+0000","last_fullsized":"2024-05-08T23:17:50.680116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.121672+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:17:50.121672+0000","last_peered":"2024-05-08T23:17:50.121672+0000","last_clean":"2024-05-08T23:17:50.121672+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:17:50.121672+0000","last_undegraded":"2024-05-08T23:17:50.121672+0000","last_fullsized":"2024-05-08T23:17:50.121672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694084+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:11.694084+0000","last_peered":"2024-05-08T23:18:11.694084+0000","last_clean":"2024-05-08T23:18:11.694084+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:11.694084+0000","last_undegraded":"2024-05-08T23:18:11.694084+0000","last_fullsized":"2024-05-08T23:18:11.694084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.587238+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:17:50.587238+0000","last_peered":"2024-05-08T23:17:50.587238+0000","last_clean":"2024-05-08T23:17:50.587238+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:17:50.587238+0000","last_undegraded":"2024-05-08T23:17:50.587238+0000","last_fullsized":"2024-05-08T23:17:50.587238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679621+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:17:50.679621+0000","last_peered":"2024-05-08T23:17:50.679621+0000","last_clean":"2024-05-08T23:17:50.679621+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:17:50.679621+0000","last_undegraded":"2024-05-08T23:17:50.679621+0000","last_fullsized":"2024-05-08T23:17:50.679621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.125049+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:17:50.125049+0000","last_peered":"2024-05-08T23:17:50.125049+0000","last_clean":"2024-05-08T23:17:50.125049+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:17:50.125049+0000","last_undegraded":"2024-05-08T23:17:50.125049+0000","last_fullsized":"2024-05-08T23:17:50.125049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.245675+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:17:50.245675+0000","last_peered":"2024-05-08T23:17:50.245675+0000","last_clean":"2024-05-08T23:17:50.245675+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:17:50.245675+0000","last_undegraded":"2024-05-08T23:17:50.245675+0000","last_fullsized":"2024-05-08T23:17:50.245675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"79'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.110986+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:17:50.110986+0000","last_peered":"2024-05-08T23:17:50.110986+0000","last_clean":"2024-05-08T23:17:50.110986+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:17:50.110986+0000","last_undegraded":"2024-05-08T23:17:50.110986+0000","last_fullsized":"2024-05-08T23:17:50.110986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119855+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:17:50.119855+0000","last_peered":"2024-05-08T23:17:50.119855+0000","last_clean":"2024-05-08T23:17:50.119855+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:17:50.119855+0000","last_undegraded":"2024-05-08T23:17:50.119855+0000","last_fullsized":"2024-05-08T23:17:50.119855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679832+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:17:50.679832+0000","last_peered":"2024-05-08T23:17:50.679832+0000","last_clean":"2024-05-08T23:17:50.679832+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:17:50.679832+0000","last_undegraded":"2024-05-08T23:17:50.679832+0000","last_fullsized":"2024-05-08T23:17:50.679832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689245+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:10.689245+0000","last_peered":"2024-05-08T23:18:10.689245+0000","last_clean":"2024-05-08T23:18:10.689245+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:10.689245+0000","last_undegraded":"2024-05-08T23:18:10.689245+0000","last_fullsized":"2024-05-08T23:18:10.689245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116639+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:17:50.116639+0000","last_peered":"2024-05-08T23:17:50.116639+0000","last_clean":"2024-05-08T23:17:50.116639+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:17:50.116639+0000","last_undegraded":"2024-05-08T23:17:50.116639+0000","last_fullsized":"2024-05-08T23:17:50.116639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679779+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:17:50.679779+0000","last_peered":"2024-05-08T23:17:50.679779+0000","last_clean":"2024-05-08T23:17:50.679779+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:17:50.679779+0000","last_undegraded":"2024-05-08T23:17:50.679779+0000","last_fullsized":"2024-05-08T23:17:50.679779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114216+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:17:50.114216+0000","last_peered":"2024-05-08T23:17:50.114216+0000","last_clean":"2024-05-08T23:17:50.114216+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:17:50.114216+0000","last_undegraded":"2024-05-08T23:17:50.114216+0000","last_fullsized":"2024-05-08T23:17:50.114216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115718+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:17:50.115718+0000","last_peered":"2024-05-08T23:17:50.115718+0000","last_clean":"2024-05-08T23:17:50.115718+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:17:50.115718+0000","last_undegraded":"2024-05-08T23:17:50.115718+0000","last_fullsized":"2024-05-08T23:17:50.115718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.130510+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:17:50.130510+0000","last_peered":"2024-05-08T23:17:50.130510+0000","last_clean":"2024-05-08T23:17:50.130510+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:17:50.130510+0000","last_undegraded":"2024-05-08T23:17:50.130510+0000","last_fullsized":"2024-05-08T23:17:50.130510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118705+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:17:50.118705+0000","last_peered":"2024-05-08T23:17:50.118705+0000","last_clean":"2024-05-08T23:17:50.118705+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:17:50.118705+0000","last_undegraded":"2024-05-08T23:17:50.118705+0000","last_fullsized":"2024-05-08T23:17:50.118705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111146+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:17:50.111146+0000","last_peered":"2024-05-08T23:17:50.111146+0000","last_clean":"2024-05-08T23:17:50.111146+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:17:50.111146+0000","last_undegraded":"2024-05-08T23:17:50.111146+0000","last_fullsized":"2024-05-08T23:17:50.111146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680483+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:17:50.680483+0000","last_peered":"2024-05-08T23:17:50.680483+0000","last_clean":"2024-05-08T23:17:50.680483+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:17:50.680483+0000","last_undegraded":"2024-05-08T23:17:50.680483+0000","last_fullsized":"2024-05-08T23:17:50.680483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118698+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:17:50.118698+0000","last_peered":"2024-05-08T23:17:50.118698+0000","last_clean":"2024-05-08T23:17:50.118698+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:17:50.118698+0000","last_undegraded":"2024-05-08T23:17:50.118698+0000","last_fullsized":"2024-05-08T23:17:50.118698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117315+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:17:50.117315+0000","last_peered":"2024-05-08T23:17:50.117315+0000","last_clean":"2024-05-08T23:17:50.117315+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:17:50.117315+0000","last_undegraded":"2024-05-08T23:17:50.117315+0000","last_fullsized":"2024-05-08T23:17:50.117315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679587+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:17:50.679587+0000","last_peered":"2024-05-08T23:17:50.679587+0000","last_clean":"2024-05-08T23:17:50.679587+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:17:50.679587+0000","last_undegraded":"2024-05-08T23:17:50.679587+0000","last_fullsized":"2024-05-08T23:17:50.679587+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689528+0000","last_change":"2024-05-08T23:17:45.614392+0000","last_active":"2024-05-08T23:18:10.689528+0000","last_peered":"2024-05-08T23:18:10.689528+0000","last_clean":"2024-05-08T23:18:10.689528+0000","last_became_active":"2024-05-08T23:17:45.614096+0000","last_became_peered":"2024-05-08T23:17:45.614096+0000","last_unstale":"2024-05-08T23:18:10.689528+0000","last_undegraded":"2024-05-08T23:18:10.689528+0000","last_fullsized":"2024-05-08T23:18:10.689528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:34.954279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120577+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:17:50.120577+0000","last_peered":"2024-05-08T23:17:50.120577+0000","last_clean":"2024-05-08T23:17:50.120577+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:17:50.120577+0000","last_undegraded":"2024-05-08T23:17:50.120577+0000","last_fullsized":"2024-05-08T23:17:50.120577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.132133+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:17:50.132133+0000","last_peered":"2024-05-08T23:17:50.132133+0000","last_clean":"2024-05-08T23:17:50.132133+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:17:50.132133+0000","last_undegraded":"2024-05-08T23:17:50.132133+0000","last_fullsized":"2024-05-08T23:17:50.132133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694539+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:11.694539+0000","last_peered":"2024-05-08T23:18:11.694539+0000","last_clean":"2024-05-08T23:18:11.694539+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:11.694539+0000","last_undegraded":"2024-05-08T23:18:11.694539+0000","last_fullsized":"2024-05-08T23:18:11.694539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120986+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:17:50.120986+0000","last_peered":"2024-05-08T23:17:50.120986+0000","last_clean":"2024-05-08T23:17:50.120986+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:17:50.120986+0000","last_undegraded":"2024-05-08T23:17:50.120986+0000","last_fullsized":"2024-05-08T23:17:50.120986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679879+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:17:50.679879+0000","last_peered":"2024-05-08T23:17:50.679879+0000","last_clean":"2024-05-08T23:17:50.679879+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:17:50.679879+0000","last_undegraded":"2024-05-08T23:17:50.679879+0000","last_fullsized":"2024-05-08T23:17:50.679879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680741+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:17:50.680741+0000","last_peered":"2024-05-08T23:17:50.680741+0000","last_clean":"2024-05-08T23:17:50.680741+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:17:50.680741+0000","last_undegraded":"2024-05-08T23:17:50.680741+0000","last_fullsized":"2024-05-08T23:17:50.680741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120758+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:17:50.120758+0000","last_peered":"2024-05-08T23:17:50.120758+0000","last_clean":"2024-05-08T23:17:50.120758+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:17:50.120758+0000","last_undegraded":"2024-05-08T23:17:50.120758+0000","last_fullsized":"2024-05-08T23:17:50.120758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681002+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:17:50.681002+0000","last_peered":"2024-05-08T23:17:50.681002+0000","last_clean":"2024-05-08T23:17:50.681002+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:17:50.681002+0000","last_undegraded":"2024-05-08T23:17:50.681002+0000","last_fullsized":"2024-05-08T23:17:50.681002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.122870+0000","last_change":"2024-05-08T23:17:45.618959+0000","last_active":"2024-05-08T23:17:50.122870+0000","last_peered":"2024-05-08T23:17:50.122870+0000","last_clean":"2024-05-08T23:17:50.122870+0000","last_became_active":"2024-05-08T23:17:45.618488+0000","last_became_peered":"2024-05-08T23:17:45.618488+0000","last_unstale":"2024-05-08T23:17:50.122870+0000","last_undegraded":"2024-05-08T23:17:50.122870+0000","last_fullsized":"2024-05-08T23:17:50.122870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:06.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.121228+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:17:50.121228+0000","last_peered":"2024-05-08T23:17:50.121228+0000","last_clean":"2024-05-08T23:17:50.121228+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:17:50.121228+0000","last_undegraded":"2024-05-08T23:17:50.121228+0000","last_fullsized":"2024-05-08T23:17:50.121228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111222+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:17:50.111222+0000","last_peered":"2024-05-08T23:17:50.111222+0000","last_clean":"2024-05-08T23:17:50.111222+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:17:50.111222+0000","last_undegraded":"2024-05-08T23:17:50.111222+0000","last_fullsized":"2024-05-08T23:17:50.111222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118588+0000","last_change":"2024-05-08T23:17:46.107173+0000","last_active":"2024-05-08T23:17:50.118588+0000","last_peered":"2024-05-08T23:17:50.118588+0000","last_clean":"2024-05-08T23:17:50.118588+0000","last_became_active":"2024-05-08T23:17:46.106848+0000","last_became_peered":"2024-05-08T23:17:46.106848+0000","last_unstale":"2024-05-08T23:17:50.118588+0000","last_undegraded":"2024-05-08T23:17:50.118588+0000","last_fullsized":"2024-05-08T23:17:50.118588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:48.932524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118322+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:17:50.118322+0000","last_peered":"2024-05-08T23:17:50.118322+0000","last_clean":"2024-05-08T23:17:50.118322+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:17:50.118322+0000","last_undegraded":"2024-05-08T23:17:50.118322+0000","last_fullsized":"2024-05-08T23:17:50.118322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116985+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:17:50.116985+0000","last_peered":"2024-05-08T23:17:50.116985+0000","last_clean":"2024-05-08T23:17:50.116985+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:17:50.116985+0000","last_undegraded":"2024-05-08T23:17:50.116985+0000","last_fullsized":"2024-05-08T23:17:50.116985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.125132+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:17:50.125132+0000","last_peered":"2024-05-08T23:17:50.125132+0000","last_clean":"2024-05-08T23:17:50.125132+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:17:50.125132+0000","last_undegraded":"2024-05-08T23:17:50.125132+0000","last_fullsized":"2024-05-08T23:17:50.125132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111676+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:17:50.111676+0000","last_peered":"2024-05-08T23:17:50.111676+0000","last_clean":"2024-05-08T23:17:50.111676+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:17:50.111676+0000","last_undegraded":"2024-05-08T23:17:50.111676+0000","last_fullsized":"2024-05-08T23:17:50.111676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120764+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:17:50.120764+0000","last_peered":"2024-05-08T23:17:50.120764+0000","last_clean":"2024-05-08T23:17:50.120764+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:17:50.120764+0000","last_undegraded":"2024-05-08T23:17:50.120764+0000","last_fullsized":"2024-05-08T23:17:50.120764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119963+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:17:50.119963+0000","last_peered":"2024-05-08T23:17:50.119963+0000","last_clean":"2024-05-08T23:17:50.119963+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:17:50.119963+0000","last_undegraded":"2024-05-08T23:17:50.119963+0000","last_fullsized":"2024-05-08T23:17:50.119963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115300+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:17:50.115300+0000","last_peered":"2024-05-08T23:17:50.115300+0000","last_clean":"2024-05-08T23:17:50.115300+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:17:50.115300+0000","last_undegraded":"2024-05-08T23:17:50.115300+0000","last_fullsized":"2024-05-08T23:17:50.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118721+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:17:50.118721+0000","last_peered":"2024-05-08T23:17:50.118721+0000","last_clean":"2024-05-08T23:17:50.118721+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:17:50.118721+0000","last_undegraded":"2024-05-08T23:17:50.118721+0000","last_fullsized":"2024-05-08T23:17:50.118721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114785+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:17:50.114785+0000","last_peered":"2024-05-08T23:17:50.114785+0000","last_clean":"2024-05-08T23:17:50.114785+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:17:50.114785+0000","last_undegraded":"2024-05-08T23:17:50.114785+0000","last_fullsized":"2024-05-08T23:17:50.114785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.127764+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:17:50.127764+0000","last_peered":"2024-05-08T23:17:50.127764+0000","last_clean":"2024-05-08T23:17:50.127764+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:17:50.127764+0000","last_undegraded":"2024-05-08T23:17:50.127764+0000","last_fullsized":"2024-05-08T23:17:50.127764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111563+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:17:50.111563+0000","last_peered":"2024-05-08T23:17:50.111563+0000","last_clean":"2024-05-08T23:17:50.111563+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:17:50.111563+0000","last_undegraded":"2024-05-08T23:17:50.111563+0000","last_fullsized":"2024-05-08T23:17:50.111563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7220,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":111,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":20932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645327,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":892928,"data_stored":439710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":10,"up_from":65,"seq":279172874259,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1527808,"data_stored":1035073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.752}]},{"osd":3,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.505},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.61799999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.77500000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":9,"up_from":60,"seq":257698037781,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":440292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76400000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69599999999999995}]},{"osd":4,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84599999999999997}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.746}]},{"osd":10,"last update":"Wed May 8 23:17:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":8,"up_from":53,"seq":227633266714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":440336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.373}]},{"osd":3,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.62}]},{"osd":4,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.432}]},{"osd":6,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.55200000000000005}]},{"osd":9,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69599999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":7,"up_from":46,"seq":197568495645,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1499136,"data_stored":1031443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.72799999999999998}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.75700000000000001}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58699999999999997}]},{"osd":8,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.71799999999999997}]},{"osd":9,"last update":"Wed May 8 23:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":6,"up_from":42,"seq":180388626465,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":439800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.436}]},{"osd":1,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85299999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91600000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.131},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed May 8 23:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36092,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709060,"statfs":{"total":95995035648,"available":95958077440,"internally_reserved":0,"allocated":905216,"data_stored":443506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.505,"15min":0.5},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.629},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed May 8 23:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.46000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.46000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.50900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76400000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.79100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.75900000000000001}]},{"osd":8,"last update":"Wed May 8 23:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.84799999999999998}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.88},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68000000000000005}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":921600,"data_stored":441188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed May 8 23:17:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78200000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 23:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80100000000000005}]},{"osd":6,"last update":"Wed May 8 23:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed May 8 23:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73999999999999999}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90300000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.876},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1,"5min":1,"15min":1},"last":0.94199999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":2,"up_from":17,"seq":73014444079,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":444808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.505,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed May 8 23:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.62}]},{"osd":5,"last update":"Wed May 8 23:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63}]},{"osd":10,"last update":"Wed May 8 23:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":3,"up_from":23,"seq":98784247850,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":901120,"data_stored":439814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed May 8 23:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84499999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65200000000000002}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.627,"15min":0.58899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59199999999999997,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.9189999999999996,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.504}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.625},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57099999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.747},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.67800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":4,"up_from":30,"seq":128849018919,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":439732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.441},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.41399999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.442,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57699999999999996}]},{"osd":8,"last update":"Wed May 8 23:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66200000000000003}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":5,"up_from":35,"seq":150323855396,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36688,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708464,"statfs":{"total":95995035648,"available":95957467136,"internally_reserved":0,"allocated":1515520,"data_stored":1031506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.439},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66800000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.59599999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.32100000000000001}]},{"osd":3,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.626,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.45400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73199999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.63100000000000001,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46999999999999997}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56399999999999995}]},{"osd":8,"last update":"Wed May 8 23:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48499999999999999}]},{"osd":9,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:18:16.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:18:16.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:15 smithi189 ceph-mon[46919]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:16.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:15 smithi189 ceph-mon[46919]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:16.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:15 smithi189 ceph-mon[46919]: pgmap v237: 129 pgs: 129 active+clean; 584 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-05-08T23:18:16.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:15 smithi189 ceph-mon[46919]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T23:18:16.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:15 smithi189 ceph-mon[46919]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:18:16.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:15 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3420945515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 -- 172.21.15.16:0/2821505417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 msgr2=0x7f0234106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2821505417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f0234106690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0224009990 tx=0x7f0224009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 -- 172.21.15.16:0/2821505417 shutdown_connections 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2821505417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234106bd0 0x7f0234108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2821505417 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0234101970 0x7f0234103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2821505417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f0234106690 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 -- 172.21.15.16:0/2821505417 >> 172.21.15.16:0/2821505417 conn(0x7f02340fb320 msgr2=0x7f02340fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 -- 172.21.15.16:0/2821505417 shutdown_connections 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 -- 172.21.15.16:0/2821505417 wait complete. 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 Processor -- start 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.348+0000 7f023b2f0700 1 -- start start 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234101970 0x7f0234196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f02341971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0234106bd0 0x7f023419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419e770 con 0x7f02341042a0 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419e8e0 con 0x7f0234106bd0 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419eaa0 con 0x7f0234101970 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f02341971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f02341971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48148/0 (socket says 172.21.15.16:48148) 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 -- 172.21.15.16:0/2776752293 learned_addr learned my addr 172.21.15.16:0/2776752293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023988d700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0234106bd0 0x7f023419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023908c700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234101970 0x7f0234196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 -- 172.21.15.16:0/2776752293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234101970 msgr2=0x7f0234196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234101970 0x7f0234196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 -- 172.21.15.16:0/2776752293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0234106bd0 msgr2=0x7f023419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0234106bd0 0x7f023419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 -- 172.21.15.16:0/2776752293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02240095f0 con 0x7f02341042a0 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023908c700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234101970 0x7f0234196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023888b700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f02341971c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0224005740 tx=0x7f0224009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022401d070 con 0x7f02341042a0 2024-05-08T23:18:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f022400f460 con 0x7f02341042a0 2024-05-08T23:18:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.349+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023419ed20 con 0x7f02341042a0 2024-05-08T23:18:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.350+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224021670 con 0x7f02341042a0 2024-05-08T23:18:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.350+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023419f1a0 con 0x7f02341042a0 2024-05-08T23:18:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.351+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02340fcea0 con 0x7f02341042a0 2024-05-08T23:18:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.352+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f022400f5d0 con 0x7f02341042a0 2024-05-08T23:18:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.352+0000 7f022a7fc700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f02200a1230 0x7f02200a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.353+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f02240ce310 con 0x7f02341042a0 2024-05-08T23:18:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.353+0000 7f023908c700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f02200a1230 0x7f02200a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.353+0000 7f023908c700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f02200a1230 0x7f02200a36f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0234198100 tx=0x7f0230009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.356+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f02240977e0 con 0x7f02341042a0 2024-05-08T23:18:16.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.568+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f023402ce10 con 0x7f02200a1230 2024-05-08T23:18:16.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.581+0000 7f022a7fc700 1 -- 172.21.15.16:0/2776752293 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418791 (secure 0 0 0) 0x7f023402ce10 con 0x7f02200a1230 2024-05-08T23:18:16.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f02200a1230 msgr2=0x7f02200a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f02200a1230 0x7f02200a36f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0234198100 tx=0x7f0230009260 comp rx=0 tx=0).stop 2024-05-08T23:18:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 msgr2=0x7f02341971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f02341971c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0224005740 tx=0x7f0224009580 comp rx=0 tx=0).stop 2024-05-08T23:18:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 shutdown_connections 2024-05-08T23:18:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0234106bd0 0x7f023419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f02200a1230 0x7f02200a36f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0234101970 0x7f0234196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 --2- 172.21.15.16:0/2776752293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02341042a0 0x7f02341971c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 >> 172.21.15.16:0/2776752293 conn(0x7f02340fb320 msgr2=0x7f02341022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 shutdown_connections 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:16.585+0000 7f023b2f0700 1 -- 172.21.15.16:0/2776752293 wait complete. 2024-05-08T23:18:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:18:16.599 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-05-08T23:18:14.572043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597705,"num_objects":55,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":293,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389860,"kb_used_data":12392,"kb_used_omap":18,"kb_used_meta":377325,"kb_avail":1124551964,"statfs":{"total":1151940427776,"available":1151541211136,"internally_reserved":0,"allocated":12689408,"data_stored":7067208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":386381128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2884,"num_objects":31,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":65,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.112156+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:17:50.112156+0000","last_peered":"2024-05-08T23:17:50.112156+0000","last_clean":"2024-05-08T23:17:50.112156+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:17:50.112156+0000","last_undegraded":"2024-05-08T23:17:50.112156+0000","last_fullsized":"2024-05-08T23:17:50.112156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116529+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:17:50.116529+0000","last_peered":"2024-05-08T23:17:50.116529+0000","last_clean":"2024-05-08T23:17:50.116529+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:17:50.116529+0000","last_undegraded":"2024-05-08T23:17:50.116529+0000","last_fullsized":"2024-05-08T23:17:50.116529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117104+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:17:50.117104+0000","last_peered":"2024-05-08T23:17:50.117104+0000","last_clean":"2024-05-08T23:17:50.117104+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:17:50.117104+0000","last_undegraded":"2024-05-08T23:17:50.117104+0000","last_fullsized":"2024-05-08T23:17:50.117104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118325+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:17:50.118325+0000","last_peered":"2024-05-08T23:17:50.118325+0000","last_clean":"2024-05-08T23:17:50.118325+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:17:50.118325+0000","last_undegraded":"2024-05-08T23:17:50.118325+0000","last_fullsized":"2024-05-08T23:17:50.118325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115828+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:17:50.115828+0000","last_peered":"2024-05-08T23:17:50.115828+0000","last_clean":"2024-05-08T23:17:50.115828+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:17:50.115828+0000","last_undegraded":"2024-05-08T23:17:50.115828+0000","last_fullsized":"2024-05-08T23:17:50.115828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.242862+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:17:50.242862+0000","last_peered":"2024-05-08T23:17:50.242862+0000","last_clean":"2024-05-08T23:17:50.242862+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:17:50.242862+0000","last_undegraded":"2024-05-08T23:17:50.242862+0000","last_fullsized":"2024-05-08T23:17:50.242862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120386+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:17:50.120386+0000","last_peered":"2024-05-08T23:17:50.120386+0000","last_clean":"2024-05-08T23:17:50.120386+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:17:50.120386+0000","last_undegraded":"2024-05-08T23:17:50.120386+0000","last_fullsized":"2024-05-08T23:17:50.120386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118752+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:17:50.118752+0000","last_peered":"2024-05-08T23:17:50.118752+0000","last_clean":"2024-05-08T23:17:50.118752+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:17:50.118752+0000","last_undegraded":"2024-05-08T23:17:50.118752+0000","last_fullsized":"2024-05-08T23:17:50.118752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114352+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:17:50.114352+0000","last_peered":"2024-05-08T23:17:50.114352+0000","last_clean":"2024-05-08T23:17:50.114352+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:17:50.114352+0000","last_undegraded":"2024-05-08T23:17:50.114352+0000","last_fullsized":"2024-05-08T23:17:50.114352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.243355+0000","last_change":"2024-05-08T23:17:46.107981+0000","last_active":"2024-05-08T23:17:50.243355+0000","last_peered":"2024-05-08T23:17:50.243355+0000","last_clean":"2024-05-08T23:17:50.243355+0000","last_became_active":"2024-05-08T23:17:46.107334+0000","last_became_peered":"2024-05-08T23:17:46.107334+0000","last_unstale":"2024-05-08T23:17:50.243355+0000","last_undegraded":"2024-05-08T23:17:50.243355+0000","last_fullsized":"2024-05-08T23:17:50.243355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.409548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120911+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:17:50.120911+0000","last_peered":"2024-05-08T23:17:50.120911+0000","last_clean":"2024-05-08T23:17:50.120911+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:17:50.120911+0000","last_undegraded":"2024-05-08T23:17:50.120911+0000","last_fullsized":"2024-05-08T23:17:50.120911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118236+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:17:50.118236+0000","last_peered":"2024-05-08T23:17:50.118236+0000","last_clean":"2024-05-08T23:17:50.118236+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:17:50.118236+0000","last_undegraded":"2024-05-08T23:17:50.118236+0000","last_fullsized":"2024-05-08T23:17:50.118236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.127869+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:17:50.127869+0000","last_peered":"2024-05-08T23:17:50.127869+0000","last_clean":"2024-05-08T23:17:50.127869+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:17:50.127869+0000","last_undegraded":"2024-05-08T23:17:50.127869+0000","last_fullsized":"2024-05-08T23:17:50.127869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116213+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:17:50.116213+0000","last_peered":"2024-05-08T23:17:50.116213+0000","last_clean":"2024-05-08T23:17:50.116213+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:17:50.116213+0000","last_undegraded":"2024-05-08T23:17:50.116213+0000","last_fullsized":"2024-05-08T23:17:50.116213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116083+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:17:50.116083+0000","last_peered":"2024-05-08T23:17:50.116083+0000","last_clean":"2024-05-08T23:17:50.116083+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:17:50.116083+0000","last_undegraded":"2024-05-08T23:17:50.116083+0000","last_fullsized":"2024-05-08T23:17:50.116083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114209+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:17:50.114209+0000","last_peered":"2024-05-08T23:17:50.114209+0000","last_clean":"2024-05-08T23:17:50.114209+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:17:50.114209+0000","last_undegraded":"2024-05-08T23:17:50.114209+0000","last_fullsized":"2024-05-08T23:17:50.114209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.112035+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:17:50.112035+0000","last_peered":"2024-05-08T23:17:50.112035+0000","last_clean":"2024-05-08T23:17:50.112035+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:17:50.112035+0000","last_undegraded":"2024-05-08T23:17:50.112035+0000","last_fullsized":"2024-05-08T23:17:50.112035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111038+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:17:50.111038+0000","last_peered":"2024-05-08T23:17:50.111038+0000","last_clean":"2024-05-08T23:17:50.111038+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:17:50.111038+0000","last_undegraded":"2024-05-08T23:17:50.111038+0000","last_fullsized":"2024-05-08T23:17:50.111038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120611+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:17:50.120611+0000","last_peered":"2024-05-08T23:17:50.120611+0000","last_clean":"2024-05-08T23:17:50.120611+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:17:50.120611+0000","last_undegraded":"2024-05-08T23:17:50.120611+0000","last_fullsized":"2024-05-08T23:17:50.120611+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679948+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:17:50.679948+0000","last_peered":"2024-05-08T23:17:50.679948+0000","last_clean":"2024-05-08T23:17:50.679948+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:17:50.679948+0000","last_undegraded":"2024-05-08T23:17:50.679948+0000","last_fullsized":"2024-05-08T23:17:50.679948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"76'4","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111847+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:17:50.111847+0000","last_peered":"2024-05-08T23:17:50.111847+0000","last_clean":"2024-05-08T23:17:50.111847+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:17:50.111847+0000","last_undegraded":"2024-05-08T23:17:50.111847+0000","last_fullsized":"2024-05-08T23:17:50.111847+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118270+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:17:50.118270+0000","last_peered":"2024-05-08T23:17:50.118270+0000","last_clean":"2024-05-08T23:17:50.118270+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:17:50.118270+0000","last_undegraded":"2024-05-08T23:17:50.118270+0000","last_fullsized":"2024-05-08T23:17:50.118270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680723+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:17:50.680723+0000","last_peered":"2024-05-08T23:17:50.680723+0000","last_clean":"2024-05-08T23:17:50.680723+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:17:50.680723+0000","last_undegraded":"2024-05-08T23:17:50.680723+0000","last_fullsized":"2024-05-08T23:17:50.680723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"76'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119459+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:17:50.119459+0000","last_peered":"2024-05-08T23:17:50.119459+0000","last_clean":"2024-05-08T23:17:50.119459+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:17:50.119459+0000","last_undegraded":"2024-05-08T23:17:50.119459+0000","last_fullsized":"2024-05-08T23:17:50.119459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116279+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:17:50.116279+0000","last_peered":"2024-05-08T23:17:50.116279+0000","last_clean":"2024-05-08T23:17:50.116279+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:17:50.116279+0000","last_undegraded":"2024-05-08T23:17:50.116279+0000","last_fullsized":"2024-05-08T23:17:50.116279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680954+0000","last_change":"2024-05-08T23:17:50.137037+0000","last_active":"2024-05-08T23:17:50.680954+0000","last_peered":"2024-05-08T23:17:50.680954+0000","last_clean":"2024-05-08T23:17:50.680954+0000","last_became_active":"2024-05-08T23:17:50.136745+0000","last_became_peered":"2024-05-08T23:17:50.136745+0000","last_unstale":"2024-05-08T23:17:50.680954+0000","last_undegraded":"2024-05-08T23:17:50.680954+0000","last_fullsized":"2024-05-08T23:17:50.680954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:27.433598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.695010+0000","last_change":"2024-05-08T23:17:45.619378+0000","last_active":"2024-05-08T23:18:11.695010+0000","last_peered":"2024-05-08T23:18:11.695010+0000","last_clean":"2024-05-08T23:18:11.695010+0000","last_became_active":"2024-05-08T23:17:45.619061+0000","last_became_peered":"2024-05-08T23:17:45.619061+0000","last_unstale":"2024-05-08T23:18:11.695010+0000","last_undegraded":"2024-05-08T23:18:11.695010+0000","last_fullsized":"2024-05-08T23:18:11.695010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:31:29.421309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114620+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:17:50.114620+0000","last_peered":"2024-05-08T23:17:50.114620+0000","last_clean":"2024-05-08T23:17:50.114620+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:17:50.114620+0000","last_undegraded":"2024-05-08T23:17:50.114620+0000","last_fullsized":"2024-05-08T23:17:50.114620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680416+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:17:50.680416+0000","last_peered":"2024-05-08T23:17:50.680416+0000","last_clean":"2024-05-08T23:17:50.680416+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:17:50.680416+0000","last_undegraded":"2024-05-08T23:17:50.680416+0000","last_fullsized":"2024-05-08T23:17:50.680416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116514+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:17:50.116514+0000","last_peered":"2024-05-08T23:17:50.116514+0000","last_clean":"2024-05-08T23:17:50.116514+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:17:50.116514+0000","last_undegraded":"2024-05-08T23:17:50.116514+0000","last_fullsized":"2024-05-08T23:17:50.116514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117513+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:17:50.117513+0000","last_peered":"2024-05-08T23:17:50.117513+0000","last_clean":"2024-05-08T23:17:50.117513+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:17:50.117513+0000","last_undegraded":"2024-05-08T23:17:50.117513+0000","last_fullsized":"2024-05-08T23:17:50.117513+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680078+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:17:50.680078+0000","last_peered":"2024-05-08T23:17:50.680078+0000","last_clean":"2024-05-08T23:17:50.680078+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:17:50.680078+0000","last_undegraded":"2024-05-08T23:17:50.680078+0000","last_fullsized":"2024-05-08T23:17:50.680078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:12.703163+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:12.703163+0000","last_peered":"2024-05-08T23:18:12.703163+0000","last_clean":"2024-05-08T23:18:12.703163+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:12.703163+0000","last_undegraded":"2024-05-08T23:18:12.703163+0000","last_fullsized":"2024-05-08T23:18:12.703163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116641+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:17:50.116641+0000","last_peered":"2024-05-08T23:17:50.116641+0000","last_clean":"2024-05-08T23:17:50.116641+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:17:50.116641+0000","last_undegraded":"2024-05-08T23:17:50.116641+0000","last_fullsized":"2024-05-08T23:17:50.116641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680065+0000","last_change":"2024-05-08T23:17:50.131565+0000","last_active":"2024-05-08T23:17:50.680065+0000","last_peered":"2024-05-08T23:17:50.680065+0000","last_clean":"2024-05-08T23:17:50.680065+0000","last_became_active":"2024-05-08T23:17:50.131136+0000","last_became_peered":"2024-05-08T23:17:50.131136+0000","last_unstale":"2024-05-08T23:17:50.680065+0000","last_undegraded":"2024-05-08T23:17:50.680065+0000","last_fullsized":"2024-05-08T23:17:50.680065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:27.930499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.696381+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:11.696381+0000","last_peered":"2024-05-08T23:18:11.696381+0000","last_clean":"2024-05-08T23:18:11.696381+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:11.696381+0000","last_undegraded":"2024-05-08T23:18:11.696381+0000","last_fullsized":"2024-05-08T23:18:11.696381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116903+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:17:50.116903+0000","last_peered":"2024-05-08T23:17:50.116903+0000","last_clean":"2024-05-08T23:17:50.116903+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:17:50.116903+0000","last_undegraded":"2024-05-08T23:17:50.116903+0000","last_fullsized":"2024-05-08T23:17:50.116903+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.587483+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:17:50.587483+0000","last_peered":"2024-05-08T23:17:50.587483+0000","last_clean":"2024-05-08T23:17:50.587483+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:17:50.587483+0000","last_undegraded":"2024-05-08T23:17:50.587483+0000","last_fullsized":"2024-05-08T23:17:50.587483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694311+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:11.694311+0000","last_peered":"2024-05-08T23:18:11.694311+0000","last_clean":"2024-05-08T23:18:11.694311+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:11.694311+0000","last_undegraded":"2024-05-08T23:18:11.694311+0000","last_fullsized":"2024-05-08T23:18:11.694311+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119368+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:17:50.119368+0000","last_peered":"2024-05-08T23:17:50.119368+0000","last_clean":"2024-05-08T23:17:50.119368+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:17:50.119368+0000","last_undegraded":"2024-05-08T23:17:50.119368+0000","last_fullsized":"2024-05-08T23:17:50.119368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680422+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:17:50.680422+0000","last_peered":"2024-05-08T23:17:50.680422+0000","last_clean":"2024-05-08T23:17:50.680422+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:17:50.680422+0000","last_undegraded":"2024-05-08T23:17:50.680422+0000","last_fullsized":"2024-05-08T23:17:50.680422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681190+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:17:50.681190+0000","last_peered":"2024-05-08T23:17:50.681190+0000","last_clean":"2024-05-08T23:17:50.681190+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:17:50.681190+0000","last_undegraded":"2024-05-08T23:17:50.681190+0000","last_fullsized":"2024-05-08T23:17:50.681190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.110894+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:17:50.110894+0000","last_peered":"2024-05-08T23:17:50.110894+0000","last_clean":"2024-05-08T23:17:50.110894+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:17:50.110894+0000","last_undegraded":"2024-05-08T23:17:50.110894+0000","last_fullsized":"2024-05-08T23:17:50.110894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.126952+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:17:50.126952+0000","last_peered":"2024-05-08T23:17:50.126952+0000","last_clean":"2024-05-08T23:17:50.126952+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:17:50.126952+0000","last_undegraded":"2024-05-08T23:17:50.126952+0000","last_fullsized":"2024-05-08T23:17:50.126952+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.688796+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:10.688796+0000","last_peered":"2024-05-08T23:18:10.688796+0000","last_clean":"2024-05-08T23:18:10.688796+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:10.688796+0000","last_undegraded":"2024-05-08T23:18:10.688796+0000","last_fullsized":"2024-05-08T23:18:10.688796+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114857+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:17:50.114857+0000","last_peered":"2024-05-08T23:17:50.114857+0000","last_clean":"2024-05-08T23:17:50.114857+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:17:50.114857+0000","last_undegraded":"2024-05-08T23:17:50.114857+0000","last_fullsized":"2024-05-08T23:17:50.114857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.135909+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:17:50.135909+0000","last_peered":"2024-05-08T23:17:50.135909+0000","last_clean":"2024-05-08T23:17:50.135909+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:17:50.135909+0000","last_undegraded":"2024-05-08T23:17:50.135909+0000","last_fullsized":"2024-05-08T23:17:50.135909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.687601+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:10.687601+0000","last_peered":"2024-05-08T23:18:10.687601+0000","last_clean":"2024-05-08T23:18:10.687601+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:10.687601+0000","last_undegraded":"2024-05-08T23:18:10.687601+0000","last_fullsized":"2024-05-08T23:18:10.687601+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116479+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:17:50.116479+0000","last_peered":"2024-05-08T23:17:50.116479+0000","last_clean":"2024-05-08T23:17:50.116479+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:17:50.116479+0000","last_undegraded":"2024-05-08T23:17:50.116479+0000","last_fullsized":"2024-05-08T23:17:50.116479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681134+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:17:50.681134+0000","last_peered":"2024-05-08T23:17:50.681134+0000","last_clean":"2024-05-08T23:17:50.681134+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:17:50.681134+0000","last_undegraded":"2024-05-08T23:17:50.681134+0000","last_fullsized":"2024-05-08T23:17:50.681134+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.688096+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:10.688096+0000","last_peered":"2024-05-08T23:18:10.688096+0000","last_clean":"2024-05-08T23:18:10.688096+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:10.688096+0000","last_undegraded":"2024-05-08T23:18:10.688096+0000","last_fullsized":"2024-05-08T23:18:10.688096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114712+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:17:50.114712+0000","last_peered":"2024-05-08T23:17:50.114712+0000","last_clean":"2024-05-08T23:17:50.114712+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:17:50.114712+0000","last_undegraded":"2024-05-08T23:17:50.114712+0000","last_fullsized":"2024-05-08T23:17:50.114712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.130450+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:17:50.130450+0000","last_peered":"2024-05-08T23:17:50.130450+0000","last_clean":"2024-05-08T23:17:50.130450+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:17:50.130450+0000","last_undegraded":"2024-05-08T23:17:50.130450+0000","last_fullsized":"2024-05-08T23:17:50.130450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'12","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.687274+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:10.687274+0000","last_peered":"2024-05-08T23:18:10.687274+0000","last_clean":"2024-05-08T23:18:10.687274+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:10.687274+0000","last_undegraded":"2024-05-08T23:18:10.687274+0000","last_fullsized":"2024-05-08T23:18:10.687274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116280+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:17:50.116280+0000","last_peered":"2024-05-08T23:17:50.116280+0000","last_clean":"2024-05-08T23:17:50.116280+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:17:50.116280+0000","last_undegraded":"2024-05-08T23:17:50.116280+0000","last_fullsized":"2024-05-08T23:17:50.116280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680882+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:17:50.680882+0000","last_peered":"2024-05-08T23:17:50.680882+0000","last_clean":"2024-05-08T23:17:50.680882+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:17:50.680882+0000","last_undegraded":"2024-05-08T23:17:50.680882+0000","last_fullsized":"2024-05-08T23:17:50.680882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694496+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:11.694496+0000","last_peered":"2024-05-08T23:18:11.694496+0000","last_clean":"2024-05-08T23:18:11.694496+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:11.694496+0000","last_undegraded":"2024-05-08T23:18:11.694496+0000","last_fullsized":"2024-05-08T23:18:11.694496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.123579+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:17:50.123579+0000","last_peered":"2024-05-08T23:17:50.123579+0000","last_clean":"2024-05-08T23:17:50.123579+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:17:50.123579+0000","last_undegraded":"2024-05-08T23:17:50.123579+0000","last_fullsized":"2024-05-08T23:17:50.123579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.587349+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:17:50.587349+0000","last_peered":"2024-05-08T23:17:50.587349+0000","last_clean":"2024-05-08T23:17:50.587349+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:17:50.587349+0000","last_undegraded":"2024-05-08T23:17:50.587349+0000","last_fullsized":"2024-05-08T23:17:50.587349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689176+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:10.689176+0000","last_peered":"2024-05-08T23:18:10.689176+0000","last_clean":"2024-05-08T23:18:10.689176+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:10.689176+0000","last_undegraded":"2024-05-08T23:18:10.689176+0000","last_fullsized":"2024-05-08T23:18:10.689176+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120699+0000","last_change":"2024-05-08T23:17:46.113407+0000","last_active":"2024-05-08T23:17:50.120699+0000","last_peered":"2024-05-08T23:17:50.120699+0000","last_clean":"2024-05-08T23:17:50.120699+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:17:50.120699+0000","last_undegraded":"2024-05-08T23:17:50.120699+0000","last_fullsized":"2024-05-08T23:17:50.120699+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:40:24.660375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.130530+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:17:50.130530+0000","last_peered":"2024-05-08T23:17:50.130530+0000","last_clean":"2024-05-08T23:17:50.130530+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:17:50.130530+0000","last_undegraded":"2024-05-08T23:17:50.130530+0000","last_fullsized":"2024-05-08T23:17:50.130530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.693293+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:11.693293+0000","last_peered":"2024-05-08T23:18:11.693293+0000","last_clean":"2024-05-08T23:18:11.693293+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:11.693293+0000","last_undegraded":"2024-05-08T23:18:11.693293+0000","last_fullsized":"2024-05-08T23:18:11.693293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120457+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:17:50.120457+0000","last_peered":"2024-05-08T23:17:50.120457+0000","last_clean":"2024-05-08T23:17:50.120457+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:17:50.120457+0000","last_undegraded":"2024-05-08T23:17:50.120457+0000","last_fullsized":"2024-05-08T23:17:50.120457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.787938+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:17:50.787938+0000","last_peered":"2024-05-08T23:17:50.787938+0000","last_clean":"2024-05-08T23:17:50.787938+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:17:50.787938+0000","last_undegraded":"2024-05-08T23:17:50.787938+0000","last_fullsized":"2024-05-08T23:17:50.787938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.695844+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:11.695844+0000","last_peered":"2024-05-08T23:18:11.695844+0000","last_clean":"2024-05-08T23:18:11.695844+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:11.695844+0000","last_undegraded":"2024-05-08T23:18:11.695844+0000","last_fullsized":"2024-05-08T23:18:11.695844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116027+0000","last_change":"2024-05-08T23:17:46.107292+0000","last_active":"2024-05-08T23:17:50.116027+0000","last_peered":"2024-05-08T23:17:50.116027+0000","last_clean":"2024-05-08T23:17:50.116027+0000","last_became_active":"2024-05-08T23:17:46.106950+0000","last_became_peered":"2024-05-08T23:17:46.106950+0000","last_unstale":"2024-05-08T23:17:50.116027+0000","last_undegraded":"2024-05-08T23:17:50.116027+0000","last_fullsized":"2024-05-08T23:17:50.116027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:36.714482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681308+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:17:50.681308+0000","last_peered":"2024-05-08T23:17:50.681308+0000","last_clean":"2024-05-08T23:17:50.681308+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:17:50.681308+0000","last_undegraded":"2024-05-08T23:17:50.681308+0000","last_fullsized":"2024-05-08T23:17:50.681308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"76'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116111+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:17:50.116111+0000","last_peered":"2024-05-08T23:17:50.116111+0000","last_clean":"2024-05-08T23:17:50.116111+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:17:50.116111+0000","last_undegraded":"2024-05-08T23:17:50.116111+0000","last_fullsized":"2024-05-08T23:17:50.116111+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111309+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:17:50.111309+0000","last_peered":"2024-05-08T23:17:50.111309+0000","last_clean":"2024-05-08T23:17:50.111309+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:17:50.111309+0000","last_undegraded":"2024-05-08T23:17:50.111309+0000","last_fullsized":"2024-05-08T23:17:50.111309+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680407+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:17:50.680407+0000","last_peered":"2024-05-08T23:17:50.680407+0000","last_clean":"2024-05-08T23:17:50.680407+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:17:50.680407+0000","last_undegraded":"2024-05-08T23:17:50.680407+0000","last_fullsized":"2024-05-08T23:17:50.680407+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.110829+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:17:50.110829+0000","last_peered":"2024-05-08T23:17:50.110829+0000","last_clean":"2024-05-08T23:17:50.110829+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:17:50.110829+0000","last_undegraded":"2024-05-08T23:17:50.110829+0000","last_fullsized":"2024-05-08T23:17:50.110829+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.243362+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:17:50.243362+0000","last_peered":"2024-05-08T23:17:50.243362+0000","last_clean":"2024-05-08T23:17:50.243362+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:17:50.243362+0000","last_undegraded":"2024-05-08T23:17:50.243362+0000","last_fullsized":"2024-05-08T23:17:50.243362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.787736+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:17:50.787736+0000","last_peered":"2024-05-08T23:17:50.787736+0000","last_clean":"2024-05-08T23:17:50.787736+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:17:50.787736+0000","last_undegraded":"2024-05-08T23:17:50.787736+0000","last_fullsized":"2024-05-08T23:17:50.787736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679548+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:17:50.679548+0000","last_peered":"2024-05-08T23:17:50.679548+0000","last_clean":"2024-05-08T23:17:50.679548+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:17:50.679548+0000","last_undegraded":"2024-05-08T23:17:50.679548+0000","last_fullsized":"2024-05-08T23:17:50.679548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.242917+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:17:50.242917+0000","last_peered":"2024-05-08T23:17:50.242917+0000","last_clean":"2024-05-08T23:17:50.242917+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:17:50.242917+0000","last_undegraded":"2024-05-08T23:17:50.242917+0000","last_fullsized":"2024-05-08T23:17:50.242917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.787867+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:17:50.787867+0000","last_peered":"2024-05-08T23:17:50.787867+0000","last_clean":"2024-05-08T23:17:50.787867+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:17:50.787867+0000","last_undegraded":"2024-05-08T23:17:50.787867+0000","last_fullsized":"2024-05-08T23:17:50.787867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115384+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:17:50.115384+0000","last_peered":"2024-05-08T23:17:50.115384+0000","last_clean":"2024-05-08T23:17:50.115384+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:17:50.115384+0000","last_undegraded":"2024-05-08T23:17:50.115384+0000","last_fullsized":"2024-05-08T23:17:50.115384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.696251+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:11.696251+0000","last_peered":"2024-05-08T23:18:11.696251+0000","last_clean":"2024-05-08T23:18:11.696251+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:11.696251+0000","last_undegraded":"2024-05-08T23:18:11.696251+0000","last_fullsized":"2024-05-08T23:18:11.696251+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.627189+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:17:53.627189+0000","last_peered":"2024-05-08T23:17:53.627189+0000","last_clean":"2024-05-08T23:17:53.627189+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:17:53.627189+0000","last_undegraded":"2024-05-08T23:17:53.627189+0000","last_fullsized":"2024-05-08T23:17:53.627189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117394+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:17:50.117394+0000","last_peered":"2024-05-08T23:17:50.117394+0000","last_clean":"2024-05-08T23:17:50.117394+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:17:50.117394+0000","last_undegraded":"2024-05-08T23:17:50.117394+0000","last_fullsized":"2024-05-08T23:17:50.117394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689040+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:10.689040+0000","last_peered":"2024-05-08T23:18:10.689040+0000","last_clean":"2024-05-08T23:18:10.689040+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:10.689040+0000","last_undegraded":"2024-05-08T23:18:10.689040+0000","last_fullsized":"2024-05-08T23:18:10.689040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":333,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679333+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:17:50.679333+0000","last_peered":"2024-05-08T23:17:50.679333+0000","last_clean":"2024-05-08T23:17:50.679333+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:17:50.679333+0000","last_undegraded":"2024-05-08T23:17:50.679333+0000","last_fullsized":"2024-05-08T23:17:50.679333+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680116+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:17:50.680116+0000","last_peered":"2024-05-08T23:17:50.680116+0000","last_clean":"2024-05-08T23:17:50.680116+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:17:50.680116+0000","last_undegraded":"2024-05-08T23:17:50.680116+0000","last_fullsized":"2024-05-08T23:17:50.680116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.121672+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:17:50.121672+0000","last_peered":"2024-05-08T23:17:50.121672+0000","last_clean":"2024-05-08T23:17:50.121672+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:17:50.121672+0000","last_undegraded":"2024-05-08T23:17:50.121672+0000","last_fullsized":"2024-05-08T23:17:50.121672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694084+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:11.694084+0000","last_peered":"2024-05-08T23:18:11.694084+0000","last_clean":"2024-05-08T23:18:11.694084+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:11.694084+0000","last_undegraded":"2024-05-08T23:18:11.694084+0000","last_fullsized":"2024-05-08T23:18:11.694084+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.587238+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:17:50.587238+0000","last_peered":"2024-05-08T23:17:50.587238+0000","last_clean":"2024-05-08T23:17:50.587238+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:17:50.587238+0000","last_undegraded":"2024-05-08T23:17:50.587238+0000","last_fullsized":"2024-05-08T23:17:50.587238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679621+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:17:50.679621+0000","last_peered":"2024-05-08T23:17:50.679621+0000","last_clean":"2024-05-08T23:17:50.679621+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:17:50.679621+0000","last_undegraded":"2024-05-08T23:17:50.679621+0000","last_fullsized":"2024-05-08T23:17:50.679621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.125049+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:17:50.125049+0000","last_peered":"2024-05-08T23:17:50.125049+0000","last_clean":"2024-05-08T23:17:50.125049+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:17:50.125049+0000","last_undegraded":"2024-05-08T23:17:50.125049+0000","last_fullsized":"2024-05-08T23:17:50.125049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.245675+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:17:50.245675+0000","last_peered":"2024-05-08T23:17:50.245675+0000","last_clean":"2024-05-08T23:17:50.245675+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:17:50.245675+0000","last_undegraded":"2024-05-08T23:17:50.245675+0000","last_fullsized":"2024-05-08T23:17:50.245675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"79'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.110986+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:17:50.110986+0000","last_peered":"2024-05-08T23:17:50.110986+0000","last_clean":"2024-05-08T23:17:50.110986+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:17:50.110986+0000","last_undegraded":"2024-05-08T23:17:50.110986+0000","last_fullsized":"2024-05-08T23:17:50.110986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119855+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:17:50.119855+0000","last_peered":"2024-05-08T23:17:50.119855+0000","last_clean":"2024-05-08T23:17:50.119855+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:17:50.119855+0000","last_undegraded":"2024-05-08T23:17:50.119855+0000","last_fullsized":"2024-05-08T23:17:50.119855+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679832+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:17:50.679832+0000","last_peered":"2024-05-08T23:17:50.679832+0000","last_clean":"2024-05-08T23:17:50.679832+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:17:50.679832+0000","last_undegraded":"2024-05-08T23:17:50.679832+0000","last_fullsized":"2024-05-08T23:17:50.679832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689245+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:10.689245+0000","last_peered":"2024-05-08T23:18:10.689245+0000","last_clean":"2024-05-08T23:18:10.689245+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:10.689245+0000","last_undegraded":"2024-05-08T23:18:10.689245+0000","last_fullsized":"2024-05-08T23:18:10.689245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116639+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:17:50.116639+0000","last_peered":"2024-05-08T23:17:50.116639+0000","last_clean":"2024-05-08T23:17:50.116639+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:17:50.116639+0000","last_undegraded":"2024-05-08T23:17:50.116639+0000","last_fullsized":"2024-05-08T23:17:50.116639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679779+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:17:50.679779+0000","last_peered":"2024-05-08T23:17:50.679779+0000","last_clean":"2024-05-08T23:17:50.679779+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:17:50.679779+0000","last_undegraded":"2024-05-08T23:17:50.679779+0000","last_fullsized":"2024-05-08T23:17:50.679779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114216+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:17:50.114216+0000","last_peered":"2024-05-08T23:17:50.114216+0000","last_clean":"2024-05-08T23:17:50.114216+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:17:50.114216+0000","last_undegraded":"2024-05-08T23:17:50.114216+0000","last_fullsized":"2024-05-08T23:17:50.114216+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115718+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:17:50.115718+0000","last_peered":"2024-05-08T23:17:50.115718+0000","last_clean":"2024-05-08T23:17:50.115718+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:17:50.115718+0000","last_undegraded":"2024-05-08T23:17:50.115718+0000","last_fullsized":"2024-05-08T23:17:50.115718+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.130510+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:17:50.130510+0000","last_peered":"2024-05-08T23:17:50.130510+0000","last_clean":"2024-05-08T23:17:50.130510+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:17:50.130510+0000","last_undegraded":"2024-05-08T23:17:50.130510+0000","last_fullsized":"2024-05-08T23:17:50.130510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118705+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:17:50.118705+0000","last_peered":"2024-05-08T23:17:50.118705+0000","last_clean":"2024-05-08T23:17:50.118705+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:17:50.118705+0000","last_undegraded":"2024-05-08T23:17:50.118705+0000","last_fullsized":"2024-05-08T23:17:50.118705+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111146+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:17:50.111146+0000","last_peered":"2024-05-08T23:17:50.111146+0000","last_clean":"2024-05-08T23:17:50.111146+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:17:50.111146+0000","last_undegraded":"2024-05-08T23:17:50.111146+0000","last_fullsized":"2024-05-08T23:17:50.111146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680483+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:17:50.680483+0000","last_peered":"2024-05-08T23:17:50.680483+0000","last_clean":"2024-05-08T23:17:50.680483+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:17:50.680483+0000","last_undegraded":"2024-05-08T23:17:50.680483+0000","last_fullsized":"2024-05-08T23:17:50.680483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118698+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:17:50.118698+0000","last_peered":"2024-05-08T23:17:50.118698+0000","last_clean":"2024-05-08T23:17:50.118698+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:17:50.118698+0000","last_undegraded":"2024-05-08T23:17:50.118698+0000","last_fullsized":"2024-05-08T23:17:50.118698+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.117315+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:17:50.117315+0000","last_peered":"2024-05-08T23:17:50.117315+0000","last_clean":"2024-05-08T23:17:50.117315+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:17:50.117315+0000","last_undegraded":"2024-05-08T23:17:50.117315+0000","last_fullsized":"2024-05-08T23:17:50.117315+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679587+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:17:50.679587+0000","last_peered":"2024-05-08T23:17:50.679587+0000","last_clean":"2024-05-08T23:17:50.679587+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:17:50.679587+0000","last_undegraded":"2024-05-08T23:17:50.679587+0000","last_fullsized":"2024-05-08T23:17:50.679587+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689528+0000","last_change":"2024-05-08T23:17:45.614392+0000","last_active":"2024-05-08T23:18:10.689528+0000","last_peered":"2024-05-08T23:18:10.689528+0000","last_clean":"2024-05-08T23:18:10.689528+0000","last_became_active":"2024-05-08T23:17:45.614096+0000","last_became_peered":"2024-05-08T23:17:45.614096+0000","last_unstale":"2024-05-08T23:18:10.689528+0000","last_undegraded":"2024-05-08T23:18:10.689528+0000","last_fullsized":"2024-05-08T23:18:10.689528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:34.954279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120577+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:17:50.120577+0000","last_peered":"2024-05-08T23:17:50.120577+0000","last_clean":"2024-05-08T23:17:50.120577+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:17:50.120577+0000","last_undegraded":"2024-05-08T23:17:50.120577+0000","last_fullsized":"2024-05-08T23:17:50.120577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.132133+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:17:50.132133+0000","last_peered":"2024-05-08T23:17:50.132133+0000","last_clean":"2024-05-08T23:17:50.132133+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:17:50.132133+0000","last_undegraded":"2024-05-08T23:17:50.132133+0000","last_fullsized":"2024-05-08T23:17:50.132133+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694539+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:11.694539+0000","last_peered":"2024-05-08T23:18:11.694539+0000","last_clean":"2024-05-08T23:18:11.694539+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:11.694539+0000","last_undegraded":"2024-05-08T23:18:11.694539+0000","last_fullsized":"2024-05-08T23:18:11.694539+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120986+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:17:50.120986+0000","last_peered":"2024-05-08T23:17:50.120986+0000","last_clean":"2024-05-08T23:17:50.120986+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:17:50.120986+0000","last_undegraded":"2024-05-08T23:17:50.120986+0000","last_fullsized":"2024-05-08T23:17:50.120986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679879+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:17:50.679879+0000","last_peered":"2024-05-08T23:17:50.679879+0000","last_clean":"2024-05-08T23:17:50.679879+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:17:50.679879+0000","last_undegraded":"2024-05-08T23:17:50.679879+0000","last_fullsized":"2024-05-08T23:17:50.679879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680741+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:17:50.680741+0000","last_peered":"2024-05-08T23:17:50.680741+0000","last_clean":"2024-05-08T23:17:50.680741+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:17:50.680741+0000","last_undegraded":"2024-05-08T23:17:50.680741+0000","last_fullsized":"2024-05-08T23:17:50.680741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120758+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:17:50.120758+0000","last_peered":"2024-05-08T23:17:50.120758+0000","last_clean":"2024-05-08T23:17:50.120758+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:17:50.120758+0000","last_undegraded":"2024-05-08T23:17:50.120758+0000","last_fullsized":"2024-05-08T23:17:50.120758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681002+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:17:50.681002+0000","last_peered":"2024-05-08T23:17:50.681002+0000","last_clean":"2024-05-08T23:17:50.681002+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:17:50.681002+0000","last_undegraded":"2024-05-08T23:17:50.681002+0000","last_fullsized":"2024-05-08T23:17:50.681002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.122870+0000","last_change":"2024-05-08T23:17:45.618959+0000","last_active":"2024-05-08T23:17:50.122870+0000","last_peered":"2024-05-08T23:17:50.122870+0000","last_clean":"2024-05-08T23:17:50.122870+0000","last_became_active":"2024-05-08T23:17:45.618488+0000","last_became_peered":"2024-05-08T23:17:45.618488+0000","last_unstale":"2024-05-08T23:17:50.122870+0000","last_undegraded":"2024-05-08T23:17:50.122870+0000","last_fullsized":"2024-05-08T23:17:50.122870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:06.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.121228+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:17:50.121228+0000","last_peered":"2024-05-08T23:17:50.121228+0000","last_clean":"2024-05-08T23:17:50.121228+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:17:50.121228+0000","last_undegraded":"2024-05-08T23:17:50.121228+0000","last_fullsized":"2024-05-08T23:17:50.121228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111222+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:17:50.111222+0000","last_peered":"2024-05-08T23:17:50.111222+0000","last_clean":"2024-05-08T23:17:50.111222+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:17:50.111222+0000","last_undegraded":"2024-05-08T23:17:50.111222+0000","last_fullsized":"2024-05-08T23:17:50.111222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118588+0000","last_change":"2024-05-08T23:17:46.107173+0000","last_active":"2024-05-08T23:17:50.118588+0000","last_peered":"2024-05-08T23:17:50.118588+0000","last_clean":"2024-05-08T23:17:50.118588+0000","last_became_active":"2024-05-08T23:17:46.106848+0000","last_became_peered":"2024-05-08T23:17:46.106848+0000","last_unstale":"2024-05-08T23:17:50.118588+0000","last_undegraded":"2024-05-08T23:17:50.118588+0000","last_fullsized":"2024-05-08T23:17:50.118588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:48.932524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118322+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:17:50.118322+0000","last_peered":"2024-05-08T23:17:50.118322+0000","last_clean":"2024-05-08T23:17:50.118322+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:17:50.118322+0000","last_undegraded":"2024-05-08T23:17:50.118322+0000","last_fullsized":"2024-05-08T23:17:50.118322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.116985+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:17:50.116985+0000","last_peered":"2024-05-08T23:17:50.116985+0000","last_clean":"2024-05-08T23:17:50.116985+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:17:50.116985+0000","last_undegraded":"2024-05-08T23:17:50.116985+0000","last_fullsized":"2024-05-08T23:17:50.116985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.125132+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:17:50.125132+0000","last_peered":"2024-05-08T23:17:50.125132+0000","last_clean":"2024-05-08T23:17:50.125132+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:17:50.125132+0000","last_undegraded":"2024-05-08T23:17:50.125132+0000","last_fullsized":"2024-05-08T23:17:50.125132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111676+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:17:50.111676+0000","last_peered":"2024-05-08T23:17:50.111676+0000","last_clean":"2024-05-08T23:17:50.111676+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:17:50.111676+0000","last_undegraded":"2024-05-08T23:17:50.111676+0000","last_fullsized":"2024-05-08T23:17:50.111676+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.120764+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:17:50.120764+0000","last_peered":"2024-05-08T23:17:50.120764+0000","last_clean":"2024-05-08T23:17:50.120764+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:17:50.120764+0000","last_undegraded":"2024-05-08T23:17:50.120764+0000","last_fullsized":"2024-05-08T23:17:50.120764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.119963+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:17:50.119963+0000","last_peered":"2024-05-08T23:17:50.119963+0000","last_clean":"2024-05-08T23:17:50.119963+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:17:50.119963+0000","last_undegraded":"2024-05-08T23:17:50.119963+0000","last_fullsized":"2024-05-08T23:17:50.119963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.115300+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:17:50.115300+0000","last_peered":"2024-05-08T23:17:50.115300+0000","last_clean":"2024-05-08T23:17:50.115300+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:17:50.115300+0000","last_undegraded":"2024-05-08T23:17:50.115300+0000","last_fullsized":"2024-05-08T23:17:50.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.118721+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:17:50.118721+0000","last_peered":"2024-05-08T23:17:50.118721+0000","last_clean":"2024-05-08T23:17:50.118721+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:17:50.118721+0000","last_undegraded":"2024-05-08T23:17:50.118721+0000","last_fullsized":"2024-05-08T23:17:50.118721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.114785+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:17:50.114785+0000","last_peered":"2024-05-08T23:17:50.114785+0000","last_clean":"2024-05-08T23:17:50.114785+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:17:50.114785+0000","last_undegraded":"2024-05-08T23:17:50.114785+0000","last_fullsized":"2024-05-08T23:17:50.114785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.127764+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:17:50.127764+0000","last_peered":"2024-05-08T23:17:50.127764+0000","last_clean":"2024-05-08T23:17:50.127764+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:17:50.127764+0000","last_undegraded":"2024-05-08T23:17:50.127764+0000","last_fullsized":"2024-05-08T23:17:50.127764+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.111563+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:17:50.111563+0000","last_peered":"2024-05-08T23:17:50.111563+0000","last_clean":"2024-05-08T23:17:50.111563+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:17:50.111563+0000","last_undegraded":"2024-05-08T23:17:50.111563+0000","last_fullsized":"2024-05-08T23:17:50.111563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7220,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":111,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":20932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645327,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":892928,"data_stored":439710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":10,"up_from":65,"seq":279172874259,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1527808,"data_stored":1035073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.752}]},{"osd":3,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.505},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.61799999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.77500000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":9,"up_from":60,"seq":257698037781,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":440292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76400000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69599999999999995}]},{"osd":4,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84599999999999997}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.82599999999999996}]},{"osd":8,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.746}]},{"osd":10,"last update":"Wed May 8 23:17:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":8,"up_from":53,"seq":227633266714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":440336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.373}]},{"osd":3,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.62}]},{"osd":4,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.432}]},{"osd":6,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.55200000000000005}]},{"osd":9,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69599999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.63},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":7,"up_from":46,"seq":197568495645,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1499136,"data_stored":1031443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.72799999999999998}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70499999999999996}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.75700000000000001}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58699999999999997}]},{"osd":8,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.71799999999999997}]},{"osd":9,"last update":"Wed May 8 23:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":6,"up_from":42,"seq":180388626465,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36084,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709068,"statfs":{"total":95995035648,"available":95958085632,"internally_reserved":0,"allocated":897024,"data_stored":439800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.436}]},{"osd":1,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85299999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91600000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56399999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.131},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58699999999999997}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60799999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed May 8 23:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.73999999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36092,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709060,"statfs":{"total":95995035648,"available":95958077440,"internally_reserved":0,"allocated":905216,"data_stored":443506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.505,"15min":0.5},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.34799999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.74299999999999999}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.629},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed May 8 23:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.46000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.46000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.50900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76400000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.79100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.75900000000000001}]},{"osd":8,"last update":"Wed May 8 23:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.84799999999999998}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.88},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68000000000000005}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36108,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709044,"statfs":{"total":95995035648,"available":95958061056,"internally_reserved":0,"allocated":921600,"data_stored":441188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57799999999999996}]},{"osd":2,"last update":"Wed May 8 23:17:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.55400000000000005},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78200000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.67600000000000005}]},{"osd":3,"last update":"Wed May 8 23:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 23:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80100000000000005}]},{"osd":6,"last update":"Wed May 8 23:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.78100000000000003}]},{"osd":7,"last update":"Wed May 8 23:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73999999999999999}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90300000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.876},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1,"5min":1,"15min":1},"last":0.94199999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]}]},{"osd":2,"up_from":17,"seq":73014444079,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":444808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.30499999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.505,"15min":0.495},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.51500000000000001,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.68300000000000005}]},{"osd":4,"last update":"Wed May 8 23:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.62}]},{"osd":5,"last update":"Wed May 8 23:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.44,"15min":0.432},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.73699999999999999}]},{"osd":8,"last update":"Wed May 8 23:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63}]},{"osd":10,"last update":"Wed May 8 23:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":3,"up_from":23,"seq":98784247850,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36096,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709056,"statfs":{"total":95995035648,"available":95958073344,"internally_reserved":0,"allocated":901120,"data_stored":439814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed May 8 23:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84499999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65200000000000002}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.627,"15min":0.58899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59199999999999997,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.9189999999999996,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.504}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.61599999999999999}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:17:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.625},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57099999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.747},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.67800000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":4,"up_from":30,"seq":128849018919,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":439732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33000000000000002},"min":{"1min":0.249,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.441},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.36499999999999999,"15min":0.35699999999999998},"min":{"1min":0.27300000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.41399999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52100000000000002,"15min":0.51200000000000001},"min":{"1min":0.442,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.46200000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57699999999999996}]},{"osd":8,"last update":"Wed May 8 23:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66200000000000003}]},{"osd":9,"last update":"Wed May 8 23:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":5,"up_from":35,"seq":150323855396,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36688,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708464,"statfs":{"total":95995035648,"available":95957467136,"internally_reserved":0,"allocated":1515520,"data_stored":1031506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.439},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66800000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80000000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.60699999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.433,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.59599999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.32100000000000001}]},{"osd":3,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.626,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.45400000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.67100000000000004}]},{"osd":4,"last update":"Wed May 8 23:17:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73199999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.63100000000000001,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46999999999999997}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56399999999999995}]},{"osd":8,"last update":"Wed May 8 23:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48499999999999999}]},{"osd":9,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73599999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:18:16.607 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T23:18:16.608 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T23:18:16.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T23:18:16.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:16 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2364987466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:16.893 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:16.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:16 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2364987466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:17.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:16 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2364987466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.946+0000 7f72d8316700 1 -- 172.21.15.16:0/3358838809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 msgr2=0x7f72d0103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.946+0000 7f72d8316700 1 --2- 172.21.15.16:0/3358838809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0103ea0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009990 tx=0x7f72cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 -- 172.21.15.16:0/3358838809 shutdown_connections 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 --2- 172.21.15.16:0/3358838809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d0106d10 0x7f72d0109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 --2- 172.21.15.16:0/3358838809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0103ea0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 --2- 172.21.15.16:0/3358838809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 0x7f72d01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 -- 172.21.15.16:0/3358838809 >> 172.21.15.16:0/3358838809 conn(0x7f72d00fb460 msgr2=0x7f72d00fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 -- 172.21.15.16:0/3358838809 shutdown_connections 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 -- 172.21.15.16:0/3358838809 wait complete. 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 Processor -- start 2024-05-08T23:18:17.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.947+0000 7f72d8316700 1 -- start start 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d8316700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d8316700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 0x7f72d01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d8316700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d0106d10 0x7f72d019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d8316700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d019e8f0 con 0x7f72d0101ab0 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d8316700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d019ea30 con 0x7f72d0106d10 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d8316700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d019ebc0 con 0x7f72d01043e0 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48166/0 (socket says 172.21.15.16:48166) 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 -- 172.21.15.16:0/352118879 learned_addr learned my addr 172.21.15.16:0/352118879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:17.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d58b1700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 0x7f72d01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d68b3700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d0106d10 0x7f72d019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 -- 172.21.15.16:0/352118879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 msgr2=0x7f72d01972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 0x7f72d01972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 -- 172.21.15.16:0/352118879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d0106d10 msgr2=0x7f72d019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d0106d10 0x7f72d019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 -- 172.21.15.16:0/352118879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0095f0 con 0x7f72d0101ab0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d58b1700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 0x7f72d01972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.948+0000 7f72d60b2700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0196db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f72cc003a50 tx=0x7f72cc003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.949+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc01d070 con 0x7f72d0101ab0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.949+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d019ee40 con 0x7f72d0101ab0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.949+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d019f1f0 con 0x7f72d0101ab0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.949+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72cc00bd70 con 0x7f72d0101ab0 2024-05-08T23:18:17.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.949+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00fbc0 con 0x7f72d0101ab0 2024-05-08T23:18:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.950+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b4005320 con 0x7f72d0101ab0 2024-05-08T23:18:17.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.951+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f72cc00fd20 con 0x7f72d0101ab0 2024-05-08T23:18:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.951+0000 7f72c77fe700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f72bc0a11b0 0x7f72bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.952+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f72cc0ce140 con 0x7f72d0101ab0 2024-05-08T23:18:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.952+0000 7f72d58b1700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f72bc0a11b0 0x7f72bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.952+0000 7f72d58b1700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f72bc0a11b0 0x7f72bc0a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005fd0 tx=0x7f72c0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:17.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:17.956+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f72cc097730 con 0x7f72d0101ab0 2024-05-08T23:18:17.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:17 smithi155 ceph-mon[47105]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:17.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:17 smithi155 ceph-mon[47105]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:17.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:17 smithi155 ceph-mon[47105]: pgmap v238: 129 pgs: 129 active+clean; 584 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-08T23:18:18.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:17 smithi016 ceph-mon[42541]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:17 smithi016 ceph-mon[42541]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:17 smithi016 ceph-mon[42541]: pgmap v238: 129 pgs: 129 active+clean; 584 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-08T23:18:18.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:17 smithi189 ceph-mon[46919]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:17 smithi189 ceph-mon[46919]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:17 smithi189 ceph-mon[46919]: pgmap v238: 129 pgs: 129 active+clean; 584 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-08T23:18:18.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.258+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f72b40055d0 con 0x7f72d0101ab0 2024-05-08T23:18:18.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.263+0000 7f72c77fe700 1 -- 172.21.15.16:0/352118879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f72cc09ad50 con 0x7f72d0101ab0 2024-05-08T23:18:18.263 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-05-08T23:18:18.263 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQA6CDxmzlZ6DxAAoz6emyVj4AAwymNVTxH2jA== 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f72bc0a11b0 msgr2=0x7f72bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f72bc0a11b0 0x7f72bc0a3670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72c0005fd0 tx=0x7f72c0005e30 comp rx=0 tx=0).stop 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 msgr2=0x7f72d0196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0196db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f72cc003a50 tx=0x7f72cc003b30 comp rx=0 tx=0).stop 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 shutdown_connections 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72d0106d10 0x7f72d019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d0101ab0 0x7f72d0196db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f72bc0a11b0 0x7f72bc0a3670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 --2- 172.21.15.16:0/352118879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72d01043e0 0x7f72d01972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 >> 172.21.15.16:0/352118879 conn(0x7f72d00fb460 msgr2=0x7f72d0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 shutdown_connections 2024-05-08T23:18:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:18.266+0000 7f72d8316700 1 -- 172.21.15.16:0/352118879 wait complete. 2024-05-08T23:18:18.649 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:18:18.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T23:18:18.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T23:18:18.701 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T23:18:18.701 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T23:18:18.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T23:18:18.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:18 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/352118879' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:18:18.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:18 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/352118879' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:18:18.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:18 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/352118879' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:18:18.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:18 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/352118879' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:18:18.998 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:19.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:18 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/352118879' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T23:18:19.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:18 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/352118879' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T23:18:19.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:19 smithi155 ceph-mon[47105]: pgmap v239: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-05-08T23:18:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 -- 172.21.15.16:0/2097280007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f0106320 msgr2=0x7fd8f010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2097280007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f0106320 0x7fd8f010de80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800b0a0 tx=0x7fd8e800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 -- 172.21.15.16:0/2097280007 shutdown_connections 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2097280007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f0106320 0x7fd8f010de80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2097280007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f00ff750 0x7fd8f00ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2097280007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8f0100100 0x7fd8f0105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 -- 172.21.15.16:0/2097280007 >> 172.21.15.16:0/2097280007 conn(0x7fd8f0076470 msgr2=0x7fd8f00fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 -- 172.21.15.16:0/2097280007 shutdown_connections 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.004+0000 7fd8f80de700 1 -- 172.21.15.16:0/2097280007 wait complete. 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 Processor -- start 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 -- start start 2024-05-08T23:18:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 0x7fd8f0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8f0100100 0x7fd8f0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0106320 0x7fd8f019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019c300 con 0x7fd8f00ff750 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019c470 con 0x7fd8f0106320 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f80de700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019c630 con 0x7fd8f0100100 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f5e7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 0x7fd8f0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f5e7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 0x7fd8f0197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48190/0 (socket says 172.21.15.16:48190) 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f5e7a700 1 -- 172.21.15.16:0/2481676302 learned_addr learned my addr 172.21.15.16:0/2481676302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f667b700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0106320 0x7fd8f019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.005+0000 7fd8f5679700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8f0100100 0x7fd8f0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f5e7a700 1 -- 172.21.15.16:0/2481676302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8f0100100 msgr2=0x7fd8f0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f5e7a700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8f0100100 0x7fd8f0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f5e7a700 1 -- 172.21.15.16:0/2481676302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0106320 msgr2=0x7fd8f019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f5e7a700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0106320 0x7fd8f019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f5e7a700 1 -- 172.21.15.16:0/2481676302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e8009d00 con 0x7fd8f00ff750 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f5e7a700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 0x7fd8f0197400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00d950 tx=0x7fd8ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00f890 con 0x7fd8f00ff750 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8ec00fe80 con 0x7fd8f00ff750 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00e3b0 con 0x7fd8f00ff750 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f019c910 con 0x7fd8f00ff750 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.006+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f01a4170 con 0x7fd8f00ff750 2024-05-08T23:18:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.007+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f01095d0 con 0x7fd8f00ff750 2024-05-08T23:18:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.008+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd8ec010460 con 0x7fd8f00ff750 2024-05-08T23:18:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.009+0000 7fd8e6ffd700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8dc0a11b0 0x7fd8dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.009+0000 7fd8f5679700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8dc0a11b0 0x7fd8dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.009+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd8ec0cca10 con 0x7fd8f00ff750 2024-05-08T23:18:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.010+0000 7fd8f5679700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8dc0a11b0 0x7fd8dc0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0198960 tx=0x7fd8e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.013+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd8ec095e50 con 0x7fd8f00ff750 2024-05-08T23:18:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:19 smithi016 ceph-mon[42541]: pgmap v239: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-05-08T23:18:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:19 smithi189 ceph-mon[46919]: pgmap v239: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-05-08T23:18:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.283+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd8f019cb00 con 0x7fd8f00ff750 2024-05-08T23:18:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.288+0000 7fd8e6ffd700 1 -- 172.21.15.16:0/2481676302 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260281 (secure 0 0 0) 0x7fd8ec016090 con 0x7fd8f00ff750 2024-05-08T23:18:20.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.291+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8dc0a11b0 msgr2=0x7fd8dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.291+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8dc0a11b0 0x7fd8dc0a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0198960 tx=0x7fd8e000b410 comp rx=0 tx=0).stop 2024-05-08T23:18:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.291+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 msgr2=0x7fd8f0197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.291+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 0x7fd8f0197400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00d950 tx=0x7fd8ec00dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 shutdown_connections 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8f0106320 0x7fd8f019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f00ff750 0x7fd8f0197400 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd8dc0a11b0 0x7fd8dc0a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 --2- 172.21.15.16:0/2481676302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8f0100100 0x7fd8f0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 >> 172.21.15.16:0/2481676302 conn(0x7fd8f0076470 msgr2=0x7fd8f0104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 shutdown_connections 2024-05-08T23:18:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:20.292+0000 7fd8f80de700 1 -- 172.21.15.16:0/2481676302 wait complete. 2024-05-08T23:18:20.674 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":781396085},{"type":"v1","addr":"172.21.15.16:6801","nonce":781396085}]},"active_addr":"172.21.15.16:6801/781396085","active_change":"2024-05-08T23:12:42.499829+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":592052660}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":4093375965}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":745120666}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3133644606}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1044015911}]}]} 2024-05-08T23:18:20.678 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T23:18:20.678 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T23:18:20.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:18:20.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:20 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2481676302' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:18:20.937 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:21.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:20 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2481676302' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:18:21.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:20 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2481676302' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T23:18:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 -- 172.21.15.16:0/3518759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 msgr2=0x7ff298107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 --2- 172.21.15.16:0/3518759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298107400 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0099c0 tx=0x7ff28c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 -- 172.21.15.16:0/3518759099 shutdown_connections 2024-05-08T23:18:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 --2- 172.21.15.16:0/3518759099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2980692f0 0x7ff29810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 --2- 172.21.15.16:0/3518759099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298107400 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 --2- 172.21.15.16:0/3518759099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff298068950 0x7ff298068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 -- 172.21.15.16:0/3518759099 >> 172.21.15.16:0/3518759099 conn(0x7ff298075f80 msgr2=0x7ff298076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 -- 172.21.15.16:0/3518759099 shutdown_connections 2024-05-08T23:18:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.968+0000 7ff29f10d700 1 -- 172.21.15.16:0/3518759099 wait complete. 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 Processor -- start 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 -- start start 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff298068950 0x7ff2980fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2980692f0 0x7ff2981004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2981a55a0 con 0x7ff298107020 2024-05-08T23:18:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2981a5710 con 0x7ff2980692f0 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29f10d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2981a58d0 con 0x7ff298068950 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29d6aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29d6aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48198/0 (socket says 172.21.15.16:48198) 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29d6aa700 1 -- 172.21.15.16:0/987378323 learned_addr learned my addr 172.21.15.16:0/987378323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff297fff700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2980692f0 0x7ff2981004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.969+0000 7ff29cea9700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff298068950 0x7ff2980fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29d6aa700 1 -- 172.21.15.16:0/987378323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff298068950 msgr2=0x7ff2980fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29d6aa700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff298068950 0x7ff2980fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29d6aa700 1 -- 172.21.15.16:0/987378323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2980692f0 msgr2=0x7ff2981004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29d6aa700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2980692f0 0x7ff2981004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29d6aa700 1 -- 172.21.15.16:0/987378323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c0095f0 con 0x7ff298107020 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29cea9700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff298068950 0x7ff2980fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:21.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29d6aa700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298100a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff290007ee0 tx=0x7ff29000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290017070 con 0x7ff298107020 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff29000f040 con 0x7ff298107020 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2981a5bb0 con 0x7ff298107020 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290005710 con 0x7ff298107020 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.970+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2981a60b0 con 0x7ff298107020 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.971+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298109540 con 0x7ff298107020 2024-05-08T23:18:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.972+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff290005870 con 0x7ff298107020 2024-05-08T23:18:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.973+0000 7ff295ffb700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff2880a1230 0x7ff2880a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.973+0000 7ff29cea9700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff2880a1230 0x7ff2880a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.973+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff2900cc420 con 0x7ff298107020 2024-05-08T23:18:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.974+0000 7ff29cea9700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff2880a1230 0x7ff2880a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0036c0 tx=0x7ff28c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:21.978+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff290095800 con 0x7ff298107020 2024-05-08T23:18:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:21 smithi016 ceph-mon[42541]: pgmap v240: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-05-08T23:18:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:21 smithi189 ceph-mon[46919]: pgmap v240: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-05-08T23:18:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.193+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2981026d0 con 0x7ff298107020 2024-05-08T23:18:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.195+0000 7ff295ffb700 1 -- 172.21.15.16:0/987378323 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7ff290098e20 con 0x7ff298107020 2024-05-08T23:18:22.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:22.195 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff2880a1230 msgr2=0x7ff2880a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff2880a1230 0x7ff2880a36f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0036c0 tx=0x7ff28c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 msgr2=0x7ff298100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298100a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff290007ee0 tx=0x7ff29000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 shutdown_connections 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff298107020 0x7ff298100a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff2880a1230 0x7ff2880a36f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:22.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff298068950 0x7ff2980fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 --2- 172.21.15.16:0/987378323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2980692f0 0x7ff2981004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 >> 172.21.15.16:0/987378323 conn(0x7ff298075f80 msgr2=0x7ff2980fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.197+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 shutdown_connections 2024-05-08T23:18:22.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:22.198+0000 7ff29f10d700 1 -- 172.21.15.16:0/987378323 wait complete. 2024-05-08T23:18:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:21 smithi155 ceph-mon[47105]: pgmap v240: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2024-05-08T23:18:22.590 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T23:18:22.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T23:18:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:22 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/987378323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:23.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:22 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/987378323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:23.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:22 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/987378323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2316825041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81051c0 msgr2=0x7f67a810cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2316825041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81051c0 0x7f67a810cd90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67a000b0a0 tx=0x7f67a000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2316825041 shutdown_connections 2024-05-08T23:18:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2316825041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81051c0 0x7f67a810cd90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2316825041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67a81000e0 0x7f67a81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2316825041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a8100a90 0x7f67a8104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2316825041 >> 172.21.15.16:0/2316825041 conn(0x7f67a8076f40 msgr2=0x7f67a8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2316825041 shutdown_connections 2024-05-08T23:18:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.896+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2316825041 wait complete. 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 Processor -- start 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 -- start start 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 0x7f67a8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67a8100a90 0x7f67a8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a81051c0 0x7f67a819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a819c320 con 0x7f67a81000e0 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a819c490 con 0x7f67a8100a90 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67ad6c0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a819c650 con 0x7f67a81051c0 2024-05-08T23:18:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 0x7f67a8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.897+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 0x7f67a8197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48228/0 (socket says 172.21.15.16:48228) 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 -- 172.21.15.16:0/2000456239 learned_addr learned my addr 172.21.15.16:0/2000456239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f679e7fc700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67a8100a90 0x7f67a8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a77fe700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a81051c0 0x7f67a819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 -- 172.21.15.16:0/2000456239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a81051c0 msgr2=0x7f67a819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a81051c0 0x7f67a819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 -- 172.21.15.16:0/2000456239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67a8100a90 msgr2=0x7f67a8197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67a8100a90 0x7f67a8197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 -- 172.21.15.16:0/2000456239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0009d00 con 0x7f67a81000e0 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a6ffd700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 0x7f67a8197420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f679800d950 tx=0x7f679800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800f890 con 0x7f67a81000e0 2024-05-08T23:18:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a819c930 con 0x7f67a81000e0 2024-05-08T23:18:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f679800fe80 con 0x7f67a81000e0 2024-05-08T23:18:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800e3b0 con 0x7f67a81000e0 2024-05-08T23:18:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.898+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a81a4190 con 0x7f67a81000e0 2024-05-08T23:18:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.899+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a81094f0 con 0x7f67a81000e0 2024-05-08T23:18:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.901+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6798010460 con 0x7f67a81000e0 2024-05-08T23:18:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.902+0000 7f67a4ff9700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f67940a11b0 0x7f67940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.902+0000 7f679e7fc700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f67940a11b0 0x7f67940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.902+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f67980cca10 con 0x7f67a81000e0 2024-05-08T23:18:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.902+0000 7f679e7fc700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f67940a11b0 0x7f67940a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67a8198980 tx=0x7f679000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:23.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:23.905+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6798095e50 con 0x7f67a81000e0 2024-05-08T23:18:24.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:23 smithi016 ceph-mon[42541]: pgmap v241: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-05-08T23:18:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.120+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67a81015c0 con 0x7f67a81000e0 2024-05-08T23:18:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.122+0000 7f67a4ff9700 1 -- 172.21.15.16:0/2000456239 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+23971 (secure 0 0 0) 0x7f6798016090 con 0x7f67a81000e0 2024-05-08T23:18:24.122 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:24.122 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":81,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:17:50.580593+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f67940a11b0 msgr2=0x7f67940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f67940a11b0 0x7f67940a3670 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67a8198980 tx=0x7f679000b410 comp rx=0 tx=0).stop 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 msgr2=0x7f67a8197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 0x7f67a8197420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f679800d950 tx=0x7f679800dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 shutdown_connections 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f67940a11b0 0x7f67940a3670 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a81051c0 0x7f67a819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a81000e0 0x7f67a8197420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 --2- 172.21.15.16:0/2000456239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67a8100a90 0x7f67a8197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 >> 172.21.15.16:0/2000456239 conn(0x7f67a8076f40 msgr2=0x7f67a80699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 shutdown_connections 2024-05-08T23:18:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:24.124+0000 7f67ad6c0700 1 -- 172.21.15.16:0/2000456239 wait complete. 2024-05-08T23:18:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:23 smithi189 ceph-mon[46919]: pgmap v241: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-05-08T23:18:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:23 smithi155 ceph-mon[47105]: pgmap v241: 129 pgs: 129 active+clean; 625 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-05-08T23:18:24.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T23:18:24.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T23:18:24.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T23:18:24.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T23:18:24.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T23:18:24.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T23:18:24.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T23:18:24.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T23:18:24.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T23:18:24.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T23:18:24.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T23:18:24.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T23:18:24.752 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:24 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2000456239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:25.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:24 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2000456239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:25.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:24 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2000456239' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:18:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:25.972 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:26.102 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:25 smithi016 ceph-mon[42541]: pgmap v242: 129 pgs: 129 active+clean; 635 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-05-08T23:18:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:25 smithi189 ceph-mon[46919]: pgmap v242: 129 pgs: 129 active+clean; 635 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-05-08T23:18:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:25 smithi155 ceph-mon[47105]: pgmap v242: 129 pgs: 129 active+clean; 635 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 -- 172.21.15.16:0/3490675993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 msgr2=0x7f65a810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 --2- 172.21.15.16:0/3490675993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a810de70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f65a000b0a0 tx=0x7f65a000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 -- 172.21.15.16:0/3490675993 shutdown_connections 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 --2- 172.21.15.16:0/3490675993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a810de70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 --2- 172.21.15.16:0/3490675993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8107030 0x7f65a8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 --2- 172.21.15.16:0/3490675993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a8100ff0 0x7f65a8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 -- 172.21.15.16:0/3490675993 >> 172.21.15.16:0/3490675993 conn(0x7f65a8078190 msgr2=0x7f65a80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 -- 172.21.15.16:0/3490675993 shutdown_connections 2024-05-08T23:18:26.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.400+0000 7f65ae741700 1 -- 172.21.15.16:0/3490675993 wait complete. 2024-05-08T23:18:26.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.401+0000 7f65ae741700 1 Processor -- start 2024-05-08T23:18:26.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.401+0000 7f65ae741700 1 -- start start 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65ae741700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a8100ff0 0x7f65a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65ae741700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65ae741700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8107030 0x7f65a8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65ae741700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81a5780 con 0x7f65a8101990 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65ae741700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81a58f0 con 0x7f65a8107030 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65ae741700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81a5ab0 con 0x7f65a8100ff0 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34652/0 (socket says 172.21.15.16:34652) 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 -- 172.21.15.16:0/2358761034 learned_addr learned my addr 172.21.15.16:0/2358761034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a7fff700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a8100ff0 0x7f65a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:26.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 -- 172.21.15.16:0/2358761034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a8100ff0 msgr2=0x7f65a80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a8100ff0 0x7f65a80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 -- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8107030 msgr2=0x7f65a8073d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a8107030 0x7f65a8073d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.402+0000 7f65a77fe700 1 -- 172.21.15.16:0/2358761034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0009d00 con 0x7f65a8101990 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.403+0000 7f65a77fe700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a80737e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d950 tx=0x7f659c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.403+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c009940 con 0x7f65a8101990 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.403+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f659c010460 con 0x7f65a8101990 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.403+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a81a5d90 con 0x7f65a8101990 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.403+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00f620 con 0x7f65a8101990 2024-05-08T23:18:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.403+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a81a6290 con 0x7f65a8101990 2024-05-08T23:18:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.404+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a8111010 con 0x7f65a8101990 2024-05-08T23:18:26.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.406+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f659c0105d0 con 0x7f65a8101990 2024-05-08T23:18:26.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a57fa700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65900a11b0 0x7f65900a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f659c0ccf50 con 0x7f65a8101990 2024-05-08T23:18:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a7fff700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65900a11b0 0x7f65900a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a57fa700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] conn(0x7f65900abd30 0x7f65900ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 --> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65900ae840 con 0x7f65900abd30 2024-05-08T23:18:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659c0cd360 con 0x7f65a8101990 2024-05-08T23:18:26.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.407+0000 7f65a7fff700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65900a11b0 0x7f65900a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6598005950 tx=0x7f659800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.408+0000 7f65accde700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] conn(0x7f65900abd30 0x7f65900ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.408+0000 7f65accde700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] conn(0x7f65900abd30 0x7f65900ae170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:26.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.409+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== osd.7 v2:172.21.15.155:6816/283518572 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f65900ae840 con 0x7f65900abd30 2024-05-08T23:18:26.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.425+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 --> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65a80616a0 con 0x7f65900abd30 2024-05-08T23:18:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.426+0000 7f65a57fa700 1 -- 172.21.15.16:0/2358761034 <== osd.7 v2:172.21.15.155:6816/283518572 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f65a80616a0 con 0x7f65900abd30 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] conn(0x7f65900abd30 msgr2=0x7f65900ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] conn(0x7f65900abd30 0x7f65900ae170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65900a11b0 msgr2=0x7f65900a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65900a11b0 0x7f65900a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6598005950 tx=0x7f659800b410 comp rx=0 tx=0).stop 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 msgr2=0x7f65a80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 --2- 172.21.15.16:0/2358761034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a8101990 0x7f65a80737e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f659c00d950 tx=0x7f659c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65accde700 1 -- 172.21.15.16:0/2358761034 reap_dead start 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 shutdown_connections 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 >> 172.21.15.16:0/2358761034 conn(0x7f65a8078190 msgr2=0x7f65a80fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 shutdown_connections 2024-05-08T23:18:26.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:26.427+0000 7f65ae741700 1 -- 172.21.15.16:0/2358761034 wait complete. 2024-05-08T23:18:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:27.115 INFO:teuthology.orchestra.run.smithi016.stdout:197568495648 2024-05-08T23:18:27.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T23:18:27.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.567+0000 7f550e5d5700 1 -- 172.21.15.16:0/2295063035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508107030 msgr2=0x7f5508107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 --2- 172.21.15.16:0/2295063035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508107030 0x7f5508107410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009990 tx=0x7f54f8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 -- 172.21.15.16:0/2295063035 shutdown_connections 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 --2- 172.21.15.16:0/2295063035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508101990 0x7f550810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 --2- 172.21.15.16:0/2295063035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508107030 0x7f5508107410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 --2- 172.21.15.16:0/2295063035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5508100ff0 0x7f5508101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 -- 172.21.15.16:0/2295063035 >> 172.21.15.16:0/2295063035 conn(0x7f5508078190 msgr2=0x7f55080785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 -- 172.21.15.16:0/2295063035 shutdown_connections 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 -- 172.21.15.16:0/2295063035 wait complete. 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 Processor -- start 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.568+0000 7f550e5d5700 1 -- start start 2024-05-08T23:18:27.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550e5d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508100ff0 0x7f55081972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550e5d5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5508101990 0x7f55081977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550e5d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508107030 0x7f550819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550e5d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819c1b0 con 0x7f5508100ff0 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550e5d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819c320 con 0x7f5508107030 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550e5d5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550819c4e0 con 0x7f5508101990 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508100ff0 0x7f55081972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508100ff0 0x7f55081972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34670/0 (socket says 172.21.15.16:34670) 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 -- 172.21.15.16:0/245603362 learned_addr learned my addr 172.21.15.16:0/245603362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f55077fe700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5508101990 0x7f55081977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550cb72700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508107030 0x7f550819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 -- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5508101990 msgr2=0x7f55081977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5508101990 0x7f55081977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 -- 172.21.15.16:0/245603362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508107030 msgr2=0x7f550819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508107030 0x7f550819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 -- 172.21.15.16:0/245603362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80095f0 con 0x7f5508100ff0 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f550cb72700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5508107030 0x7f550819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.569+0000 7f5507fff700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508100ff0 0x7f55081972b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54f8006010 tx=0x7f54f8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.570+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f801d070 con 0x7f5508100ff0 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.570+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54f800fc80 con 0x7f5508100ff0 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.570+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550819c760 con 0x7f5508100ff0 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.570+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f8022ce0 con 0x7f5508100ff0 2024-05-08T23:18:27.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.570+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55081a4020 con 0x7f5508100ff0 2024-05-08T23:18:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.571+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55081026e0 con 0x7f5508100ff0 2024-05-08T23:18:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.572+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f54f8022470 con 0x7f5508100ff0 2024-05-08T23:18:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.573+0000 7f55057fa700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f54f00a1230 0x7f54f00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:27.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.573+0000 7f55077fe700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f54f00a1230 0x7f54f00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.573+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54f80cda60 con 0x7f5508100ff0 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.573+0000 7f55057fa700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] conn(0x7f54f00abdb0 0x7f54f00ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.573+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 --> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54f00ae8c0 con 0x7f54f00abdb0 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.573+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f80d22c0 con 0x7f5508100ff0 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.574+0000 7f55077fe700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f54f00a1230 0x7f54f00a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005950 tx=0x7f54fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.574+0000 7f550cb72700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] conn(0x7f54f00abdb0 0x7f54f00ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:27.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.574+0000 7f550cb72700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] conn(0x7f54f00abdb0 0x7f54f00ae1f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:27.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.575+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== osd.11 v2:172.21.15.189:6824/2978765234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f54f00ae8c0 con 0x7f54f00abdb0 2024-05-08T23:18:27.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.592+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 --> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5508110fa0 con 0x7f54f00abdb0 2024-05-08T23:18:27.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f55057fa700 1 -- 172.21.15.16:0/245603362 <== osd.11 v2:172.21.15.189:6824/2978765234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5508110fa0 con 0x7f54f00abdb0 2024-05-08T23:18:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] conn(0x7f54f00abdb0 msgr2=0x7f54f00ae1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] conn(0x7f54f00abdb0 0x7f54f00ae1f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f54f00a1230 msgr2=0x7f54f00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:27.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f54f00a1230 0x7f54f00a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005950 tx=0x7f54fc0058e0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508100ff0 msgr2=0x7f55081972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 --2- 172.21.15.16:0/245603362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5508100ff0 0x7f55081972b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54f8006010 tx=0x7f54f8009ee0 comp rx=0 tx=0).stop 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550cb72700 1 -- 172.21.15.16:0/245603362 reap_dead start 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 shutdown_connections 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 >> 172.21.15.16:0/245603362 conn(0x7f5508078190 msgr2=0x7f55080fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.593+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 shutdown_connections 2024-05-08T23:18:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:27.594+0000 7f550e5d5700 1 -- 172.21.15.16:0/245603362 wait complete. 2024-05-08T23:18:27.733 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:27.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:27 smithi016 ceph-mon[42541]: pgmap v243: 129 pgs: 129 active+clean; 635 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-05-08T23:18:28.005 INFO:teuthology.orchestra.run.smithi016.stdout:309237645331 2024-05-08T23:18:28.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T23:18:28.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:27 smithi189 ceph-mon[46919]: pgmap v243: 129 pgs: 129 active+clean; 635 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-05-08T23:18:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:27 smithi155 ceph-mon[47105]: pgmap v243: 129 pgs: 129 active+clean; 635 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-05-08T23:18:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.412+0000 7f3057d98700 1 -- 172.21.15.16:0/762551277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050069210 msgr2=0x7f3050069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.412+0000 7f3057d98700 1 --2- 172.21.15.16:0/762551277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050069210 0x7f3050069690 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3040009990 tx=0x7f3040009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.412+0000 7f3057d98700 1 -- 172.21.15.16:0/762551277 shutdown_connections 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.412+0000 7f3057d98700 1 --2- 172.21.15.16:0/762551277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069bd0 0x7f305010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.413+0000 7f3057d98700 1 --2- 172.21.15.16:0/762551277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3050106fe0 0x7f30501073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.413+0000 7f3057d98700 1 --2- 172.21.15.16:0/762551277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050069210 0x7f3050069690 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.413+0000 7f3057d98700 1 -- 172.21.15.16:0/762551277 >> 172.21.15.16:0/762551277 conn(0x7f3050076e70 msgr2=0x7f3050077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.413+0000 7f3057d98700 1 -- 172.21.15.16:0/762551277 shutdown_connections 2024-05-08T23:18:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.413+0000 7f3057d98700 1 -- 172.21.15.16:0/762551277 wait complete. 2024-05-08T23:18:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 Processor -- start 2024-05-08T23:18:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 -- start start 2024-05-08T23:18:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069210 0x7f3050199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3050069bd0 0x7f3050199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050106fe0 0x7f305019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305019e330 con 0x7f3050069bd0 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305019e4a0 con 0x7f3050069210 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3057d98700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305019e660 con 0x7f3050106fe0 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3055b34700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069210 0x7f3050199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3055b34700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069210 0x7f3050199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38838/0 (socket says 172.21.15.16:38838) 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.414+0000 7f3055b34700 1 -- 172.21.15.16:0/173952764 learned_addr learned my addr 172.21.15.16:0/173952764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3056335700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050106fe0 0x7f305019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055333700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3050069bd0 0x7f3050199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055b34700 1 -- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050106fe0 msgr2=0x7f305019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055b34700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050106fe0 0x7f305019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055b34700 1 -- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3050069bd0 msgr2=0x7f3050199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055b34700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3050069bd0 0x7f3050199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055b34700 1 -- 172.21.15.16:0/173952764 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c009520 con 0x7f3050069210 2024-05-08T23:18:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3056335700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3050106fe0 0x7f305019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055333700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3050069bd0 0x7f3050199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3055b34700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069210 0x7f3050199430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f304c00eb30 tx=0x7f304c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c009880 con 0x7f3050069210 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f304c0044b0 con 0x7f3050069210 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30400095f0 con 0x7f3050069210 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c003e40 con 0x7f3050069210 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.415+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30501a5fd0 con 0x7f3050069210 2024-05-08T23:18:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.416+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3050102690 con 0x7f3050069210 2024-05-08T23:18:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.418+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f304c003fa0 con 0x7f3050069210 2024-05-08T23:18:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.418+0000 7f3046ffd700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f303c0a1230 0x7f303c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f304c014070 con 0x7f3050069210 2024-05-08T23:18:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3055333700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f303c0a1230 0x7f303c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3046ffd700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] conn(0x7f303c0abdb0 0x7f303c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 --> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f303c0ae8c0 con 0x7f303c0abdb0 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f304c0cd5a0 con 0x7f3050069210 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3055333700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f303c0a1230 0x7f303c0a36f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3040000c00 tx=0x7f3040005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.419+0000 7f3056335700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] conn(0x7f303c0abdb0 0x7f303c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.420+0000 7f3056335700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] conn(0x7f303c0abdb0 0x7f303c0ae1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.420+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== osd.5 v2:172.21.15.189:6808/925680266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f303c0ae8c0 con 0x7f303c0abdb0 2024-05-08T23:18:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.438+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 --> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3050101c90 con 0x7f303c0abdb0 2024-05-08T23:18:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.439+0000 7f3046ffd700 1 -- 172.21.15.16:0/173952764 <== osd.5 v2:172.21.15.189:6808/925680266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3050101c90 con 0x7f303c0abdb0 2024-05-08T23:18:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.439+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] conn(0x7f303c0abdb0 msgr2=0x7f303c0ae1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.439+0000 7f3057d98700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] conn(0x7f303c0abdb0 0x7f303c0ae1f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.440 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f303c0a1230 msgr2=0x7f303c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f303c0a1230 0x7f303c0a36f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3040000c00 tx=0x7f3040005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069210 msgr2=0x7f3050199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 --2- 172.21.15.16:0/173952764 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3050069210 0x7f3050199430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f304c00eb30 tx=0x7f304c00eef0 comp rx=0 tx=0).stop 2024-05-08T23:18:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3056335700 1 -- 172.21.15.16:0/173952764 reap_dead start 2024-05-08T23:18:28.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 shutdown_connections 2024-05-08T23:18:28.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 >> 172.21.15.16:0/173952764 conn(0x7f3050076e70 msgr2=0x7f30500fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:28.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 shutdown_connections 2024-05-08T23:18:28.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:28.441+0000 7f3057d98700 1 -- 172.21.15.16:0/173952764 wait complete. 2024-05-08T23:18:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:29.072 INFO:teuthology.orchestra.run.smithi016.stdout:150323855399 2024-05-08T23:18:29.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T23:18:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:30.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:29 smithi016 ceph-mon[42541]: pgmap v244: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-08T23:18:30.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:29 smithi189 ceph-mon[46919]: pgmap v244: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-08T23:18:30.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:29 smithi155 ceph-mon[47105]: pgmap v244: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-05-08T23:18:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 -- 172.21.15.16:0/1304589191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c105f40 msgr2=0x7fc75c106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 --2- 172.21.15.16:0/1304589191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c105f40 0x7fc75c106320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc750009990 tx=0x7fc750009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 -- 172.21.15.16:0/1304589191 shutdown_connections 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 --2- 172.21.15.16:0/1304589191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c100850 0x7fc75c10ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 --2- 172.21.15.16:0/1304589191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c105f40 0x7fc75c106320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 --2- 172.21.15.16:0/1304589191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc75c0ffe90 0x7fc75c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 -- 172.21.15.16:0/1304589191 >> 172.21.15.16:0/1304589191 conn(0x7fc75c076e70 msgr2=0x7fc75c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 -- 172.21.15.16:0/1304589191 shutdown_connections 2024-05-08T23:18:31.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.993+0000 7fc762c54700 1 -- 172.21.15.16:0/1304589191 wait complete. 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 Processor -- start 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 -- start start 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c0ffe90 0x7fc75c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc75c100850 0x7fc75c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c105f40 0x7fc75c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c19c0d0 con 0x7fc75c0ffe90 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c19c240 con 0x7fc75c105f40 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc762c54700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c19c400 con 0x7fc75c100850 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc7609f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c0ffe90 0x7fc75c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc7609f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c0ffe90 0x7fc75c197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34704/0 (socket says 172.21.15.16:34704) 2024-05-08T23:18:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.994+0000 7fc7609f0700 1 -- 172.21.15.16:0/3717072528 learned_addr learned my addr 172.21.15.16:0/3717072528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc75bfff700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc75c100850 0x7fc75c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7611f1700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c105f40 0x7fc75c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7609f0700 1 -- 172.21.15.16:0/3717072528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc75c100850 msgr2=0x7fc75c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7609f0700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc75c100850 0x7fc75c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7609f0700 1 -- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c105f40 msgr2=0x7fc75c19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7609f0700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c105f40 0x7fc75c19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7609f0700 1 -- 172.21.15.16:0/3717072528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500095f0 con 0x7fc75c0ffe90 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc7609f0700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c0ffe90 0x7fc75c197220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc750006010 tx=0x7fc750005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75001d070 con 0x7fc75c0ffe90 2024-05-08T23:18:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc75000fd40 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c19c680 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750022d10 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.995+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c1a3f40 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.996+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc75c19c950 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.997+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc750022470 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.998+0000 7fc759ffb700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc74c0a11b0 0x7fc74c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.998+0000 7fc75bfff700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc74c0a11b0 0x7fc74c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.998+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc7500cebd0 con 0x7fc75c0ffe90 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.998+0000 7fc759ffb700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] conn(0x7fc74c0abd30 0x7fc74c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.998+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 --> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc74c0ae840 con 0x7fc74c0abd30 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.998+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7500cef40 con 0x7fc75c0ffe90 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.999+0000 7fc75bfff700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc74c0a11b0 0x7fc74c0a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc748005fd0 tx=0x7fc74800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.999+0000 7fc7611f1700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] conn(0x7fc74c0abd30 0x7fc74c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:30.999+0000 7fc7611f1700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] conn(0x7fc74c0abd30 0x7fc74c0ae170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.000+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== osd.4 v2:172.21.15.155:6808/2036488025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc74c0ae840 con 0x7fc74c0abd30 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.017+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 --> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc75c10ff00 con 0x7fc74c0abd30 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc759ffb700 1 -- 172.21.15.16:0/3717072528 <== osd.4 v2:172.21.15.155:6808/2036488025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc75c10ff00 con 0x7fc74c0abd30 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] conn(0x7fc74c0abd30 msgr2=0x7fc74c0ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] conn(0x7fc74c0abd30 0x7fc74c0ae170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc74c0a11b0 msgr2=0x7fc74c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc74c0a11b0 0x7fc74c0a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc748005fd0 tx=0x7fc74800a560 comp rx=0 tx=0).stop 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c0ffe90 msgr2=0x7fc75c197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 --2- 172.21.15.16:0/3717072528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75c0ffe90 0x7fc75c197220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc750006010 tx=0x7fc750005740 comp rx=0 tx=0).stop 2024-05-08T23:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc7611f1700 1 -- 172.21.15.16:0/3717072528 reap_dead start 2024-05-08T23:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 shutdown_connections 2024-05-08T23:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 >> 172.21.15.16:0/3717072528 conn(0x7fc75c076e70 msgr2=0x7fc75c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 shutdown_connections 2024-05-08T23:18:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:31.018+0000 7fc762c54700 1 -- 172.21.15.16:0/3717072528 wait complete. 2024-05-08T23:18:32.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:31 smithi016 ceph-mon[42541]: pgmap v245: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:18:32.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:31 smithi189 ceph-mon[46919]: pgmap v245: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:18:32.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:31 smithi155 ceph-mon[47105]: pgmap v245: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:18:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 -- 172.21.15.16:0/2551140913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01009d0 msgr2=0x7f65a0100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2551140913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01009d0 0x7f65a0100db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f65900099c0 tx=0x7f6590009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 -- 172.21.15.16:0/2551140913 shutdown_connections 2024-05-08T23:18:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2551140913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a010da80 0x7f65a010fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2551140913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01009d0 0x7f65a0100db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2551140913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a01012f0 0x7f65a010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 -- 172.21.15.16:0/2551140913 >> 172.21.15.16:0/2551140913 conn(0x7f65a0075f80 msgr2=0x7f65a0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 -- 172.21.15.16:0/2551140913 shutdown_connections 2024-05-08T23:18:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.933+0000 7f65a4d55700 1 -- 172.21.15.16:0/2551140913 wait complete. 2024-05-08T23:18:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.934+0000 7f65a4d55700 1 Processor -- start 2024-05-08T23:18:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.934+0000 7f65a4d55700 1 -- start start 2024-05-08T23:18:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f65a4d55700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a01009d0 0x7f65a01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f65a4d55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01012f0 0x7f65a0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f65a4d55700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a010da80 0x7f65a019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f65a4d55700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a019e3d0 con 0x7f65a01012f0 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f65a4d55700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a019e540 con 0x7f65a01009d0 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f65a4d55700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a019e700 con 0x7f65a010da80 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01012f0 0x7f65a0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01012f0 0x7f65a0199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34720/0 (socket says 172.21.15.16:34720) 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 -- 172.21.15.16:0/2573150128 learned_addr learned my addr 172.21.15.16:0/2573150128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659ed9d700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a010da80 0x7f65a019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659e59c700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a01009d0 0x7f65a01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 -- 172.21.15.16:0/2573150128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a010da80 msgr2=0x7f65a019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65a010da80 0x7f65a019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 -- 172.21.15.16:0/2573150128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a01009d0 msgr2=0x7f65a01994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65a01009d0 0x7f65a01994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 -- 172.21.15.16:0/2573150128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900095f0 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f659dd9b700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01012f0 0x7f65a0199a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f659400d950 tx=0x7f659400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.935+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594009940 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.936+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a019e9e0 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.936+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6594010460 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.936+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400f620 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.936+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a01a6240 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.936+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a0113240 con 0x7f65a01012f0 2024-05-08T23:18:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.938+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f65940105d0 con 0x7f65a01012f0 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f658f7fe700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65880a11b0 0x7f65880a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f659e59c700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65880a11b0 0x7f65880a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f65940ccf90 con 0x7f65a01012f0 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f658f7fe700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] conn(0x7f65880abd30 0x7f65880ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 --> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65880ae840 con 0x7f65880abd30 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65940cd3a0 con 0x7f65a01012f0 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f659ed9d700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] conn(0x7f65880abd30 0x7f65880ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f659e59c700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65880a11b0 0x7f65880a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6590009fd0 tx=0x7f6590005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.939+0000 7f659ed9d700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] conn(0x7f65880abd30 0x7f65880ae170 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.940+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== osd.6 v2:172.21.15.16:6818/3858453894 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f65880ae840 con 0x7f65880abd30 2024-05-08T23:18:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.959+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 --> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65a0061620 con 0x7f65880abd30 2024-05-08T23:18:32.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.960+0000 7f658f7fe700 1 -- 172.21.15.16:0/2573150128 <== osd.6 v2:172.21.15.16:6818/3858453894 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f65a0061620 con 0x7f65880abd30 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.960+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] conn(0x7f65880abd30 msgr2=0x7f65880ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.960+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] conn(0x7f65880abd30 0x7f65880ae170 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65880a11b0 msgr2=0x7f65880a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f65880a11b0 0x7f65880a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6590009fd0 tx=0x7f6590005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01012f0 msgr2=0x7f65a0199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 --2- 172.21.15.16:0/2573150128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a01012f0 0x7f65a0199a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f659400d950 tx=0x7f659400dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f659ed9d700 1 -- 172.21.15.16:0/2573150128 reap_dead start 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 shutdown_connections 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 >> 172.21.15.16:0/2573150128 conn(0x7f65a0075f80 msgr2=0x7f65a010f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 shutdown_connections 2024-05-08T23:18:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:32.961+0000 7f65a4d55700 1 -- 172.21.15.16:0/2573150128 wait complete. 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 -- 172.21.15.16:0/630279068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c107030 msgr2=0x7ff61c107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 --2- 172.21.15.16:0/630279068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c107030 0x7ff61c107410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009990 tx=0x7ff60c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 -- 172.21.15.16:0/630279068 shutdown_connections 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 --2- 172.21.15.16:0/630279068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c06a040 0x7ff61c10de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 --2- 172.21.15.16:0/630279068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c107030 0x7ff61c107410 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 --2- 172.21.15.16:0/630279068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff61c069680 0x7ff61c069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 -- 172.21.15.16:0/630279068 >> 172.21.15.16:0/630279068 conn(0x7ff61c076ec0 msgr2=0x7ff61c0772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 -- 172.21.15.16:0/630279068 shutdown_connections 2024-05-08T23:18:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 -- 172.21.15.16:0/630279068 wait complete. 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.023+0000 7ff6228cb700 1 Processor -- start 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 -- start start 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c069680 0x7ff61c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff61c06a040 0x7ff61c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c107030 0x7ff61c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19c230 con 0x7ff61c069680 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19c3a0 con 0x7ff61c107030 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff6228cb700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19c560 con 0x7ff61c06a040 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c069680 0x7ff61c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff61bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c069680 0x7ff61c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34736/0 (socket says 172.21.15.16:34736) 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff61bfff700 1 -- 172.21.15.16:0/1487931371 learned_addr learned my addr 172.21.15.16:0/1487931371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.024+0000 7ff61b7fe700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff61c06a040 0x7ff61c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff620e68700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c107030 0x7ff61c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61bfff700 1 -- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff61c06a040 msgr2=0x7ff61c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61bfff700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff61c06a040 0x7ff61c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61bfff700 1 -- 172.21.15.16:0/1487931371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c107030 msgr2=0x7ff61c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61bfff700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c107030 0x7ff61c19bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61bfff700 1 -- 172.21.15.16:0/1487931371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0095f0 con 0x7ff61c069680 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61b7fe700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff61c06a040 0x7ff61c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff620e68700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff61c107030 0x7ff61c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff61bfff700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c069680 0x7ff61c197380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b650 tx=0x7ff60c00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c01d070 con 0x7ff61c069680 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff60c022470 con 0x7ff61c069680 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c19c7e0 con 0x7ff61c069680 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c017650 con 0x7ff61c069680 2024-05-08T23:18:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.025+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c1a40a0 con 0x7ff61c069680 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.026+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff61c19cab0 con 0x7ff61c069680 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.027+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff60c0225e0 con 0x7ff61c069680 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.028+0000 7ff6197fa700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff6040a11b0 0x7ff6040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.028+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff60c0ce290 con 0x7ff61c069680 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.028+0000 7ff61b7fe700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff6040a11b0 0x7ff6040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.028+0000 7ff6197fa700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] conn(0x7ff6040abd30 0x7ff6040ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.028+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 --> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6040ae840 con 0x7ff6040abd30 2024-05-08T23:18:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.028+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60c0ce6a0 con 0x7ff61c069680 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.029+0000 7ff61b7fe700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff6040a11b0 0x7ff6040a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff61c198890 tx=0x7ff610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.029+0000 7ff620e68700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] conn(0x7ff6040abd30 0x7ff6040ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.029+0000 7ff620e68700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] conn(0x7ff6040abd30 0x7ff6040ae170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.030+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== osd.8 v2:172.21.15.189:6816/2123508255 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff6040ae840 con 0x7ff6040abd30 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.048+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 --> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff61c110fd0 con 0x7ff6040abd30 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6197fa700 1 -- 172.21.15.16:0/1487931371 <== osd.8 v2:172.21.15.189:6816/2123508255 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff61c110fd0 con 0x7ff6040abd30 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] conn(0x7ff6040abd30 msgr2=0x7ff6040ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] conn(0x7ff6040abd30 0x7ff6040ae170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff6040a11b0 msgr2=0x7ff6040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff6040a11b0 0x7ff6040a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff61c198890 tx=0x7ff610009260 comp rx=0 tx=0).stop 2024-05-08T23:18:33.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c069680 msgr2=0x7ff61c197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 --2- 172.21.15.16:0/1487931371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c069680 0x7ff61c197380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b650 tx=0x7ff60c00bfc0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff620e68700 1 -- 172.21.15.16:0/1487931371 reap_dead start 2024-05-08T23:18:33.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 shutdown_connections 2024-05-08T23:18:33.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 >> 172.21.15.16:0/1487931371 conn(0x7ff61c076ec0 msgr2=0x7ff61c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:33.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 shutdown_connections 2024-05-08T23:18:33.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.049+0000 7ff6228cb700 1 -- 172.21.15.16:0/1487931371 wait complete. 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.921+0000 7f8cd5878700 1 -- 172.21.15.16:0/2305557203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd00ff940 msgr2=0x7f8cd00ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.921+0000 7f8cd5878700 1 --2- 172.21.15.16:0/2305557203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd00ff940 0x7f8cd00ffdc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400aaa0 tx=0x7f8cc400adb0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 -- 172.21.15.16:0/2305557203 shutdown_connections 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 --2- 172.21.15.16:0/2305557203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd0100300 0x7f8cd010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 --2- 172.21.15.16:0/2305557203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd0104650 0x7f8cd0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 --2- 172.21.15.16:0/2305557203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd00ff940 0x7f8cd00ffdc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 -- 172.21.15.16:0/2305557203 >> 172.21.15.16:0/2305557203 conn(0x7f8cd0075260 msgr2=0x7f8cd0075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 -- 172.21.15.16:0/2305557203 shutdown_connections 2024-05-08T23:18:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 -- 172.21.15.16:0/2305557203 wait complete. 2024-05-08T23:18:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f8cd5878700 1 Processor -- start 2024-05-08T23:18:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 -- start start 2024-05-08T23:18:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd00ff940 0x7f8cd01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd0100300 0x7f8cd01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd0104650 0x7f8cd019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019e360 con 0x7f8cd0100300 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019e4d0 con 0x7f8cd0104650 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cd5878700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019e690 con 0x7f8cd00ff940 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cce7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd0100300 0x7f8cd01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cce7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd0100300 0x7f8cd01999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34770/0 (socket says 172.21.15.16:34770) 2024-05-08T23:18:33.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cce7fc700 1 -- 172.21.15.16:0/4032111473 learned_addr learned my addr 172.21.15.16:0/4032111473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.923+0000 7f8cceffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd00ff940 0x7f8cd01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8ccf7fe700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd0104650 0x7f8cd019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cceffd700 1 -- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd0104650 msgr2=0x7f8cd019dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cceffd700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8cd0104650 0x7f8cd019dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cceffd700 1 -- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd0100300 msgr2=0x7f8cd01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cceffd700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd0100300 0x7f8cd01999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cceffd700 1 -- 172.21.15.16:0/4032111473 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009520 con 0x7f8cd00ff940 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cceffd700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd00ff940 0x7f8cd01994b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000e9f0 tx=0x7f8cc000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000cb70 con 0x7f8cd00ff940 2024-05-08T23:18:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8cc000ee90 con 0x7f8cd00ff940 2024-05-08T23:18:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc400a700 con 0x7f8cd00ff940 2024-05-08T23:18:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.924+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd01a6000 con 0x7f8cd00ff940 2024-05-08T23:18:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.925+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0003bc0 con 0x7f8cd00ff940 2024-05-08T23:18:33.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.925+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cbc000fc0 con 0x7f8cd00ff940 2024-05-08T23:18:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.927+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8cc000ccd0 con 0x7f8cd00ff940 2024-05-08T23:18:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.928+0000 7f8cd4876700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8cb80a1230 0x7f8cb80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.928+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8cc0014070 con 0x7f8cd00ff940 2024-05-08T23:18:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.928+0000 7f8cd4876700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] conn(0x7f8cb80abdb0 0x7f8cb80ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.928+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 --> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cb80ae8c0 con 0x7f8cb80abdb0 2024-05-08T23:18:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.928+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc00cd250 con 0x7f8cd00ff940 2024-05-08T23:18:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.929+0000 7f8ccf7fe700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] conn(0x7f8cb80abdb0 0x7f8cb80ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.929+0000 7f8cce7fc700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8cb80a1230 0x7f8cb80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.929+0000 7f8ccf7fe700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] conn(0x7f8cb80abdb0 0x7f8cb80ae1f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.930+0000 7f8cce7fc700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8cb80a1230 0x7f8cb80a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8cd019a9c0 tx=0x7f8cc4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.931+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== osd.1 v2:172.21.15.155:6800/2944272573 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8cb80ae8c0 con 0x7f8cb80abdb0 2024-05-08T23:18:33.931 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:33 smithi016 ceph-mon[42541]: pgmap v246: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:18:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.952+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 --> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cbc002d40 con 0x7f8cb80abdb0 2024-05-08T23:18:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.953+0000 7f8cd4876700 1 -- 172.21.15.16:0/4032111473 <== osd.1 v2:172.21.15.155:6800/2944272573 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8cbc002d40 con 0x7f8cb80abdb0 2024-05-08T23:18:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.953+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] conn(0x7f8cb80abdb0 msgr2=0x7f8cb80ae1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.953+0000 7f8cd5878700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] conn(0x7f8cb80abdb0 0x7f8cb80ae1f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8cb80a1230 msgr2=0x7f8cb80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8cb80a1230 0x7f8cb80a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8cd019a9c0 tx=0x7f8cc4005960 comp rx=0 tx=0).stop 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd00ff940 msgr2=0x7f8cd01994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 --2- 172.21.15.16:0/4032111473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd00ff940 0x7f8cd01994b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000e9f0 tx=0x7f8cc000ed00 comp rx=0 tx=0).stop 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8ccf7fe700 1 -- 172.21.15.16:0/4032111473 reap_dead start 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 shutdown_connections 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 >> 172.21.15.16:0/4032111473 conn(0x7f8cd0075260 msgr2=0x7f8cd00fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 shutdown_connections 2024-05-08T23:18:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.954+0000 7f8cd5878700 1 -- 172.21.15.16:0/4032111473 wait complete. 2024-05-08T23:18:34.088 INFO:teuthology.orchestra.run.smithi016.stdout:128849018923 2024-05-08T23:18:34.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T23:18:34.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:33 smithi189 ceph-mon[46919]: pgmap v246: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:18:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 -- 172.21.15.16:0/2107749738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186010da40 msgr2=0x7f186010fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 --2- 172.21.15.16:0/2107749738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186010da40 0x7f186010fe30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f185800b0a0 tx=0x7f185800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 -- 172.21.15.16:0/2107749738 shutdown_connections 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 --2- 172.21.15.16:0/2107749738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186010da40 0x7f186010fe30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 --2- 172.21.15.16:0/2107749738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1860069220 0x7f1860069600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 --2- 172.21.15.16:0/2107749738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1860069b40 0x7f186010d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 -- 172.21.15.16:0/2107749738 >> 172.21.15.16:0/2107749738 conn(0x7f1860076e70 msgr2=0x7f1860077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 -- 172.21.15.16:0/2107749738 shutdown_connections 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.893+0000 7f18670d5700 1 -- 172.21.15.16:0/2107749738 wait complete. 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.894+0000 7f18670d5700 1 Processor -- start 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.894+0000 7f18670d5700 1 -- start start 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f18670d5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1860069220 0x7f18601995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f18670d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860069b40 0x7f1860199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f18670d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186010da40 0x7f186019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f18670d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019e4f0 con 0x7f1860069b40 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f18670d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019e660 con 0x7f186010da40 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f18670d5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019e820 con 0x7f1860069220 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860069b40 0x7f1860199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860069b40 0x7f1860199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34758/0 (socket says 172.21.15.16:34758) 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 -- 172.21.15.16:0/2956604440 learned_addr learned my addr 172.21.15.16:0/2956604440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f1865672700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186010da40 0x7f186019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 -- 172.21.15.16:0/2956604440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1860069220 msgr2=0x7f18601995f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1860069220 0x7f18601995f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 -- 172.21.15.16:0/2956604440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186010da40 msgr2=0x7f186019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186010da40 0x7f186019dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f185ffff700 1 -- 172.21.15.16:0/2956604440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1858009d00 con 0x7f1860069b40 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.895+0000 7f1865672700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f186010da40 0x7f186019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.896+0000 7f185ffff700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860069b40 0x7f1860199b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f184c00e9a0 tx=0x7f184c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.896+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c009790 con 0x7f1860069b40 2024-05-08T23:18:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.896+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186019eb00 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.896+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f184c00cbd0 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.896+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c018680 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.896+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18601a6360 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.897+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1860113180 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.900+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f184c00cd40 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.900+0000 7f185dffb700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18500a11b0 0x7f18500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f184c014070 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f1864e71700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18500a11b0 0x7f18500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f185dffb700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] conn(0x7f18500abd30 0x7f18500ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 --> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18500ae840 con 0x7f18500abd30 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f184c0d55a0 con 0x7f1860069b40 2024-05-08T23:18:34.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f1865672700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] conn(0x7f18500abd30 0x7f18500ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f1864e71700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18500a11b0 0x7f18500a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1854009af0 tx=0x7f1854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.901+0000 7f1865672700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] conn(0x7f18500abd30 0x7f18500ae170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.902+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== osd.3 v2:172.21.15.16:6810/2397977008 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f18500ae840 con 0x7f18500abd30 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.921+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 --> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1860061620 con 0x7f18500abd30 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f185dffb700 1 -- 172.21.15.16:0/2956604440 <== osd.3 v2:172.21.15.16:6810/2397977008 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1860061620 con 0x7f18500abd30 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] conn(0x7f18500abd30 msgr2=0x7f18500ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] conn(0x7f18500abd30 0x7f18500ae170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18500a11b0 msgr2=0x7f18500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18500a11b0 0x7f18500a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1854009af0 tx=0x7f1854009260 comp rx=0 tx=0).stop 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860069b40 msgr2=0x7f1860199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 --2- 172.21.15.16:0/2956604440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1860069b40 0x7f1860199b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f184c00e9a0 tx=0x7f184c00ed60 comp rx=0 tx=0).stop 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f1865672700 1 -- 172.21.15.16:0/2956604440 reap_dead start 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 shutdown_connections 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.922+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 >> 172.21.15.16:0/2956604440 conn(0x7f1860076e70 msgr2=0x7f186010f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.925+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 shutdown_connections 2024-05-08T23:18:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:33.925+0000 7f18670d5700 1 -- 172.21.15.16:0/2956604440 wait complete. 2024-05-08T23:18:34.205 INFO:teuthology.orchestra.run.smithi016.stdout:227633266719 2024-05-08T23:18:34.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T23:18:34.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:33 smithi155 ceph-mon[47105]: pgmap v246: 129 pgs: 129 active+clean; 639 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-05-08T23:18:34.589 INFO:teuthology.orchestra.run.smithi016.stdout:180388626470 2024-05-08T23:18:34.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T23:18:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.885+0000 7fcec5068700 1 -- 172.21.15.16:0/435266414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 msgr2=0x7fcec010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.885+0000 7fcec5068700 1 --2- 172.21.15.16:0/435266414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec010de60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fceb800b0d0 tx=0x7fceb800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 -- 172.21.15.16:0/435266414 shutdown_connections 2024-05-08T23:18:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 --2- 172.21.15.16:0/435266414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec010de60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 --2- 172.21.15.16:0/435266414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0107020 0x7fcec0107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 --2- 172.21.15.16:0/435266414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec0068950 0x7fcec0068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 -- 172.21.15.16:0/435266414 >> 172.21.15.16:0/435266414 conn(0x7fcec0075f80 msgr2=0x7fcec0076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 -- 172.21.15.16:0/435266414 shutdown_connections 2024-05-08T23:18:34.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.886+0000 7fcec5068700 1 -- 172.21.15.16:0/435266414 wait complete. 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 Processor -- start 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 -- start start 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0068950 0x7fcec01973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec0197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec0107020 0x7fcec019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec019c2d0 con 0x7fcec00692f0 2024-05-08T23:18:34.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec019c440 con 0x7fcec0068950 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.887+0000 7fcec5068700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec019c600 con 0x7fcec0107020 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec0197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec0197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34814/0 (socket says 172.21.15.16:34814) 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 -- 172.21.15.16:0/1890204055 learned_addr learned my addr 172.21.15.16:0/1890204055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebed9d700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0068950 0x7fcec01973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebf59e700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec0107020 0x7fcec019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 -- 172.21.15.16:0/1890204055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec0107020 msgr2=0x7fcec019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec0107020 0x7fcec019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 -- 172.21.15.16:0/1890204055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0068950 msgr2=0x7fcec01973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcec0068950 0x7fcec01973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 -- 172.21.15.16:0/1890204055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb8009d00 con 0x7fcec00692f0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebf59e700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcec0107020 0x7fcec019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcebe59c700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec0197910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fceb400e9a0 tx=0x7fceb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4009790 con 0x7fcec00692f0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec019c8e0 con 0x7fcec00692f0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceb400cbd0 con 0x7fcec00692f0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.888+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4018680 con 0x7fcec00692f0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.889+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec01a4140 con 0x7fcec00692f0 2024-05-08T23:18:34.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.889+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcec0110fc0 con 0x7fcec00692f0 2024-05-08T23:18:34.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.891+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fceb400cd40 con 0x7fcec00692f0 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.891+0000 7fcea7fff700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcea80a11b0 0x7fcea80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.891+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fceb4014070 con 0x7fcec00692f0 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.891+0000 7fcebed9d700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcea80a11b0 0x7fcea80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.892+0000 7fcea7fff700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] conn(0x7fcea80abd30 0x7fcea80ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.892+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 --> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcea80ae840 con 0x7fcea80abd30 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.892+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceb40cd760 con 0x7fcec00692f0 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.892+0000 7fcebf59e700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] conn(0x7fcea80abd30 0x7fcea80ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.892+0000 7fcebed9d700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcea80a11b0 0x7fcea80a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009be0 tx=0x7fceb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.892+0000 7fcebf59e700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] conn(0x7fcea80abd30 0x7fcea80ae170 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.893+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== osd.0 v2:172.21.15.16:6802/3035598518 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fcea80ae840 con 0x7fcea80abd30 2024-05-08T23:18:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 -- 172.21.15.16:0/3502784264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 msgr2=0x7f8e84068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/3502784264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84068db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009990 tx=0x7f8e74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 -- 172.21.15.16:0/3502784264 shutdown_connections 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/3502784264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e840692f0 0x7f8e8410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/3502784264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e84107020 0x7f8e84107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/3502784264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84068db0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 -- 172.21.15.16:0/3502784264 >> 172.21.15.16:0/3502784264 conn(0x7f8e84075f80 msgr2=0x7f8e84076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 -- 172.21.15.16:0/3502784264 shutdown_connections 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.738+0000 7f8e8bd71700 1 -- 172.21.15.16:0/3502784264 wait complete. 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.739+0000 7f8e8bd71700 1 Processor -- start 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.739+0000 7f8e8bd71700 1 -- start start 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8bd71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8bd71700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e840692f0 0x7f8e84197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8bd71700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e84107020 0x7f8e8419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8bd71700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8419c230 con 0x7f8e84068950 2024-05-08T23:18:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8bd71700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8419c3a0 con 0x7f8e84107020 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8bd71700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8419c560 con 0x7f8e840692f0 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34784/0 (socket says 172.21.15.16:34784) 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 -- 172.21.15.16:0/622184370 learned_addr learned my addr 172.21.15.16:0/622184370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8a30e700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e84107020 0x7f8e8419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e8930c700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e840692f0 0x7f8e84197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 -- 172.21.15.16:0/622184370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e840692f0 msgr2=0x7f8e84197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e840692f0 0x7f8e84197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 -- 172.21.15.16:0/622184370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e84107020 msgr2=0x7f8e8419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e84107020 0x7f8e8419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 -- 172.21.15.16:0/622184370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740095f0 con 0x7f8e84068950 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e89b0d700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84197330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000e9a0 tx=0x7f8e8000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.740+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80009790 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.741+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e8000cbd0 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.741+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8419c840 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.741+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80018680 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.741+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e841a40a0 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.741+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e841026d0 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.743+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8e8000cd40 con 0x7f8e84068950 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e7affd700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e700a11b0 0x7f8e700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8e80014070 con 0x7f8e84068950 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e8930c700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e700a11b0 0x7f8e700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e7affd700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] conn(0x7f8e700abd30 0x7f8e700ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 --> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e700ae840 con 0x7f8e700abd30 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e800cd5e0 con 0x7f8e84068950 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e8a30e700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] conn(0x7f8e700abd30 0x7f8e700ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.744+0000 7f8e8930c700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e700a11b0 0x7f8e700a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e74006010 tx=0x7f8e7400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.745+0000 7f8e8a30e700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] conn(0x7f8e700abd30 0x7f8e700ae170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.746+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== osd.9 v2:172.21.15.16:6826/3426310247 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8e700ae840 con 0x7f8e700abd30 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.766+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 --> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e84111020 con 0x7f8e700abd30 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.766+0000 7f8e7affd700 1 -- 172.21.15.16:0/622184370 <== osd.9 v2:172.21.15.16:6826/3426310247 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8e84111020 con 0x7f8e700abd30 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] conn(0x7f8e700abd30 msgr2=0x7f8e700ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] conn(0x7f8e700abd30 0x7f8e700ae170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e700a11b0 msgr2=0x7f8e700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8e700a11b0 0x7f8e700a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8e74006010 tx=0x7f8e7400b260 comp rx=0 tx=0).stop 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 msgr2=0x7f8e84197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 --2- 172.21.15.16:0/622184370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e84068950 0x7f8e84197330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000e9a0 tx=0x7f8e8000ed60 comp rx=0 tx=0).stop 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8a30e700 1 -- 172.21.15.16:0/622184370 reap_dead start 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 shutdown_connections 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 >> 172.21.15.16:0/622184370 conn(0x7f8e84075f80 msgr2=0x7f8e840fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 shutdown_connections 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.767+0000 7f8e8bd71700 1 -- 172.21.15.16:0/622184370 wait complete. 2024-05-08T23:18:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.911+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 --> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcec00616a0 con 0x7fcea80abd30 2024-05-08T23:18:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.911+0000 7fcea7fff700 1 -- 172.21.15.16:0/1890204055 <== osd.0 v2:172.21.15.16:6802/3035598518 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcec00616a0 con 0x7fcea80abd30 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] conn(0x7fcea80abd30 msgr2=0x7fcea80ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] conn(0x7fcea80abd30 0x7fcea80ae170 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcea80a11b0 msgr2=0x7fcea80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcea80a11b0 0x7fcea80a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009be0 tx=0x7fceb0009260 comp rx=0 tx=0).stop 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 msgr2=0x7fcec0197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 --2- 172.21.15.16:0/1890204055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00692f0 0x7fcec0197910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fceb400e9a0 tx=0x7fceb400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcebf59e700 1 -- 172.21.15.16:0/1890204055 reap_dead start 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 shutdown_connections 2024-05-08T23:18:34.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 >> 172.21.15.16:0/1890204055 conn(0x7fcec0075f80 msgr2=0x7fcec00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 shutdown_connections 2024-05-08T23:18:34.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:34.912+0000 7fcec5068700 1 -- 172.21.15.16:0/1890204055 wait complete. 2024-05-08T23:18:35.012 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:35.035 INFO:teuthology.orchestra.run.smithi016.stdout:98784247855 2024-05-08T23:18:35.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T23:18:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 -- 172.21.15.16:0/2225198198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 msgr2=0x7faf54105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 --2- 172.21.15.16:0/2225198198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54105be0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009990 tx=0x7faf3c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 -- 172.21.15.16:0/2225198198 shutdown_connections 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 --2- 172.21.15.16:0/2225198198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf541062c0 0x7faf5410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 --2- 172.21.15.16:0/2225198198 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf54069960 0x7faf54069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 --2- 172.21.15.16:0/2225198198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54105be0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 -- 172.21.15.16:0/2225198198 >> 172.21.15.16:0/2225198198 conn(0x7faf54076f40 msgr2=0x7faf54077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 -- 172.21.15.16:0/2225198198 shutdown_connections 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.082+0000 7faf592ee700 1 -- 172.21.15.16:0/2225198198 wait complete. 2024-05-08T23:18:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 Processor -- start 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 -- start start 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf54069960 0x7faf54197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf541062c0 0x7faf5419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5419c240 con 0x7faf5406a310 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5419c3b0 con 0x7faf541062c0 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf592ee700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5419c570 con 0x7faf54069960 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf4a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf52ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf54069960 0x7faf54197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf4a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34822/0 (socket says 172.21.15.16:34822) 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf4a7fc700 1 -- 172.21.15.16:0/3387575273 learned_addr learned my addr 172.21.15.16:0/3387575273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.083+0000 7faf537fe700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf541062c0 0x7faf5419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf4a7fc700 1 -- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf54069960 msgr2=0x7faf54197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf4a7fc700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf54069960 0x7faf54197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf4a7fc700 1 -- 172.21.15.16:0/3387575273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf541062c0 msgr2=0x7faf5419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf4a7fc700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf541062c0 0x7faf5419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf4a7fc700 1 -- 172.21.15.16:0/3387575273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0095f0 con 0x7faf5406a310 2024-05-08T23:18:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf4a7fc700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54197880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faf3c004730 tx=0x7faf3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c01d070 con 0x7faf5406a310 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5419c7f0 con 0x7faf5406a310 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf3c00fb00 con 0x7faf5406a310 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c022ae0 con 0x7faf5406a310 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.084+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf541a40b0 con 0x7faf5406a310 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.085+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf54111070 con 0x7faf5406a310 2024-05-08T23:18:35.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.087+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faf3c00fc70 con 0x7faf5406a310 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.087+0000 7faf50ff9700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faf400a11b0 0x7faf400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.087+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faf3c0cea30 con 0x7faf5406a310 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.087+0000 7faf52ffd700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faf400a11b0 0x7faf400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.088+0000 7faf50ff9700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] conn(0x7faf400abd30 0x7faf400ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.088+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 --> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf400ae840 con 0x7faf400abd30 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.088+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf3c0cee40 con 0x7faf5406a310 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.088+0000 7faf52ffd700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faf400a11b0 0x7faf400a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf44007be0 tx=0x7faf440073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.088+0000 7faf537fe700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] conn(0x7faf400abd30 0x7faf400ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.088+0000 7faf537fe700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] conn(0x7faf400abd30 0x7faf400ae170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:35.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.089+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== osd.2 v2:172.21.15.189:6800/3853442888 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7faf400ae840 con 0x7faf400abd30 2024-05-08T23:18:35.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.126+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 --> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf540616a0 con 0x7faf400abd30 2024-05-08T23:18:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.126+0000 7faf50ff9700 1 -- 172.21.15.16:0/3387575273 <== osd.2 v2:172.21.15.189:6800/3853442888 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faf540616a0 con 0x7faf400abd30 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] conn(0x7faf400abd30 msgr2=0x7faf400ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] conn(0x7faf400abd30 0x7faf400ae170 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faf400a11b0 msgr2=0x7faf400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faf400a11b0 0x7faf400a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf44007be0 tx=0x7faf440073d0 comp rx=0 tx=0).stop 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 msgr2=0x7faf54197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 --2- 172.21.15.16:0/3387575273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf5406a310 0x7faf54197880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faf3c004730 tx=0x7faf3c009580 comp rx=0 tx=0).stop 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf537fe700 1 -- 172.21.15.16:0/3387575273 reap_dead start 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 shutdown_connections 2024-05-08T23:18:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 >> 172.21.15.16:0/3387575273 conn(0x7faf54076f40 msgr2=0x7faf540fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 shutdown_connections 2024-05-08T23:18:35.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.127+0000 7faf592ee700 1 -- 172.21.15.16:0/3387575273 wait complete. 2024-05-08T23:18:35.562 INFO:teuthology.orchestra.run.smithi016.stdout:34359738427 2024-05-08T23:18:35.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T23:18:35.776 INFO:teuthology.orchestra.run.smithi016.stdout:55834574903 2024-05-08T23:18:35.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T23:18:35.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:35 smithi016 ceph-mon[42541]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-08T23:18:36.012 INFO:teuthology.orchestra.run.smithi016.stdout:73014444084 2024-05-08T23:18:36.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T23:18:36.023 INFO:teuthology.orchestra.run.smithi016.stdout:257698037786 2024-05-08T23:18:36.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T23:18:36.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:35 smithi189 ceph-mon[46919]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-08T23:18:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:35 smithi155 ceph-mon[47105]: pgmap v247: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-08T23:18:36.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 -- 172.21.15.16:0/1709948295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834107030 msgr2=0x7f2834107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:36.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 --2- 172.21.15.16:0/1709948295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834107030 0x7f2834107410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2824009990 tx=0x7f2824009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 -- 172.21.15.16:0/1709948295 shutdown_connections 2024-05-08T23:18:36.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 --2- 172.21.15.16:0/1709948295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834101990 0x7f283410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 --2- 172.21.15.16:0/1709948295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834107030 0x7f2834107410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 --2- 172.21.15.16:0/1709948295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2834100ff0 0x7f2834101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 -- 172.21.15.16:0/1709948295 >> 172.21.15.16:0/1709948295 conn(0x7f2834078190 msgr2=0x7f28340785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 -- 172.21.15.16:0/1709948295 shutdown_connections 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.235+0000 7f283911a700 1 -- 172.21.15.16:0/1709948295 wait complete. 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 Processor -- start 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 -- start start 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834100ff0 0x7f28341995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2834101990 0x7f2834199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834107030 0x7f283419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283419e4b0 con 0x7f2834101990 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283419e620 con 0x7f2834100ff0 2024-05-08T23:18:36.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.236+0000 7f283911a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283419e7e0 con 0x7f2834107030 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2834101990 0x7f2834199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f2832d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834100ff0 0x7f28341995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f2832d9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834100ff0 0x7f28341995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39002/0 (socket says 172.21.15.16:39002) 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f2832d9d700 1 -- 172.21.15.16:0/3535825953 learned_addr learned my addr 172.21.15.16:0/3535825953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283359e700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834107030 0x7f283419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 -- 172.21.15.16:0/3535825953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834107030 msgr2=0x7f283419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2834107030 0x7f283419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 -- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834100ff0 msgr2=0x7f28341995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834100ff0 0x7f28341995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 -- 172.21.15.16:0/3535825953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28240095f0 con 0x7f2834101990 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f2832d9d700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2834100ff0 0x7f28341995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.237+0000 7f283259c700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2834101990 0x7f2834199af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f282800b450 tx=0x7f282800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.238+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828011830 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.238+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f282800be60 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.238+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828010ca0 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.238+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283419eac0 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.238+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28341a6320 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.239+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28341110a0 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.241+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2828011990 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f281bfff700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f281c0a11b0 0x7f281c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f28280cc4e0 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f2832d9d700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f281c0a11b0 0x7f281c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f281bfff700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] conn(0x7f281c0abd20 0x7f281c0ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 --> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f281c0ae830 con 0x7f281c0abd20 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28280ceac0 con 0x7f2834101990 2024-05-08T23:18:36.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f2832d9d700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f281c0a11b0 0x7f281c0a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2824006010 tx=0x7f282400baa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.242+0000 7f283359e700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] conn(0x7f281c0abd20 0x7f281c0ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.243+0000 7f283359e700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] conn(0x7f281c0abd20 0x7f281c0ae160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.244+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== osd.10 v2:172.21.15.155:6824/1164143288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f281c0ae830 con 0x7f281c0abd20 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.263+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 --> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28340616a0 con 0x7f281c0abd20 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f281bfff700 1 -- 172.21.15.16:0/3535825953 <== osd.10 v2:172.21.15.155:6824/1164143288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f28340616a0 con 0x7f281c0abd20 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] conn(0x7f281c0abd20 msgr2=0x7f281c0ae160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] conn(0x7f281c0abd20 0x7f281c0ae160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f281c0a11b0 msgr2=0x7f281c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f281c0a11b0 0x7f281c0a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2824006010 tx=0x7f282400baa0 comp rx=0 tx=0).stop 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2834101990 msgr2=0x7f2834199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 --2- 172.21.15.16:0/3535825953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2834101990 0x7f2834199af0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f282800b450 tx=0x7f282800b760 comp rx=0 tx=0).stop 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283359e700 1 -- 172.21.15.16:0/3535825953 reap_dead start 2024-05-08T23:18:36.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 shutdown_connections 2024-05-08T23:18:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 >> 172.21.15.16:0/3535825953 conn(0x7f2834078190 msgr2=0x7f283410c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 shutdown_connections 2024-05-08T23:18:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:35.264+0000 7f283911a700 1 -- 172.21.15.16:0/3535825953 wait complete. 2024-05-08T23:18:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/3372983691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc100a50 msgr2=0x7ffbdc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/3372983691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc100a50 0x7ffbdc100ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009990 tx=0x7ffbcc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/3372983691 shutdown_connections 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/3372983691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbdc101410 0x7ffbdc10cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/3372983691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbdc104ed0 0x7ffbdc1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/3372983691 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc100a50 0x7ffbdc100ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/3372983691 >> 172.21.15.16:0/3372983691 conn(0x7ffbdc075260 msgr2=0x7ffbdc075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:37.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/3372983691 shutdown_connections 2024-05-08T23:18:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.531+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/3372983691 wait complete. 2024-05-08T23:18:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.532+0000 7ffbe0ebc700 1 Processor -- start 2024-05-08T23:18:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.532+0000 7ffbe0ebc700 1 -- start start 2024-05-08T23:18:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbe0ebc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbdc100a50 0x7ffbdc194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbe0ebc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 0x7ffbdc195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbe0ebc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbdc104ed0 0x7ffbdc195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbe0ebc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc1a9b30 con 0x7ffbdc100a50 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbe0ebc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc1a9c70 con 0x7ffbdc101410 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbe0ebc700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc1a9de0 con 0x7ffbdc104ed0 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 0x7ffbdc195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbda59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbdc100a50 0x7ffbdc194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 0x7ffbdc195440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38712/0 (socket says 172.21.15.16:38712) 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 -- 172.21.15.16:0/2807641908 learned_addr learned my addr 172.21.15.16:0/2807641908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbdad9d700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbdc104ed0 0x7ffbdc195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:37.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 -- 172.21.15.16:0/2807641908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbdc104ed0 msgr2=0x7ffbdc195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbdc104ed0 0x7ffbdc195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 -- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbdc100a50 msgr2=0x7ffbdc194f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.533+0000 7ffbd9d9b700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbdc100a50 0x7ffbdc194f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbd9d9b700 1 -- 172.21.15.16:0/2807641908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc0095f0 con 0x7ffbdc101410 2024-05-08T23:18:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbd9d9b700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 0x7ffbdc195440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc006010 tx=0x7ffbcc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01d070 con 0x7ffbdc101410 2024-05-08T23:18:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffbcc022470 con 0x7ffbdc101410 2024-05-08T23:18:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc017650 con 0x7ffbdc101410 2024-05-08T23:18:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc1aa060 con 0x7ffbdc101410 2024-05-08T23:18:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.534+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc1aa420 con 0x7ffbdc101410 2024-05-08T23:18:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.535+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc1084c0 con 0x7ffbdc101410 2024-05-08T23:18:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.537+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ffbcc0225e0 con 0x7ffbdc101410 2024-05-08T23:18:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.538+0000 7ffbd37fe700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ffbc80a1190 0x7ffbc80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.538+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffbcc0cdfa0 con 0x7ffbdc101410 2024-05-08T23:18:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.538+0000 7ffbda59c700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ffbc80a1190 0x7ffbc80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.539+0000 7ffbda59c700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ffbc80a1190 0x7ffbc80a3650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009c60 tx=0x7ffbc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:37.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.542+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffbcc097410 con 0x7ffbdc101410 2024-05-08T23:18:37.565 INFO:teuthology.orchestra.run.smithi016.stdout:279172874264 2024-05-08T23:18:37.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T23:18:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.799+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ffbdc04ec90 con 0x7ffbdc101410 2024-05-08T23:18:37.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.800+0000 7ffbd37fe700 1 -- 172.21.15.16:0/2807641908 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ffbcc09aa30 con 0x7ffbdc101410 2024-05-08T23:18:37.801 INFO:teuthology.orchestra.run.smithi016.stdout:309237645333 2024-05-08T23:18:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.803+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ffbc80a1190 msgr2=0x7ffbc80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ffbc80a1190 0x7ffbc80a3650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009c60 tx=0x7ffbc4009260 comp rx=0 tx=0).stop 2024-05-08T23:18:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 msgr2=0x7ffbdc195440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 0x7ffbdc195440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc006010 tx=0x7ffbcc004730 comp rx=0 tx=0).stop 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 shutdown_connections 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ffbc80a1190 0x7ffbc80a3650 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbdc104ed0 0x7ffbdc195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbdc100a50 0x7ffbdc194f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 --2- 172.21.15.16:0/2807641908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbdc101410 0x7ffbdc195440 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 >> 172.21.15.16:0/2807641908 conn(0x7ffbdc075260 msgr2=0x7ffbdc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 shutdown_connections 2024-05-08T23:18:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.804+0000 7ffbe0ebc700 1 -- 172.21.15.16:0/2807641908 wait complete. 2024-05-08T23:18:37.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:37 smithi016 ceph-mon[42541]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-08T23:18:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 -- 172.21.15.16:0/3533043545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f41005e0 msgr2=0x7f76f4105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:38.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 --2- 172.21.15.16:0/3533043545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f41005e0 0x7f76f4105d20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f76e80099c0 tx=0x7f76e8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 -- 172.21.15.16:0/3533043545 shutdown_connections 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 --2- 172.21.15.16:0/3533043545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f4106450 0x7f76f410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 --2- 172.21.15.16:0/3533043545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f40ffc30 0x7f76f4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 --2- 172.21.15.16:0/3533043545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f41005e0 0x7f76f4105d20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 -- 172.21.15.16:0/3533043545 >> 172.21.15.16:0/3533043545 conn(0x7f76f4075220 msgr2=0x7f76f4075630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 -- 172.21.15.16:0/3533043545 shutdown_connections 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.994+0000 7f76f95c0700 1 -- 172.21.15.16:0/3533043545 wait complete. 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 Processor -- start 2024-05-08T23:18:38.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 -- start start 2024-05-08T23:18:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 0x7f76f4100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f41005e0 0x7f76f41010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f4106450 0x7f76f4101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41a56b0 con 0x7f76f40ffc30 2024-05-08T23:18:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41a5820 con 0x7f76f41005e0 2024-05-08T23:18:38.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f95c0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f41a59e0 con 0x7f76f4106450 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 0x7f76f4100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 0x7f76f4100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35874/0 (socket says 172.21.15.16:35874) 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f2ffd700 1 -- 172.21.15.16:0/1894062132 learned_addr learned my addr 172.21.15.16:0/1894062132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f37fe700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f4106450 0x7f76f4101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.995+0000 7f76f27fc700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f41005e0 0x7f76f41010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f2ffd700 1 -- 172.21.15.16:0/1894062132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f4106450 msgr2=0x7f76f4101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f2ffd700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f4106450 0x7f76f4101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f2ffd700 1 -- 172.21.15.16:0/1894062132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f41005e0 msgr2=0x7f76f41010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:38.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f2ffd700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f41005e0 0x7f76f41010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f2ffd700 1 -- 172.21.15.16:0/1894062132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80095f0 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f2ffd700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 0x7f76f4100bb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f76e400d950 tx=0x7f76e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400f890 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f41a5cc0 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76e400fe80 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.996+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400e3b0 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.997+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f41a6190 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.998+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f4109630 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.999+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f76e4010460 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:37.999+0000 7f76dbfff700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f76dc0a11b0 0x7f76dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.000+0000 7f76f27fc700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f76dc0a11b0 0x7f76dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.000+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f76e40cc430 con 0x7f76f40ffc30 2024-05-08T23:18:38.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.000+0000 7f76f27fc700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f76dc0a11b0 0x7f76dc0a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b580 tx=0x7f76e8005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:38.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.005+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f76e4095870 con 0x7f76f40ffc30 2024-05-08T23:18:38.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:37 smithi189 ceph-mon[46919]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-08T23:18:38.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:37 smithi155 ceph-mon[47105]: pgmap v248: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-08T23:18:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.225+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f76f4102090 con 0x7f76f40ffc30 2024-05-08T23:18:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.225+0000 7f76dbfff700 1 -- 172.21.15.16:0/1894062132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76e4098e90 con 0x7f76f40ffc30 2024-05-08T23:18:38.226 INFO:teuthology.orchestra.run.smithi016.stdout:197568495650 2024-05-08T23:18:38.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.227+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f76dc0a11b0 msgr2=0x7f76dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.227+0000 7f76f95c0700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f76dc0a11b0 0x7f76dc0a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b580 tx=0x7f76e8005bc0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.227+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 msgr2=0x7f76f4100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:38.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.227+0000 7f76f95c0700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 0x7f76f4100bb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f76e400d950 tx=0x7f76e400dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 shutdown_connections 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f76dc0a11b0 0x7f76dc0a3670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f4106450 0x7f76f4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76f40ffc30 0x7f76f4100bb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 --2- 172.21.15.16:0/1894062132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76f41005e0 0x7f76f41010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 >> 172.21.15.16:0/1894062132 conn(0x7f76f4075220 msgr2=0x7f76f4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 shutdown_connections 2024-05-08T23:18:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:38.228+0000 7f76f95c0700 1 -- 172.21.15.16:0/1894062132 wait complete. 2024-05-08T23:18:38.257 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:38.870 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:38.961 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645331 got 309237645333 for osd.11 2024-05-08T23:18:38.961 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:39.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:38 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2807641908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:18:39.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:38 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1894062132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:18:39.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:38 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2807641908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:18:39.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:38 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1894062132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:18:39.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:38 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2807641908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T23:18:39.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:38 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1894062132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 -- 172.21.15.16:0/2445616821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054106fe0 msgr2=0x7fe0541073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2445616821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054106fe0 0x7fe0541073c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe0500099c0 tx=0x7fe050009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 -- 172.21.15.16:0/2445616821 shutdown_connections 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2445616821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe054069bd0 0x7fe05410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2445616821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054106fe0 0x7fe0541073c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2445616821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054069210 0x7fe054069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 -- 172.21.15.16:0/2445616821 >> 172.21.15.16:0/2445616821 conn(0x7fe054076e70 msgr2=0x7fe054077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 -- 172.21.15.16:0/2445616821 shutdown_connections 2024-05-08T23:18:39.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.222+0000 7fe05adfc700 1 -- 172.21.15.16:0/2445616821 wait complete. 2024-05-08T23:18:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 Processor -- start 2024-05-08T23:18:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 -- start start 2024-05-08T23:18:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 0x7fe054199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054069bd0 0x7fe0541999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe054106fe0 0x7fe05419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419e380 con 0x7fe054069210 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419e4f0 con 0x7fe054106fe0 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe05adfc700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419e6b0 con 0x7fe054069bd0 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe058b98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 0x7fe054199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe058b98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 0x7fe054199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35890/0 (socket says 172.21.15.16:35890) 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe058b98700 1 -- 172.21.15.16:0/2783779046 learned_addr learned my addr 172.21.15.16:0/2783779046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe059399700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe054106fe0 0x7fe05419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.223+0000 7fe04bfff700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054069bd0 0x7fe0541999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe058b98700 1 -- 172.21.15.16:0/2783779046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054069bd0 msgr2=0x7fe0541999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe058b98700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054069bd0 0x7fe0541999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe058b98700 1 -- 172.21.15.16:0/2783779046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe054106fe0 msgr2=0x7fe05419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe058b98700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe054106fe0 0x7fe05419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe058b98700 1 -- 172.21.15.16:0/2783779046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0500095f0 con 0x7fe054069210 2024-05-08T23:18:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe058b98700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 0x7fe054199480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe05000f690 tx=0x7fe05000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05001c070 con 0x7fe054069210 2024-05-08T23:18:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05419e930 con 0x7fe054069210 2024-05-08T23:18:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe05000fe60 con 0x7fe054069210 2024-05-08T23:18:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050004970 con 0x7fe054069210 2024-05-08T23:18:39.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.224+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0541a6020 con 0x7fe054069210 2024-05-08T23:18:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.225+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054102690 con 0x7fe054069210 2024-05-08T23:18:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.226+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe050020070 con 0x7fe054069210 2024-05-08T23:18:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.227+0000 7fe049ffb700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe03c0a11b0 0x7fe03c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.227+0000 7fe04bfff700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe03c0a11b0 0x7fe03c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.227+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe0500cd770 con 0x7fe054069210 2024-05-08T23:18:39.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.228+0000 7fe04bfff700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe03c0a11b0 0x7fe03c0a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe05419a9e0 tx=0x7fe044009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:39.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.232+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe050096be0 con 0x7fe054069210 2024-05-08T23:18:39.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.450+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe054110d50 con 0x7fe054069210 2024-05-08T23:18:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.451+0000 7fe049ffb700 1 -- 172.21.15.16:0/2783779046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe050005740 con 0x7fe054069210 2024-05-08T23:18:39.452 INFO:teuthology.orchestra.run.smithi016.stdout:150323855401 2024-05-08T23:18:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.453+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe03c0a11b0 msgr2=0x7fe03c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.453+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe03c0a11b0 0x7fe03c0a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe05419a9e0 tx=0x7fe044009260 comp rx=0 tx=0).stop 2024-05-08T23:18:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.453+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 msgr2=0x7fe054199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.453+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 0x7fe054199480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe05000f690 tx=0x7fe05000f6c0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 shutdown_connections 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe054106fe0 0x7fe05419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe054069210 0x7fe054199480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe03c0a11b0 0x7fe03c0a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 --2- 172.21.15.16:0/2783779046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054069bd0 0x7fe0541999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 >> 172.21.15.16:0/2783779046 conn(0x7fe054076e70 msgr2=0x7fe05410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 shutdown_connections 2024-05-08T23:18:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:39.454+0000 7fe05adfc700 1 -- 172.21.15.16:0/2783779046 wait complete. 2024-05-08T23:18:39.511 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495648 got 197568495650 for osd.7 2024-05-08T23:18:39.511 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:39.881 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:40.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:39 smithi016 ceph-mon[42541]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T23:18:40.083 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:39 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2783779046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:18:40.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:39 smithi189 ceph-mon[46919]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T23:18:40.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:39 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2783779046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:18:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:39 smithi155 ceph-mon[47105]: pgmap v249: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-08T23:18:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:39 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2783779046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T23:18:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 -- 172.21.15.16:0/74261922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e580ffed0 msgr2=0x7f1e58100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/74261922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e580ffed0 0x7f1e58100350 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009990 tx=0x7f1e54009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 -- 172.21.15.16:0/74261922 shutdown_connections 2024-05-08T23:18:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/74261922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e58100890 0x7f1e5810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/74261922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e58105f00 0x7f1e581062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/74261922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e580ffed0 0x7f1e58100350 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 -- 172.21.15.16:0/74261922 >> 172.21.15.16:0/74261922 conn(0x7f1e58075f80 msgr2=0x7f1e58076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 -- 172.21.15.16:0/74261922 shutdown_connections 2024-05-08T23:18:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.253+0000 7f1e5fba3700 1 -- 172.21.15.16:0/74261922 wait complete. 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 Processor -- start 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 -- start start 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e580ffed0 0x7f1e58197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 0x7f1e58197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e58105f00 0x7f1e5819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819c1d0 con 0x7f1e58100890 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819c340 con 0x7f1e580ffed0 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5fba3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5819c500 con 0x7f1e58105f00 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5d13e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 0x7f1e58197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5d13e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 0x7f1e58197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35916/0 (socket says 172.21.15.16:35916) 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.254+0000 7f1e5d13e700 1 -- 172.21.15.16:0/220941508 learned_addr learned my addr 172.21.15.16:0/220941508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5e140700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e58105f00 0x7f1e5819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d93f700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e580ffed0 0x7f1e58197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d13e700 1 -- 172.21.15.16:0/220941508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e58105f00 msgr2=0x7f1e5819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d13e700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e58105f00 0x7f1e5819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d13e700 1 -- 172.21.15.16:0/220941508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e580ffed0 msgr2=0x7f1e58197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d13e700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e580ffed0 0x7f1e58197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d13e700 1 -- 172.21.15.16:0/220941508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540095f0 con 0x7f1e58100890 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5d13e700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 0x7f1e58197860 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e54004760 tx=0x7f1e54004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5401d070 con 0x7f1e58100890 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1e54005a00 con 0x7f1e58100890 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5819c780 con 0x7f1e58100890 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e54005280 con 0x7f1e58100890 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.255+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e581a3e70 con 0x7f1e58100890 2024-05-08T23:18:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.256+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e581015b0 con 0x7f1e58100890 2024-05-08T23:18:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.258+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1e54004930 con 0x7f1e58100890 2024-05-08T23:18:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.258+0000 7f1e4affd700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1e440a11b0 0x7f1e440a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.258+0000 7f1e5d93f700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1e440a11b0 0x7f1e440a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.258+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1e540ce130 con 0x7f1e58100890 2024-05-08T23:18:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.259+0000 7f1e5d93f700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1e440a11b0 0x7f1e440a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c005950 tx=0x7f1e4c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:40.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.262+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1e54097600 con 0x7f1e58100890 2024-05-08T23:18:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:40.414 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855399 got 150323855401 for osd.5 2024-05-08T23:18:40.414 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:40.480 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.521+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1e5804ec60 con 0x7f1e58100890 2024-05-08T23:18:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.521+0000 7f1e4affd700 1 -- 172.21.15.16:0/220941508 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e5409ac20 con 0x7f1e58100890 2024-05-08T23:18:40.522 INFO:teuthology.orchestra.run.smithi016.stdout:180388626471 2024-05-08T23:18:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1e440a11b0 msgr2=0x7f1e440a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1e440a11b0 0x7f1e440a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c005950 tx=0x7f1e4c00b410 comp rx=0 tx=0).stop 2024-05-08T23:18:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 msgr2=0x7f1e58197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 0x7f1e58197860 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e54004760 tx=0x7f1e54004790 comp rx=0 tx=0).stop 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 shutdown_connections 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1e440a11b0 0x7f1e440a3670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e58105f00 0x7f1e5819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e580ffed0 0x7f1e58197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 --2- 172.21.15.16:0/220941508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e58100890 0x7f1e58197860 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 >> 172.21.15.16:0/220941508 conn(0x7f1e58075f80 msgr2=0x7f1e58068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 shutdown_connections 2024-05-08T23:18:40.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.524+0000 7f1e5fba3700 1 -- 172.21.15.16:0/220941508 wait complete. 2024-05-08T23:18:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 -- 172.21.15.16:0/1885646879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 msgr2=0x7fc71c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 --2- 172.21.15.16:0/1885646879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c107400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0099c0 tx=0x7fc70c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 -- 172.21.15.16:0/1885646879 shutdown_connections 2024-05-08T23:18:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 --2- 172.21.15.16:0/1885646879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc71c069fa0 0x7fc71c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 --2- 172.21.15.16:0/1885646879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c107400 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 --2- 172.21.15.16:0/1885646879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc71c0695e0 0x7fc71c069a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 -- 172.21.15.16:0/1885646879 >> 172.21.15.16:0/1885646879 conn(0x7fc71c076c50 msgr2=0x7fc71c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 -- 172.21.15.16:0/1885646879 shutdown_connections 2024-05-08T23:18:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.960+0000 7fc721d8f700 1 -- 172.21.15.16:0/1885646879 wait complete. 2024-05-08T23:18:40.962 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:40 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/220941508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 Processor -- start 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 -- start start 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc71c0695e0 0x7fc71c1002e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc71c069fa0 0x7fc71c100820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1a5690 con 0x7fc71c107020 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1a5800 con 0x7fc71c0695e0 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc721d8f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1a59c0 con 0x7fc71c069fa0 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc71bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c100d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc71bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c100d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35922/0 (socket says 172.21.15.16:35922) 2024-05-08T23:18:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc71bfff700 1 -- 172.21.15.16:0/3859561805 learned_addr learned my addr 172.21.15.16:0/3859561805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.961+0000 7fc71affd700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc71c069fa0 0x7fc71c100820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71b7fe700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc71c0695e0 0x7fc71c1002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71bfff700 1 -- 172.21.15.16:0/3859561805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc71c069fa0 msgr2=0x7fc71c100820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71bfff700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc71c069fa0 0x7fc71c100820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71bfff700 1 -- 172.21.15.16:0/3859561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc71c0695e0 msgr2=0x7fc71c1002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71bfff700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc71c0695e0 0x7fc71c1002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71bfff700 1 -- 172.21.15.16:0/3859561805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0095f0 con 0x7fc71c107020 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc71bfff700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c100d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc71400ef30 tx=0x7fc71400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714018070 con 0x7fc71c107020 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc71400f040 con 0x7fc71c107020 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400cbc0 con 0x7fc71c107020 2024-05-08T23:18:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c1a5c40 con 0x7fc71c107020 2024-05-08T23:18:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.962+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c1a60f0 con 0x7fc71c107020 2024-05-08T23:18:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.963+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c109520 con 0x7fc71c107020 2024-05-08T23:18:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.965+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc71400cd80 con 0x7fc71c107020 2024-05-08T23:18:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.965+0000 7fc718ff9700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7040a12b0 0x7fc7040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.965+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc7140ccca0 con 0x7fc71c107020 2024-05-08T23:18:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.965+0000 7fc71b7fe700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7040a12b0 0x7fc7040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:40.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.966+0000 7fc71b7fe700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7040a12b0 0x7fc7040a3770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b580 tx=0x7fc70c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:40.969+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc714096110 con 0x7fc71c107020 2024-05-08T23:18:41.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:40 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/220941508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:18:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.200+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc71c1026b0 con 0x7fc71c107020 2024-05-08T23:18:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.201+0000 7fc718ff9700 1 -- 172.21.15.16:0/3859561805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc714099730 con 0x7fc71c107020 2024-05-08T23:18:41.202 INFO:teuthology.orchestra.run.smithi016.stdout:128849018925 2024-05-08T23:18:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.203+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7040a12b0 msgr2=0x7fc7040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.203+0000 7fc721d8f700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7040a12b0 0x7fc7040a3770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b580 tx=0x7fc70c019040 comp rx=0 tx=0).stop 2024-05-08T23:18:41.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.203+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 msgr2=0x7fc71c100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.203+0000 7fc721d8f700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c100d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc71400ef30 tx=0x7fc71400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 shutdown_connections 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c107020 0x7fc71c100d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc71c0695e0 0x7fc71c1002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fc7040a12b0 0x7fc7040a3770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 --2- 172.21.15.16:0/3859561805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc71c069fa0 0x7fc71c100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 >> 172.21.15.16:0/3859561805 conn(0x7fc71c076c50 msgr2=0x7fc71c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 shutdown_connections 2024-05-08T23:18:41.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:41.204+0000 7fc721d8f700 1 -- 172.21.15.16:0/3859561805 wait complete. 2024-05-08T23:18:41.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:40 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/220941508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T23:18:41.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626470 got 180388626471 for osd.6 2024-05-08T23:18:41.235 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:41.402 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:41.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:41 smithi016 ceph-mon[42541]: pgmap v250: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:18:41.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:41 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3859561805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:18:42.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:41 smithi189 ceph-mon[46919]: pgmap v250: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:18:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:41 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3859561805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:18:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 -- 172.21.15.16:0/3217612769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300692f0 msgr2=0x7f033010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 --2- 172.21.15.16:0/3217612769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300692f0 0x7f033010de60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f032800b0a0 tx=0x7f032800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 -- 172.21.15.16:0/3217612769 shutdown_connections 2024-05-08T23:18:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 --2- 172.21.15.16:0/3217612769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03300692f0 0x7f033010de60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 --2- 172.21.15.16:0/3217612769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330107020 0x7f0330107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 --2- 172.21.15.16:0/3217612769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0330068950 0x7f0330068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 -- 172.21.15.16:0/3217612769 >> 172.21.15.16:0/3217612769 conn(0x7f0330075f80 msgr2=0x7f0330076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 -- 172.21.15.16:0/3217612769 shutdown_connections 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.117+0000 7f0336fc0700 1 -- 172.21.15.16:0/3217612769 wait complete. 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 Processor -- start 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 -- start start 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 0x7f03301973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03300692f0 0x7f0330197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330107020 0x7f033019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033019c2d0 con 0x7f0330068950 2024-05-08T23:18:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033019c440 con 0x7f03300692f0 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0336fc0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033019c600 con 0x7f0330107020 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0334d5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 0x7f03301973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0334d5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 0x7f03301973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35944/0 (socket says 172.21.15.16:35944) 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.118+0000 7f0334d5c700 1 -- 172.21.15.16:0/1567364066 learned_addr learned my addr 172.21.15.16:0/1567364066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f032ffff700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03300692f0 0x7f0330197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f033555d700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330107020 0x7f033019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0334d5c700 1 -- 172.21.15.16:0/1567364066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330107020 msgr2=0x7f033019bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0334d5c700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330107020 0x7f033019bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0334d5c700 1 -- 172.21.15.16:0/1567364066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03300692f0 msgr2=0x7f0330197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0334d5c700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03300692f0 0x7f0330197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0334d5c700 1 -- 172.21.15.16:0/1567364066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328009d00 con 0x7f0330068950 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0334d5c700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 0x7f03301973d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f032400e9a0 tx=0x7f032400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324009790 con 0x7f0330068950 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0324010430 con 0x7f0330068950 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033019c8e0 con 0x7f0330068950 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.119+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400cea0 con 0x7f0330068950 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.120+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03301a4140 con 0x7f0330068950 2024-05-08T23:18:42.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.121+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0330109540 con 0x7f0330068950 2024-05-08T23:18:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.122+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f03240105a0 con 0x7f0330068950 2024-05-08T23:18:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.123+0000 7f032dffb700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f03200a1230 0x7f03200a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.123+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0324014070 con 0x7f0330068950 2024-05-08T23:18:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.123+0000 7f032ffff700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f03200a1230 0x7f03200a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.123+0000 7f032ffff700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f03200a1230 0x7f03200a36f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0330198930 tx=0x7f031c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.126+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0324096350 con 0x7f0330068950 2024-05-08T23:18:42.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018923 got 128849018925 for osd.4 2024-05-08T23:18:42.178 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:41 smithi155 ceph-mon[47105]: pgmap v250: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:18:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:41 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3859561805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T23:18:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.360+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f03301026d0 con 0x7f0330068950 2024-05-08T23:18:42.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.361+0000 7f032dffb700 1 -- 172.21.15.16:0/1567364066 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f03301026d0 con 0x7f0330068950 2024-05-08T23:18:42.362 INFO:teuthology.orchestra.run.smithi016.stdout:279172874265 2024-05-08T23:18:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f03200a1230 msgr2=0x7f03200a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f03200a1230 0x7f03200a36f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0330198930 tx=0x7f031c009260 comp rx=0 tx=0).stop 2024-05-08T23:18:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 msgr2=0x7f03301973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 0x7f03301973d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f032400e9a0 tx=0x7f032400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 shutdown_connections 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f03200a1230 0x7f03200a36f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0330107020 0x7f033019bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0330068950 0x7f03301973d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 --2- 172.21.15.16:0/1567364066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03300692f0 0x7f0330197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.363+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 >> 172.21.15.16:0/1567364066 conn(0x7f0330075f80 msgr2=0x7f03300fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.364+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 shutdown_connections 2024-05-08T23:18:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:42.364+0000 7f0336fc0700 1 -- 172.21.15.16:0/1567364066 wait complete. 2024-05-08T23:18:43.008 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1567364066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:18:43.008 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-08T23:18:43.008 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-08T23:18:43.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:42 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-08T23:18:43.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1567364066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:18:43.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-08T23:18:43.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-08T23:18:43.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-05-08T23:18:43.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-05-08T23:18:43.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-08T23:18:43.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-08T23:18:43.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-08T23:18:43.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-08T23:18:43.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:42 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-08T23:18:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1567364066' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T23:18:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-08T23:18:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-08T23:18:43.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]: dispatch 2024-05-08T23:18:43.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-05-08T23:18:43.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-08T23:18:43.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-08T23:18:43.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-08T23:18:43.219 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-05-08T23:18:43.219 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:42 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-08T23:18:43.718 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874264 got 279172874265 for osd.10 2024-05-08T23:18:43.719 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:43.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: pgmap v251: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:18:43.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-08T23:18:43.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:18:43.947 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:43 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: pgmap v251: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-08T23:18:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-08T23:18:44.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-08T23:18:44.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-08T23:18:44.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-08T23:18:44.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:18:44.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:43 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:18:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: pgmap v251: 129 pgs: 129 active+clean; 640 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:18:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-08T23:18:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-08T23:18:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T23:18:44.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:43 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:18:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 -- 172.21.15.16:0/2665913393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c41009d0 msgr2=0x7fd7c4100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 --2- 172.21.15.16:0/2665913393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c41009d0 0x7fd7c4100db0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40099c0 tx=0x7fd7b4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 -- 172.21.15.16:0/2665913393 shutdown_connections 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 --2- 172.21.15.16:0/2665913393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c410da80 0x7fd7c410fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 --2- 172.21.15.16:0/2665913393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c41009d0 0x7fd7c4100db0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 --2- 172.21.15.16:0/2665913393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 0x7fd7c410d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 -- 172.21.15.16:0/2665913393 >> 172.21.15.16:0/2665913393 conn(0x7fd7c4075f80 msgr2=0x7fd7c4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 -- 172.21.15.16:0/2665913393 shutdown_connections 2024-05-08T23:18:44.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.193+0000 7fd7caa33700 1 -- 172.21.15.16:0/2665913393 wait complete. 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 Processor -- start 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 -- start start 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c41009d0 0x7fd7c4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 0x7fd7c4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 0x7fd7c419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419e420 con 0x7fd7c410da80 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419e590 con 0x7fd7c41012f0 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7caa33700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419e750 con 0x7fd7c41009d0 2024-05-08T23:18:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7c8fd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 0x7fd7c419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7c37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 0x7fd7c4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7c8fd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 0x7fd7c419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35952/0 (socket says 172.21.15.16:35952) 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7c8fd0700 1 -- 172.21.15.16:0/3759403330 learned_addr learned my addr 172.21.15.16:0/3759403330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.194+0000 7fd7c3fff700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c41009d0 0x7fd7c4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c8fd0700 1 -- 172.21.15.16:0/3759403330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c41009d0 msgr2=0x7fd7c4199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c8fd0700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c41009d0 0x7fd7c4199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c8fd0700 1 -- 172.21.15.16:0/3759403330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 msgr2=0x7fd7c4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c8fd0700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 0x7fd7c4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c8fd0700 1 -- 172.21.15.16:0/3759403330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8009520 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c3fff700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c41009d0 0x7fd7c4199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c37fe700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 0x7fd7c4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c8fd0700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 0x7fd7c419de10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00ef30 tx=0x7fd7bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc018070 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7bc00f040 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00cbc0 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b40095f0 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.195+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c41a60c0 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.196+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c4108ca0 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.197+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd7bc007500 con 0x7fd7c410da80 2024-05-08T23:18:44.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.198+0000 7fd7c17fa700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7ac0a1230 0x7fd7ac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.199+0000 7fd7c3fff700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7ac0a1230 0x7fd7ac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.199+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd7bc0cc730 con 0x7fd7c410da80 2024-05-08T23:18:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.199+0000 7fd7c3fff700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7ac0a1230 0x7fd7ac0a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006010 tx=0x7fd7b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.204+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd7bc095af0 con 0x7fd7c410da80 2024-05-08T23:18:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 -- 172.21.15.16:0/3875096513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41019e0 msgr2=0x7f23c410ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/3875096513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41019e0 0x7f23c410ddf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00b0a0 tx=0x7f23bc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 -- 172.21.15.16:0/3875096513 shutdown_connections 2024-05-08T23:18:44.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/3875096513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41019e0 0x7f23c410ddf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/3875096513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c4107040 0x7f23c4107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/3875096513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4101020 0x7f23c41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 -- 172.21.15.16:0/3875096513 >> 172.21.15.16:0/3875096513 conn(0x7f23c4078100 msgr2=0x7f23c4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 -- 172.21.15.16:0/3875096513 shutdown_connections 2024-05-08T23:18:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.398+0000 7f23c8d9e700 1 -- 172.21.15.16:0/3875096513 wait complete. 2024-05-08T23:18:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 Processor -- start 2024-05-08T23:18:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 -- start start 2024-05-08T23:18:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 0x7f23c4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41019e0 0x7f23c4199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4107040 0x7f23c419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419e520 con 0x7f23c4101020 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419e690 con 0x7f23c41019e0 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c8d9e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c419e850 con 0x7f23c4107040 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 0x7f23c4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 0x7f23c4199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35980/0 (socket says 172.21.15.16:35980) 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c259c700 1 -- 172.21.15.16:0/1998354280 learned_addr learned my addr 172.21.15.16:0/1998354280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c2d9d700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4107040 0x7f23c419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.399+0000 7f23c1d9b700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41019e0 0x7f23c4199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c2d9d700 1 -- 172.21.15.16:0/1998354280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41019e0 msgr2=0x7f23c4199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c2d9d700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41019e0 0x7f23c4199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c2d9d700 1 -- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 msgr2=0x7f23c4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c2d9d700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 0x7f23c4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c2d9d700 1 -- 172.21.15.16:0/1998354280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc009d00 con 0x7f23c4107040 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c259c700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 0x7f23c4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c2d9d700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4107040 0x7f23c419df10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00b0a0 tx=0x7f23bc007d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00e040 con 0x7f23c4107040 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c419ead0 con 0x7f23c4107040 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23bc01b450 con 0x7f23c4107040 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0052a0 con 0x7f23c4107040 2024-05-08T23:18:44.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.400+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c41a6390 con 0x7f23c4107040 2024-05-08T23:18:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.401+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c4104310 con 0x7f23c4107040 2024-05-08T23:18:44.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.403+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f23bc01b5c0 con 0x7f23c4107040 2024-05-08T23:18:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.404+0000 7f23b37fe700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23ac0a11b0 0x7f23ac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.404+0000 7f23c259c700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23ac0a11b0 0x7f23ac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.404+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f23bc0ce2c0 con 0x7f23c4107040 2024-05-08T23:18:44.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.404+0000 7f23c259c700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23ac0a11b0 0x7f23ac0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f23b4011550 tx=0x7f23b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.407+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f23bc097740 con 0x7f23c4107040 2024-05-08T23:18:44.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.434+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd7c41004e0 con 0x7fd7c410da80 2024-05-08T23:18:44.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.435+0000 7fd7c17fa700 1 -- 172.21.15.16:0/3759403330 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd7bc099110 con 0x7fd7c410da80 2024-05-08T23:18:44.436 INFO:teuthology.orchestra.run.smithi016.stdout:98784247857 2024-05-08T23:18:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.437+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7ac0a1230 msgr2=0x7fd7ac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.437+0000 7fd7caa33700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7ac0a1230 0x7fd7ac0a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4006010 tx=0x7fd7b4000bc0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.437+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 msgr2=0x7fd7c419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.437+0000 7fd7caa33700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 0x7fd7c419de10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00ef30 tx=0x7fd7bc00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 shutdown_connections 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c410da80 0x7fd7c419de10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd7ac0a1230 0x7fd7ac0a36f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c41009d0 0x7fd7c4199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 --2- 172.21.15.16:0/3759403330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7c41012f0 0x7fd7c4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 >> 172.21.15.16:0/3759403330 conn(0x7fd7c4075f80 msgr2=0x7fd7c410f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 shutdown_connections 2024-05-08T23:18:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.438+0000 7fd7caa33700 1 -- 172.21.15.16:0/3759403330 wait complete. 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 -- 172.21.15.16:0/3288973595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1500107040 msgr2=0x7f1500107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 --2- 172.21.15.16:0/3288973595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1500107040 0x7f1500107420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0098f0 tx=0x7f14fc009c00 comp rx=0 tx=0).stop 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 -- 172.21.15.16:0/3288973595 shutdown_connections 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 --2- 172.21.15.16:0/3288973595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15001019e0 0x7f150010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 --2- 172.21.15.16:0/3288973595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1500107040 0x7f1500107420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 --2- 172.21.15.16:0/3288973595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1500101020 0x7f15001014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 -- 172.21.15.16:0/3288973595 >> 172.21.15.16:0/3288973595 conn(0x7f1500078100 msgr2=0x7f1500078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 -- 172.21.15.16:0/3288973595 shutdown_connections 2024-05-08T23:18:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.539+0000 7f15079b2700 1 -- 172.21.15.16:0/3288973595 wait complete. 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 Processor -- start 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 -- start start 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 0x7f15000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15001019e0 0x7f15000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1500107040 0x7f1500073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500075260 con 0x7f1500101020 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019b4a0 con 0x7f15001019e0 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.540+0000 7f15079b2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019b660 con 0x7f1500107040 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 0x7f15000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f1504f4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15001019e0 0x7f15000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 0x7f15000732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35998/0 (socket says 172.21.15.16:35998) 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 -- 172.21.15.16:0/3498837522 learned_addr learned my addr 172.21.15.16:0/3498837522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f1505f4f700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1500107040 0x7f1500073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 -- 172.21.15.16:0/3498837522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1500107040 msgr2=0x7f1500073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1500107040 0x7f1500073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 -- 172.21.15.16:0/3498837522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15001019e0 msgr2=0x7f15000737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15001019e0 0x7f15000737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 -- 172.21.15.16:0/3498837522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f8009d00 con 0x7f1500101020 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.541+0000 7f150574e700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 0x7f15000732a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00fa00 tx=0x7f14fc00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.542+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc01c070 con 0x7f1500101020 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.542+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14fc00baf0 con 0x7f1500101020 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.542+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc004930 con 0x7f1500101020 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.542+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc009520 con 0x7f1500101020 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.542+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150019bba0 con 0x7f1500101020 2024-05-08T23:18:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.543+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15001094d0 con 0x7f1500101020 2024-05-08T23:18:44.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.549+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f14fc020070 con 0x7f1500101020 2024-05-08T23:18:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.550+0000 7f14f67fc700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f14ec0a1300 0x7f14ec0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.550+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f14fc0cde90 con 0x7f1500101020 2024-05-08T23:18:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.550+0000 7f1504f4d700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f14ec0a1300 0x7f14ec0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.551+0000 7f1504f4d700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f14ec0a1300 0x7f14ec0a37c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15000749d0 tx=0x7f14f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.551+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f14fc0972d0 con 0x7f1500101020 2024-05-08T23:18:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.666+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f23c4102680 con 0x7f23c4107040 2024-05-08T23:18:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.667+0000 7f23b37fe700 1 -- 172.21.15.16:0/1998354280 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23bc020090 con 0x7f23c4107040 2024-05-08T23:18:44.668 INFO:teuthology.orchestra.run.smithi016.stdout:73014444085 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23ac0a11b0 msgr2=0x7f23ac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23ac0a11b0 0x7f23ac0a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f23b4011550 tx=0x7f23b4009260 comp rx=0 tx=0).stop 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4107040 msgr2=0x7f23c419df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4107040 0x7f23c419df10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00b0a0 tx=0x7f23bc007d90 comp rx=0 tx=0).stop 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 shutdown_connections 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23ac0a11b0 0x7f23ac0a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c4107040 0x7f23c419df10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c4101020 0x7f23c4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 --2- 172.21.15.16:0/1998354280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23c41019e0 0x7f23c4199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 >> 172.21.15.16:0/1998354280 conn(0x7f23c4078100 msgr2=0x7f23c410c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 shutdown_connections 2024-05-08T23:18:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.669+0000 7f23c8d9e700 1 -- 172.21.15.16:0/1998354280 wait complete. 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 -- 172.21.15.16:0/1858824213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 msgr2=0x7fde4c069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 --2- 172.21.15.16:0/1858824213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c069690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fde380099c0 tx=0x7fde38009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 -- 172.21.15.16:0/1858824213 shutdown_connections 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 --2- 172.21.15.16:0/1858824213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4c069bd0 0x7fde4c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 --2- 172.21.15.16:0/1858824213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c106fe0 0x7fde4c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 --2- 172.21.15.16:0/1858824213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c069690 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 -- 172.21.15.16:0/1858824213 >> 172.21.15.16:0/1858824213 conn(0x7fde4c076e70 msgr2=0x7fde4c077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 -- 172.21.15.16:0/1858824213 shutdown_connections 2024-05-08T23:18:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.692+0000 7fde52ddc700 1 -- 172.21.15.16:0/1858824213 wait complete. 2024-05-08T23:18:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 Processor -- start 2024-05-08T23:18:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 -- start start 2024-05-08T23:18:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c069bd0 0x7fde4c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4c106fe0 0x7fde4c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19c120 con 0x7fde4c069210 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19c290 con 0x7fde4c106fe0 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.693+0000 7fde52ddc700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19c450 con 0x7fde4c069bd0 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde50b78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde50b78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36008/0 (socket says 172.21.15.16:36008) 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde50b78700 1 -- 172.21.15.16:0/2364332482 learned_addr learned my addr 172.21.15.16:0/2364332482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c069bd0 0x7fde4c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde51379700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4c106fe0 0x7fde4c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 -- 172.21.15.16:0/2364332482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4c106fe0 msgr2=0x7fde4c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4c106fe0 0x7fde4c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 -- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 msgr2=0x7fde4c197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 -- 172.21.15.16:0/2364332482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40009520 con 0x7fde4c069bd0 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde50b78700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.694+0000 7fde4bfff700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c069bd0 0x7fde4c197760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde380099c0 tx=0x7fde38005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.695+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3801d070 con 0x7fde4c069bd0 2024-05-08T23:18:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.695+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde380095f0 con 0x7fde4c069bd0 2024-05-08T23:18:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.695+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde38005630 con 0x7fde4c069bd0 2024-05-08T23:18:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.695+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800f9b0 con 0x7fde4c069bd0 2024-05-08T23:18:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.695+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c19c950 con 0x7fde4c069bd0 2024-05-08T23:18:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.696+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c1094e0 con 0x7fde4c069bd0 2024-05-08T23:18:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.698+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fde38004ca0 con 0x7fde4c069bd0 2024-05-08T23:18:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.698+0000 7fde49ffb700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fde3c0a11b0 0x7fde3c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.699+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fde380cd870 con 0x7fde4c069bd0 2024-05-08T23:18:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.699+0000 7fde50b78700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fde3c0a11b0 0x7fde3c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.699+0000 7fde50b78700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fde3c0a11b0 0x7fde3c0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde40009fd0 tx=0x7fde40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.703+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fde38096bd0 con 0x7fde4c069bd0 2024-05-08T23:18:44.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.789+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f1500102680 con 0x7f1500101020 2024-05-08T23:18:44.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.789+0000 7f14f67fc700 1 -- 172.21.15.16:0/3498837522 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14fc027020 con 0x7f1500101020 2024-05-08T23:18:44.790 INFO:teuthology.orchestra.run.smithi016.stdout:257698037788 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f14ec0a1300 msgr2=0x7f14ec0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f14ec0a1300 0x7f14ec0a37c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f15000749d0 tx=0x7f14f0009190 comp rx=0 tx=0).stop 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 msgr2=0x7f15000732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 0x7f15000732a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00fa00 tx=0x7f14fc00bec0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 shutdown_connections 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f14ec0a1300 0x7f14ec0a37c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1500107040 0x7f1500073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500101020 0x7f15000732a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 --2- 172.21.15.16:0/3498837522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15001019e0 0x7f15000737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 >> 172.21.15.16:0/3498837522 conn(0x7f1500078100 msgr2=0x7f15000fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 shutdown_connections 2024-05-08T23:18:44.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.792+0000 7f15079b2700 1 -- 172.21.15.16:0/3498837522 wait complete. 2024-05-08T23:18:44.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.964+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fde4c04ec90 con 0x7fde4c069bd0 2024-05-08T23:18:44.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.964+0000 7fde49ffb700 1 -- 172.21.15.16:0/2364332482 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde3809a1f0 con 0x7fde4c069bd0 2024-05-08T23:18:44.965 INFO:teuthology.orchestra.run.smithi016.stdout:34359738428 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fde3c0a11b0 msgr2=0x7fde3c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fde3c0a11b0 0x7fde3c0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde40009fd0 tx=0x7fde40009260 comp rx=0 tx=0).stop 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c069bd0 msgr2=0x7fde4c197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c069bd0 0x7fde4c197760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde380099c0 tx=0x7fde38005270 comp rx=0 tx=0).stop 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 shutdown_connections 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde4c106fe0 0x7fde4c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c069210 0x7fde4c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fde3c0a11b0 0x7fde3c0a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 --2- 172.21.15.16:0/2364332482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c069bd0 0x7fde4c197760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:44.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 >> 172.21.15.16:0/2364332482 conn(0x7fde4c076e70 msgr2=0x7fde4c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:44.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 shutdown_connections 2024-05-08T23:18:44.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:44.967+0000 7fde52ddc700 1 -- 172.21.15.16:0/2364332482 wait complete. 2024-05-08T23:18:45.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:18:45.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3759403330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:18:45.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1998354280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:18:45.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3498837522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:18:45.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3759403330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1998354280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:18:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3498837522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:18:45.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.195+0000 7f52d4910700 1 -- 172.21.15.16:0/1243921271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0069210 msgr2=0x7f52d0069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.195+0000 7f52d4910700 1 --2- 172.21.15.16:0/1243921271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0069210 0x7f52d0069690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f52c00099c0 tx=0x7f52c0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 -- 172.21.15.16:0/1243921271 shutdown_connections 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 --2- 172.21.15.16:0/1243921271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069bd0 0x7f52d010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 --2- 172.21.15.16:0/1243921271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d0106fe0 0x7f52d01073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 --2- 172.21.15.16:0/1243921271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0069210 0x7f52d0069690 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 -- 172.21.15.16:0/1243921271 >> 172.21.15.16:0/1243921271 conn(0x7f52d0076e70 msgr2=0x7f52d0077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 -- 172.21.15.16:0/1243921271 shutdown_connections 2024-05-08T23:18:45.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.196+0000 7f52d4910700 1 -- 172.21.15.16:0/1243921271 wait complete. 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 Processor -- start 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 -- start start 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069210 0x7f52d0199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d0069bd0 0x7f52d0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 0x7f52d019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d019e310 con 0x7f52d0106fe0 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d019e480 con 0x7f52d0069bd0 2024-05-08T23:18:45.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52d4910700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d019e640 con 0x7f52d0069210 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52ced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 0x7f52d019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52ced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 0x7f52d019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36014/0 (socket says 172.21.15.16:36014) 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52ced9d700 1 -- 172.21.15.16:0/3060444259 learned_addr learned my addr 172.21.15.16:0/3060444259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52ce59c700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069210 0x7f52d0199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52cdd9b700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d0069bd0 0x7f52d0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52ced9d700 1 -- 172.21.15.16:0/3060444259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069210 msgr2=0x7f52d0199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.197+0000 7f52ced9d700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069210 0x7f52d0199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52ced9d700 1 -- 172.21.15.16:0/3060444259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d0069bd0 msgr2=0x7f52d0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52ced9d700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d0069bd0 0x7f52d0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52ced9d700 1 -- 172.21.15.16:0/3060444259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00095f0 con 0x7f52d0106fe0 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52ce59c700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069210 0x7f52d0199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52ced9d700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 0x7f52d019dd00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ef30 tx=0x7f52c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8018070 con 0x7f52d0106fe0 2024-05-08T23:18:45.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52c800f040 con 0x7f52d0106fe0 2024-05-08T23:18:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d019e8c0 con 0x7f52d0106fe0 2024-05-08T23:18:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cbc0 con 0x7f52d0106fe0 2024-05-08T23:18:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.198+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d019eb70 con 0x7f52d0106fe0 2024-05-08T23:18:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.199+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d0102690 con 0x7f52d0106fe0 2024-05-08T23:18:45.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.200+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f52c8007500 con 0x7f52d0106fe0 2024-05-08T23:18:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.201+0000 7f52c77fe700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f52bc0a11b0 0x7f52bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.201+0000 7f52ce59c700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f52bc0a11b0 0x7f52bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.201+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f52c80cc6e0 con 0x7f52d0106fe0 2024-05-08T23:18:45.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.201+0000 7f52ce59c700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f52bc0a11b0 0x7f52bc0a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005950 tx=0x7f52b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:45.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.205+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f52c8095aa0 con 0x7f52d0106fe0 2024-05-08T23:18:45.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T23:18:45.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3759403330' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T23:18:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1998354280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T23:18:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3498837522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T23:18:45.332 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444084 got 73014444085 for osd.2 2024-05-08T23:18:45.333 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:45.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.424+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f52d019eb00 con 0x7f52d0106fe0 2024-05-08T23:18:45.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.424+0000 7f52c77fe700 1 -- 172.21.15.16:0/3060444259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52c80990c0 con 0x7f52d0106fe0 2024-05-08T23:18:45.425 INFO:teuthology.orchestra.run.smithi016.stdout:227633266721 2024-05-08T23:18:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f52bc0a11b0 msgr2=0x7f52bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f52bc0a11b0 0x7f52bc0a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005950 tx=0x7f52b800b410 comp rx=0 tx=0).stop 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 msgr2=0x7f52d019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 0x7f52d019dd00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ef30 tx=0x7f52c800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 shutdown_connections 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d0106fe0 0x7f52d019dd00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f52bc0a11b0 0x7f52bc0a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52d0069210 0x7f52d0199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 --2- 172.21.15.16:0/3060444259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52d0069bd0 0x7f52d0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 >> 172.21.15.16:0/3060444259 conn(0x7f52d0076e70 msgr2=0x7f52d010c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 shutdown_connections 2024-05-08T23:18:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:45.427+0000 7f52d4910700 1 -- 172.21.15.16:0/3060444259 wait complete. 2024-05-08T23:18:45.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247855 got 98784247857 for osd.3 2024-05-08T23:18:45.452 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:45.951 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266719 got 227633266721 for osd.8 2024-05-08T23:18:45.951 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:46.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:45 smithi016 ceph-mon[42541]: pgmap v254: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T23:18:46.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:45 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2364332482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:18:46.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:45 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3060444259' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:18:46.118 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738427 got 34359738428 for osd.0 2024-05-08T23:18:46.118 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:46.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:45 smithi189 ceph-mon[46919]: pgmap v254: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T23:18:46.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:45 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2364332482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:18:46.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:45 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3060444259' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:18:46.170 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037786 got 257698037788 for osd.9 2024-05-08T23:18:46.170 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:45 smithi155 ceph-mon[47105]: pgmap v254: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-08T23:18:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:45 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2364332482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T23:18:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:45 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3060444259' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 -- 172.21.15.16:0/1729273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c941001c0 msgr2=0x7f0c94100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 --2- 172.21.15.16:0/1729273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c941001c0 0x7f0c94100640 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0c840099c0 tx=0x7f0c84009cd0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 -- 172.21.15.16:0/1729273 shutdown_connections 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 --2- 172.21.15.16:0/1729273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c94100b80 0x7f0c9410cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 --2- 172.21.15.16:0/1729273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c94106040 0x7f0c94106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 --2- 172.21.15.16:0/1729273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c941001c0 0x7f0c94100640 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 -- 172.21.15.16:0/1729273 >> 172.21.15.16:0/1729273 conn(0x7f0c94076c40 msgr2=0x7f0c94077050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 -- 172.21.15.16:0/1729273 shutdown_connections 2024-05-08T23:18:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.455+0000 7f0c9b788700 1 -- 172.21.15.16:0/1729273 wait complete. 2024-05-08T23:18:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 Processor -- start 2024-05-08T23:18:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 -- start start 2024-05-08T23:18:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c941001c0 0x7f0c94199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 0x7f0c94199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c94106040 0x7f0c9419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419e3d0 con 0x7f0c94100b80 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419e540 con 0x7f0c94106040 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c9b788700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9419e700 con 0x7f0c941001c0 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c98d23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 0x7f0c94199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c98d23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 0x7f0c94199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50290/0 (socket says 172.21.15.16:50290) 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c98d23700 1 -- 172.21.15.16:0/4160900886 learned_addr learned my addr 172.21.15.16:0/4160900886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c99524700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c941001c0 0x7f0c94199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.456+0000 7f0c99d25700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c94106040 0x7f0c9419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c98d23700 1 -- 172.21.15.16:0/4160900886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c941001c0 msgr2=0x7f0c94199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c98d23700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c941001c0 0x7f0c94199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c98d23700 1 -- 172.21.15.16:0/4160900886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c94106040 msgr2=0x7f0c9419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c98d23700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c94106040 0x7f0c9419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c98d23700 1 -- 172.21.15.16:0/4160900886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c840095f0 con 0x7f0c94100b80 2024-05-08T23:18:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c98d23700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 0x7f0c94199a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400b580 tx=0x7f0c8400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8401c070 con 0x7f0c94100b80 2024-05-08T23:18:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c8400fd50 con 0x7f0c94100b80 2024-05-08T23:18:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9419e980 con 0x7f0c94100b80 2024-05-08T23:18:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84004970 con 0x7f0c94100b80 2024-05-08T23:18:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.457+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c941a6070 con 0x7f0c94100b80 2024-05-08T23:18:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.458+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c94104260 con 0x7f0c94100b80 2024-05-08T23:18:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.459+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0c84020070 con 0x7f0c94100b80 2024-05-08T23:18:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.460+0000 7f0c8a7fc700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c800a11b0 0x7f0c800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.460+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0c840cd880 con 0x7f0c94100b80 2024-05-08T23:18:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.460+0000 7f0c99524700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c800a11b0 0x7f0c800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.460+0000 7f0c99524700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c800a11b0 0x7f0c800a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000abd0 tx=0x7f0c9000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.465+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0c84096be0 con 0x7f0c94100b80 2024-05-08T23:18:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.682+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0c9419a7e0 con 0x7f0c94100b80 2024-05-08T23:18:46.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.682+0000 7f0c8a7fc700 1 -- 172.21.15.16:0/4160900886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c8409a200 con 0x7f0c94100b80 2024-05-08T23:18:46.684 INFO:teuthology.orchestra.run.smithi016.stdout:55834574905 2024-05-08T23:18:46.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c800a11b0 msgr2=0x7f0c800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c800a11b0 0x7f0c800a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000abd0 tx=0x7f0c9000a380 comp rx=0 tx=0).stop 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 msgr2=0x7f0c94199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 0x7f0c94199a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400b580 tx=0x7f0c8400f690 comp rx=0 tx=0).stop 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 shutdown_connections 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c94106040 0x7f0c9419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0c800a11b0 0x7f0c800a3670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c941001c0 0x7f0c94199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 --2- 172.21.15.16:0/4160900886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94100b80 0x7f0c94199a60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 >> 172.21.15.16:0/4160900886 conn(0x7f0c94076c40 msgr2=0x7f0c940698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 shutdown_connections 2024-05-08T23:18:46.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:46.685+0000 7f0c9b788700 1 -- 172.21.15.16:0/4160900886 wait complete. 2024-05-08T23:18:47.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4160900886' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:18:47.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574903 got 55834574905 for osd.1 2024-05-08T23:18:47.087 DEBUG:teuthology.parallel:result is None 2024-05-08T23:18:47.087 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T23:18:47.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:18:47.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4160900886' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:18:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4160900886' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T23:18:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:47 smithi016 ceph-mon[42541]: pgmap v255: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:47 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:47 smithi189 ceph-mon[46919]: pgmap v255: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:47 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:47 smithi155 ceph-mon[47105]: pgmap v255: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:47 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 -- 172.21.15.16:0/424901207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d980692f0 msgr2=0x7f6d9810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/424901207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d980692f0 0x7f6d9810de60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b0a0 tx=0x7f6d9000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 -- 172.21.15.16:0/424901207 shutdown_connections 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/424901207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d980692f0 0x7f6d9810de60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/424901207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d98107020 0x7f6d98107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/424901207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d98068950 0x7f6d98068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 -- 172.21.15.16:0/424901207 >> 172.21.15.16:0/424901207 conn(0x7f6d98075f80 msgr2=0x7f6d98076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 -- 172.21.15.16:0/424901207 shutdown_connections 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.403+0000 7f6d9d23b700 1 -- 172.21.15.16:0/424901207 wait complete. 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 Processor -- start 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 -- start start 2024-05-08T23:18:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 0x7f6d980fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d980692f0 0x7f6d981004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d98107020 0x7f6d98100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981a5770 con 0x7f6d98068950 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981a58e0 con 0x7f6d980692f0 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.404+0000 7f6d9d23b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d981a5aa0 con 0x7f6d98107020 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 0x7f6d980fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 0x7f6d980fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50306/0 (socket says 172.21.15.16:50306) 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 -- 172.21.15.16:0/1322518378 learned_addr learned my addr 172.21.15.16:0/1322518378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d9759e700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d98107020 0x7f6d98100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d8ffff700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d980692f0 0x7f6d981004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 -- 172.21.15.16:0/1322518378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d98107020 msgr2=0x7f6d98100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d98107020 0x7f6d98100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 -- 172.21.15.16:0/1322518378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d980692f0 msgr2=0x7f6d981004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d980692f0 0x7f6d981004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 -- 172.21.15.16:0/1322518378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90009d00 con 0x7f6d98068950 2024-05-08T23:18:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d96d9d700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 0x7f6d980fffa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e9a0 tx=0x7f6d8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88009790 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d8800cbd0 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d981a5d80 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88018680 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.405+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981a6220 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.407+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d98109540 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.408+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6d8800cd40 con 0x7f6d98068950 2024-05-08T23:18:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.408+0000 7f6d94d99700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d840a11b0 0x7f6d840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.409+0000 7f6d8ffff700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d840a11b0 0x7f6d840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.409+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6d88014070 con 0x7f6d98068950 2024-05-08T23:18:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.409+0000 7f6d8ffff700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d840a11b0 0x7f6d840a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d981016d0 tx=0x7f6d80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.412+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d88096650 con 0x7f6d98068950 2024-05-08T23:18:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.624+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d98110d80 con 0x7f6d840a11b0 2024-05-08T23:18:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.638+0000 7f6d94d99700 1 -- 172.21.15.16:0/1322518378 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423260 (secure 0 0 0) 0x7f6d98110d80 con 0x7f6d840a11b0 2024-05-08T23:18:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d840a11b0 msgr2=0x7f6d840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d840a11b0 0x7f6d840a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6d981016d0 tx=0x7f6d80009260 comp rx=0 tx=0).stop 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 msgr2=0x7f6d980fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 0x7f6d980fffa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800e9a0 tx=0x7f6d8800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 shutdown_connections 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f6d840a11b0 0x7f6d840a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d98107020 0x7f6d98100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98068950 0x7f6d980fffa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 --2- 172.21.15.16:0/1322518378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d980692f0 0x7f6d981004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 >> 172.21.15.16:0/1322518378 conn(0x7f6d98075f80 msgr2=0x7f6d980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 shutdown_connections 2024-05-08T23:18:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:48.642+0000 7f6d9d23b700 1 -- 172.21.15.16:0/1322518378 wait complete. 2024-05-08T23:18:48.649 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:18:48.978 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:18:48.978 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:18:48.979 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:48 smithi016 ceph-mon[42541]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:49.045 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T23:18:48.593596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":365,"num_read_kb":304,"num_write":339,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12924,"kb_used_omap":18,"kb_used_meta":381421,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":13234176,"data_stored":7442643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":390575432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002200"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802073+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:18:42.802073+0000","last_peered":"2024-05-08T23:18:42.802073+0000","last_clean":"2024-05-08T23:18:42.802073+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:18:42.802073+0000","last_undegraded":"2024-05-08T23:18:42.802073+0000","last_fullsized":"2024-05-08T23:18:42.802073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850382+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:18:42.850382+0000","last_peered":"2024-05-08T23:18:42.850382+0000","last_clean":"2024-05-08T23:18:42.850382+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:18:42.850382+0000","last_undegraded":"2024-05-08T23:18:42.850382+0000","last_fullsized":"2024-05-08T23:18:42.850382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850219+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:18:42.850219+0000","last_peered":"2024-05-08T23:18:42.850219+0000","last_clean":"2024-05-08T23:18:42.850219+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:18:42.850219+0000","last_undegraded":"2024-05-08T23:18:42.850219+0000","last_fullsized":"2024-05-08T23:18:42.850219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793298+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:18:42.793298+0000","last_peered":"2024-05-08T23:18:42.793298+0000","last_clean":"2024-05-08T23:18:42.793298+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:18:42.793298+0000","last_undegraded":"2024-05-08T23:18:42.793298+0000","last_fullsized":"2024-05-08T23:18:42.793298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799491+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:18:42.799491+0000","last_peered":"2024-05-08T23:18:42.799491+0000","last_clean":"2024-05-08T23:18:42.799491+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:18:42.799491+0000","last_undegraded":"2024-05-08T23:18:42.799491+0000","last_fullsized":"2024-05-08T23:18:42.799491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796401+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:18:42.796401+0000","last_peered":"2024-05-08T23:18:42.796401+0000","last_clean":"2024-05-08T23:18:42.796401+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:18:42.796401+0000","last_undegraded":"2024-05-08T23:18:42.796401+0000","last_fullsized":"2024-05-08T23:18:42.796401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680125+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:17:50.680125+0000","last_peered":"2024-05-08T23:17:50.680125+0000","last_clean":"2024-05-08T23:17:50.680125+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:17:50.680125+0000","last_undegraded":"2024-05-08T23:17:50.680125+0000","last_fullsized":"2024-05-08T23:17:50.680125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795761+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:18:42.795761+0000","last_peered":"2024-05-08T23:18:42.795761+0000","last_clean":"2024-05-08T23:18:42.795761+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:18:42.795761+0000","last_undegraded":"2024-05-08T23:18:42.795761+0000","last_fullsized":"2024-05-08T23:18:42.795761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800861+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:18:42.800861+0000","last_peered":"2024-05-08T23:18:42.800861+0000","last_clean":"2024-05-08T23:18:42.800861+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:18:42.800861+0000","last_undegraded":"2024-05-08T23:18:42.800861+0000","last_fullsized":"2024-05-08T23:18:42.800861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034666+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:18:44.034666+0000","last_peered":"2024-05-08T23:18:44.034666+0000","last_clean":"2024-05-08T23:18:44.034666+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:18:44.034666+0000","last_undegraded":"2024-05-08T23:18:44.034666+0000","last_fullsized":"2024-05-08T23:18:44.034666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680536+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:17:50.680536+0000","last_peered":"2024-05-08T23:17:50.680536+0000","last_clean":"2024-05-08T23:17:50.680536+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:17:50.680536+0000","last_undegraded":"2024-05-08T23:17:50.680536+0000","last_fullsized":"2024-05-08T23:17:50.680536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793543+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:18:42.793543+0000","last_peered":"2024-05-08T23:18:42.793543+0000","last_clean":"2024-05-08T23:18:42.793543+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:18:42.793543+0000","last_undegraded":"2024-05-08T23:18:42.793543+0000","last_fullsized":"2024-05-08T23:18:42.793543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800158+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:18:42.800158+0000","last_peered":"2024-05-08T23:18:42.800158+0000","last_clean":"2024-05-08T23:18:42.800158+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:18:42.800158+0000","last_undegraded":"2024-05-08T23:18:42.800158+0000","last_fullsized":"2024-05-08T23:18:42.800158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849931+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:18:42.849931+0000","last_peered":"2024-05-08T23:18:42.849931+0000","last_clean":"2024-05-08T23:18:42.849931+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:18:42.849931+0000","last_undegraded":"2024-05-08T23:18:42.849931+0000","last_fullsized":"2024-05-08T23:18:42.849931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849831+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:18:42.849831+0000","last_peered":"2024-05-08T23:18:42.849831+0000","last_clean":"2024-05-08T23:18:42.849831+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:18:42.849831+0000","last_undegraded":"2024-05-08T23:18:42.849831+0000","last_fullsized":"2024-05-08T23:18:42.849831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799862+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:18:42.799862+0000","last_peered":"2024-05-08T23:18:42.799862+0000","last_clean":"2024-05-08T23:18:42.799862+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.799862+0000","last_undegraded":"2024-05-08T23:18:42.799862+0000","last_fullsized":"2024-05-08T23:18:42.799862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801991+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:18:42.801991+0000","last_peered":"2024-05-08T23:18:42.801991+0000","last_clean":"2024-05-08T23:18:42.801991+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:18:42.801991+0000","last_undegraded":"2024-05-08T23:18:42.801991+0000","last_fullsized":"2024-05-08T23:18:42.801991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793563+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:18:42.793563+0000","last_peered":"2024-05-08T23:18:42.793563+0000","last_clean":"2024-05-08T23:18:42.793563+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:18:42.793563+0000","last_undegraded":"2024-05-08T23:18:42.793563+0000","last_fullsized":"2024-05-08T23:18:42.793563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799013+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:18:42.799013+0000","last_peered":"2024-05-08T23:18:42.799013+0000","last_clean":"2024-05-08T23:18:42.799013+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:18:42.799013+0000","last_undegraded":"2024-05-08T23:18:42.799013+0000","last_fullsized":"2024-05-08T23:18:42.799013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800896+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:42.800896+0000","last_peered":"2024-05-08T23:18:42.800896+0000","last_clean":"2024-05-08T23:18:42.800896+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:42.800896+0000","last_undegraded":"2024-05-08T23:18:42.800896+0000","last_fullsized":"2024-05-08T23:18:42.800896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793673+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:42.793673+0000","last_peered":"2024-05-08T23:18:42.793673+0000","last_clean":"2024-05-08T23:18:42.793673+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:42.793673+0000","last_undegraded":"2024-05-08T23:18:42.793673+0000","last_fullsized":"2024-05-08T23:18:42.793673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797273+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:18:42.797273+0000","last_peered":"2024-05-08T23:18:42.797273+0000","last_clean":"2024-05-08T23:18:42.797273+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:18:42.797273+0000","last_undegraded":"2024-05-08T23:18:42.797273+0000","last_fullsized":"2024-05-08T23:18:42.797273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792847+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:42.792847+0000","last_peered":"2024-05-08T23:18:42.792847+0000","last_clean":"2024-05-08T23:18:42.792847+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:42.792847+0000","last_undegraded":"2024-05-08T23:18:42.792847+0000","last_fullsized":"2024-05-08T23:18:42.792847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850495+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:18:42.850495+0000","last_peered":"2024-05-08T23:18:42.850495+0000","last_clean":"2024-05-08T23:18:42.850495+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:18:42.850495+0000","last_undegraded":"2024-05-08T23:18:42.850495+0000","last_fullsized":"2024-05-08T23:18:42.850495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803307+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803307+0000","last_peered":"2024-05-08T23:18:43.803307+0000","last_clean":"2024-05-08T23:18:43.803307+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803307+0000","last_undegraded":"2024-05-08T23:18:43.803307+0000","last_fullsized":"2024-05-08T23:18:43.803307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.813513+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:43.813513+0000","last_peered":"2024-05-08T23:18:43.813513+0000","last_clean":"2024-05-08T23:18:43.813513+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:43.813513+0000","last_undegraded":"2024-05-08T23:18:43.813513+0000","last_fullsized":"2024-05-08T23:18:43.813513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800094+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:18:42.800094+0000","last_peered":"2024-05-08T23:18:42.800094+0000","last_clean":"2024-05-08T23:18:42.800094+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:18:42.800094+0000","last_undegraded":"2024-05-08T23:18:42.800094+0000","last_fullsized":"2024-05-08T23:18:42.800094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792881+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:42.792881+0000","last_peered":"2024-05-08T23:18:42.792881+0000","last_clean":"2024-05-08T23:18:42.792881+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:42.792881+0000","last_undegraded":"2024-05-08T23:18:42.792881+0000","last_fullsized":"2024-05-08T23:18:42.792881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797321+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:42.797321+0000","last_peered":"2024-05-08T23:18:42.797321+0000","last_clean":"2024-05-08T23:18:42.797321+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:42.797321+0000","last_undegraded":"2024-05-08T23:18:42.797321+0000","last_fullsized":"2024-05-08T23:18:42.797321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680608+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:17:50.680608+0000","last_peered":"2024-05-08T23:17:50.680608+0000","last_clean":"2024-05-08T23:17:50.680608+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:17:50.680608+0000","last_undegraded":"2024-05-08T23:17:50.680608+0000","last_fullsized":"2024-05-08T23:17:50.680608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798260+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:42.798260+0000","last_peered":"2024-05-08T23:18:42.798260+0000","last_clean":"2024-05-08T23:18:42.798260+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:42.798260+0000","last_undegraded":"2024-05-08T23:18:42.798260+0000","last_fullsized":"2024-05-08T23:18:42.798260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.747822+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:42.747822+0000","last_peered":"2024-05-08T23:18:42.747822+0000","last_clean":"2024-05-08T23:18:42.747822+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:42.747822+0000","last_undegraded":"2024-05-08T23:18:42.747822+0000","last_fullsized":"2024-05-08T23:18:42.747822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796860+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:18:42.796860+0000","last_peered":"2024-05-08T23:18:42.796860+0000","last_clean":"2024-05-08T23:18:42.796860+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:18:42.796860+0000","last_undegraded":"2024-05-08T23:18:42.796860+0000","last_fullsized":"2024-05-08T23:18:42.796860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803558+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803558+0000","last_peered":"2024-05-08T23:18:43.803558+0000","last_clean":"2024-05-08T23:18:43.803558+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803558+0000","last_undegraded":"2024-05-08T23:18:43.803558+0000","last_fullsized":"2024-05-08T23:18:43.803558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796303+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:42.796303+0000","last_peered":"2024-05-08T23:18:42.796303+0000","last_clean":"2024-05-08T23:18:42.796303+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:42.796303+0000","last_undegraded":"2024-05-08T23:18:42.796303+0000","last_fullsized":"2024-05-08T23:18:42.796303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680486+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:17:50.680486+0000","last_peered":"2024-05-08T23:17:50.680486+0000","last_clean":"2024-05-08T23:17:50.680486+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:17:50.680486+0000","last_undegraded":"2024-05-08T23:17:50.680486+0000","last_fullsized":"2024-05-08T23:17:50.680486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801918+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:42.801918+0000","last_peered":"2024-05-08T23:18:42.801918+0000","last_clean":"2024-05-08T23:18:42.801918+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:42.801918+0000","last_undegraded":"2024-05-08T23:18:42.801918+0000","last_fullsized":"2024-05-08T23:18:42.801918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799336+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:42.799336+0000","last_peered":"2024-05-08T23:18:42.799336+0000","last_clean":"2024-05-08T23:18:42.799336+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:42.799336+0000","last_undegraded":"2024-05-08T23:18:42.799336+0000","last_fullsized":"2024-05-08T23:18:42.799336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795875+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:18:42.795875+0000","last_peered":"2024-05-08T23:18:42.795875+0000","last_clean":"2024-05-08T23:18:42.795875+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:18:42.795875+0000","last_undegraded":"2024-05-08T23:18:42.795875+0000","last_fullsized":"2024-05-08T23:18:42.795875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850309+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:42.850309+0000","last_peered":"2024-05-08T23:18:42.850309+0000","last_clean":"2024-05-08T23:18:42.850309+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:42.850309+0000","last_undegraded":"2024-05-08T23:18:42.850309+0000","last_fullsized":"2024-05-08T23:18:42.850309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.688533+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:10.688533+0000","last_peered":"2024-05-08T23:18:10.688533+0000","last_clean":"2024-05-08T23:18:10.688533+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:10.688533+0000","last_undegraded":"2024-05-08T23:18:10.688533+0000","last_fullsized":"2024-05-08T23:18:10.688533+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792376+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:18:42.792376+0000","last_peered":"2024-05-08T23:18:42.792376+0000","last_clean":"2024-05-08T23:18:42.792376+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:18:42.792376+0000","last_undegraded":"2024-05-08T23:18:42.792376+0000","last_fullsized":"2024-05-08T23:18:42.792376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795976+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:42.795976+0000","last_peered":"2024-05-08T23:18:42.795976+0000","last_clean":"2024-05-08T23:18:42.795976+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:42.795976+0000","last_undegraded":"2024-05-08T23:18:42.795976+0000","last_fullsized":"2024-05-08T23:18:42.795976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.750725+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:45.750725+0000","last_peered":"2024-05-08T23:18:45.750725+0000","last_clean":"2024-05-08T23:18:45.750725+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:45.750725+0000","last_undegraded":"2024-05-08T23:18:45.750725+0000","last_fullsized":"2024-05-08T23:18:45.750725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800494+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:18:42.800494+0000","last_peered":"2024-05-08T23:18:42.800494+0000","last_clean":"2024-05-08T23:18:42.800494+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:18:42.800494+0000","last_undegraded":"2024-05-08T23:18:42.800494+0000","last_fullsized":"2024-05-08T23:18:42.800494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792335+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:42.792335+0000","last_peered":"2024-05-08T23:18:42.792335+0000","last_clean":"2024-05-08T23:18:42.792335+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:42.792335+0000","last_undegraded":"2024-05-08T23:18:42.792335+0000","last_fullsized":"2024-05-08T23:18:42.792335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689757+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:10.689757+0000","last_peered":"2024-05-08T23:18:10.689757+0000","last_clean":"2024-05-08T23:18:10.689757+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:10.689757+0000","last_undegraded":"2024-05-08T23:18:10.689757+0000","last_fullsized":"2024-05-08T23:18:10.689757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850542+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:18:42.850542+0000","last_peered":"2024-05-08T23:18:42.850542+0000","last_clean":"2024-05-08T23:18:42.850542+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:18:42.850542+0000","last_undegraded":"2024-05-08T23:18:42.850542+0000","last_fullsized":"2024-05-08T23:18:42.850542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793231+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:42.793231+0000","last_peered":"2024-05-08T23:18:42.793231+0000","last_clean":"2024-05-08T23:18:42.793231+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:42.793231+0000","last_undegraded":"2024-05-08T23:18:42.793231+0000","last_fullsized":"2024-05-08T23:18:42.793231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:12.703676+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:12.703676+0000","last_peered":"2024-05-08T23:18:12.703676+0000","last_clean":"2024-05-08T23:18:12.703676+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:12.703676+0000","last_undegraded":"2024-05-08T23:18:12.703676+0000","last_fullsized":"2024-05-08T23:18:12.703676+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800205+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:18:42.800205+0000","last_peered":"2024-05-08T23:18:42.800205+0000","last_clean":"2024-05-08T23:18:42.800205+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:18:42.800205+0000","last_undegraded":"2024-05-08T23:18:42.800205+0000","last_fullsized":"2024-05-08T23:18:42.800205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792502+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:42.792502+0000","last_peered":"2024-05-08T23:18:42.792502+0000","last_clean":"2024-05-08T23:18:42.792502+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:42.792502+0000","last_undegraded":"2024-05-08T23:18:42.792502+0000","last_fullsized":"2024-05-08T23:18:42.792502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:46.751917+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:46.751917+0000","last_peered":"2024-05-08T23:18:46.751917+0000","last_clean":"2024-05-08T23:18:46.751917+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:46.751917+0000","last_undegraded":"2024-05-08T23:18:46.751917+0000","last_fullsized":"2024-05-08T23:18:46.751917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61146,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":78,"num_read_kb":99,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797717+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:18:42.797717+0000","last_peered":"2024-05-08T23:18:42.797717+0000","last_clean":"2024-05-08T23:18:42.797717+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:18:42.797717+0000","last_undegraded":"2024-05-08T23:18:42.797717+0000","last_fullsized":"2024-05-08T23:18:42.797717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616558+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:17:53.616558+0000","last_peered":"2024-05-08T23:17:53.616558+0000","last_clean":"2024-05-08T23:17:53.616558+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:17:53.616558+0000","last_undegraded":"2024-05-08T23:17:53.616558+0000","last_fullsized":"2024-05-08T23:17:53.616558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798509+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:42.798509+0000","last_peered":"2024-05-08T23:18:42.798509+0000","last_clean":"2024-05-08T23:18:42.798509+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:42.798509+0000","last_undegraded":"2024-05-08T23:18:42.798509+0000","last_fullsized":"2024-05-08T23:18:42.798509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799523+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:18:42.799523+0000","last_peered":"2024-05-08T23:18:42.799523+0000","last_clean":"2024-05-08T23:18:42.799523+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:18:42.799523+0000","last_undegraded":"2024-05-08T23:18:42.799523+0000","last_fullsized":"2024-05-08T23:18:42.799523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801502+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:42.801502+0000","last_peered":"2024-05-08T23:18:42.801502+0000","last_clean":"2024-05-08T23:18:42.801502+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:42.801502+0000","last_undegraded":"2024-05-08T23:18:42.801502+0000","last_fullsized":"2024-05-08T23:18:42.801502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689277+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:10.689277+0000","last_peered":"2024-05-08T23:18:10.689277+0000","last_clean":"2024-05-08T23:18:10.689277+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:10.689277+0000","last_undegraded":"2024-05-08T23:18:10.689277+0000","last_fullsized":"2024-05-08T23:18:10.689277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"peering","last_fresh":"2024-05-08T23:18:42.796388+0000","last_change":"2024-05-08T23:18:42.796590+0000","last_active":"2024-05-08T23:18:42.796388+0000","last_peered":"2024-05-08T23:17:50.679860+0000","last_clean":"2024-05-08T23:17:50.679860+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.796388+0000","last_undegraded":"2024-05-08T23:18:42.796388+0000","last_fullsized":"2024-05-08T23:18:42.796388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:52.632483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793500+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:42.793500+0000","last_peered":"2024-05-08T23:18:42.793500+0000","last_clean":"2024-05-08T23:18:42.793500+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:42.793500+0000","last_undegraded":"2024-05-08T23:18:42.793500+0000","last_fullsized":"2024-05-08T23:18:42.793500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.749834+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:44.749834+0000","last_peered":"2024-05-08T23:18:44.749834+0000","last_clean":"2024-05-08T23:18:44.749834+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:44.749834+0000","last_undegraded":"2024-05-08T23:18:44.749834+0000","last_fullsized":"2024-05-08T23:18:44.749834+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679597+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:17:50.679597+0000","last_peered":"2024-05-08T23:17:50.679597+0000","last_clean":"2024-05-08T23:17:50.679597+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:17:50.679597+0000","last_undegraded":"2024-05-08T23:17:50.679597+0000","last_fullsized":"2024-05-08T23:17:50.679597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800355+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:42.800355+0000","last_peered":"2024-05-08T23:18:42.800355+0000","last_clean":"2024-05-08T23:18:42.800355+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:42.800355+0000","last_undegraded":"2024-05-08T23:18:42.800355+0000","last_fullsized":"2024-05-08T23:18:42.800355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796252+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:42.796252+0000","last_peered":"2024-05-08T23:18:42.796252+0000","last_clean":"2024-05-08T23:18:42.796252+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:42.796252+0000","last_undegraded":"2024-05-08T23:18:42.796252+0000","last_fullsized":"2024-05-08T23:18:42.796252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"peering","last_fresh":"2024-05-08T23:18:42.796575+0000","last_change":"2024-05-08T23:18:42.796719+0000","last_active":"2024-05-08T23:18:42.796575+0000","last_peered":"2024-05-08T23:17:50.681270+0000","last_clean":"2024-05-08T23:17:50.681270+0000","last_became_active":"2024-05-08T23:17:46.106950+0000","last_became_peered":"2024-05-08T23:17:46.106950+0000","last_unstale":"2024-05-08T23:18:42.796575+0000","last_undegraded":"2024-05-08T23:18:42.796575+0000","last_fullsized":"2024-05-08T23:18:42.796575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:23.833341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616453+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:17:53.616453+0000","last_peered":"2024-05-08T23:17:53.616453+0000","last_clean":"2024-05-08T23:17:53.616453+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:17:53.616453+0000","last_undegraded":"2024-05-08T23:17:53.616453+0000","last_fullsized":"2024-05-08T23:17:53.616453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798692+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:42.798692+0000","last_peered":"2024-05-08T23:18:42.798692+0000","last_clean":"2024-05-08T23:18:42.798692+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:42.798692+0000","last_undegraded":"2024-05-08T23:18:42.798692+0000","last_fullsized":"2024-05-08T23:18:42.798692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802290+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:18:42.802290+0000","last_peered":"2024-05-08T23:18:42.802290+0000","last_clean":"2024-05-08T23:18:42.802290+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:18:42.802290+0000","last_undegraded":"2024-05-08T23:18:42.802290+0000","last_fullsized":"2024-05-08T23:18:42.802290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796770+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:42.796770+0000","last_peered":"2024-05-08T23:18:42.796770+0000","last_clean":"2024-05-08T23:18:42.796770+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:42.796770+0000","last_undegraded":"2024-05-08T23:18:42.796770+0000","last_fullsized":"2024-05-08T23:18:42.796770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792981+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:42.792981+0000","last_peered":"2024-05-08T23:18:42.792981+0000","last_clean":"2024-05-08T23:18:42.792981+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:42.792981+0000","last_undegraded":"2024-05-08T23:18:42.792981+0000","last_fullsized":"2024-05-08T23:18:42.792981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796821+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:18:42.796821+0000","last_peered":"2024-05-08T23:18:42.796821+0000","last_clean":"2024-05-08T23:18:42.796821+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:18:42.796821+0000","last_undegraded":"2024-05-08T23:18:42.796821+0000","last_fullsized":"2024-05-08T23:18:42.796821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799638+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:42.799638+0000","last_peered":"2024-05-08T23:18:42.799638+0000","last_clean":"2024-05-08T23:18:42.799638+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:42.799638+0000","last_undegraded":"2024-05-08T23:18:42.799638+0000","last_fullsized":"2024-05-08T23:18:42.799638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694313+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:11.694313+0000","last_peered":"2024-05-08T23:18:11.694313+0000","last_clean":"2024-05-08T23:18:11.694313+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:11.694313+0000","last_undegraded":"2024-05-08T23:18:11.694313+0000","last_fullsized":"2024-05-08T23:18:11.694313+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796378+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:18:42.796378+0000","last_peered":"2024-05-08T23:18:42.796378+0000","last_clean":"2024-05-08T23:18:42.796378+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:18:42.796378+0000","last_undegraded":"2024-05-08T23:18:42.796378+0000","last_fullsized":"2024-05-08T23:18:42.796378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799747+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:42.799747+0000","last_peered":"2024-05-08T23:18:42.799747+0000","last_clean":"2024-05-08T23:18:42.799747+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:42.799747+0000","last_undegraded":"2024-05-08T23:18:42.799747+0000","last_fullsized":"2024-05-08T23:18:42.799747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681301+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:17:50.681301+0000","last_peered":"2024-05-08T23:17:50.681301+0000","last_clean":"2024-05-08T23:17:50.681301+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:17:50.681301+0000","last_undegraded":"2024-05-08T23:17:50.681301+0000","last_fullsized":"2024-05-08T23:17:50.681301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.750437+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:45.750437+0000","last_peered":"2024-05-08T23:18:45.750437+0000","last_clean":"2024-05-08T23:18:45.750437+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:45.750437+0000","last_undegraded":"2024-05-08T23:18:45.750437+0000","last_fullsized":"2024-05-08T23:18:45.750437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:13.588999+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:13.588999+0000","last_peered":"2024-05-08T23:18:13.588999+0000","last_clean":"2024-05-08T23:18:13.588999+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:13.588999+0000","last_undegraded":"2024-05-08T23:18:13.588999+0000","last_fullsized":"2024-05-08T23:18:13.588999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796295+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:18:42.796295+0000","last_peered":"2024-05-08T23:18:42.796295+0000","last_clean":"2024-05-08T23:18:42.796295+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:18:42.796295+0000","last_undegraded":"2024-05-08T23:18:42.796295+0000","last_fullsized":"2024-05-08T23:18:42.796295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793103+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:42.793103+0000","last_peered":"2024-05-08T23:18:42.793103+0000","last_clean":"2024-05-08T23:18:42.793103+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:42.793103+0000","last_undegraded":"2024-05-08T23:18:42.793103+0000","last_fullsized":"2024-05-08T23:18:42.793103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:03.628232+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:03.628232+0000","last_peered":"2024-05-08T23:18:03.628232+0000","last_clean":"2024-05-08T23:18:03.628232+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:03.628232+0000","last_undegraded":"2024-05-08T23:18:03.628232+0000","last_fullsized":"2024-05-08T23:18:03.628232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616580+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:17:53.616580+0000","last_peered":"2024-05-08T23:17:53.616580+0000","last_clean":"2024-05-08T23:17:53.616580+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:17:53.616580+0000","last_undegraded":"2024-05-08T23:17:53.616580+0000","last_fullsized":"2024-05-08T23:17:53.616580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799236+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:18:42.799236+0000","last_peered":"2024-05-08T23:18:42.799236+0000","last_clean":"2024-05-08T23:18:42.799236+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:18:42.799236+0000","last_undegraded":"2024-05-08T23:18:42.799236+0000","last_fullsized":"2024-05-08T23:18:42.799236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694199+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:11.694199+0000","last_peered":"2024-05-08T23:18:11.694199+0000","last_clean":"2024-05-08T23:18:11.694199+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:11.694199+0000","last_undegraded":"2024-05-08T23:18:11.694199+0000","last_fullsized":"2024-05-08T23:18:11.694199+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802361+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:42.802361+0000","last_peered":"2024-05-08T23:18:42.802361+0000","last_clean":"2024-05-08T23:18:42.802361+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:42.802361+0000","last_undegraded":"2024-05-08T23:18:42.802361+0000","last_fullsized":"2024-05-08T23:18:42.802361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797799+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:42.797799+0000","last_peered":"2024-05-08T23:18:42.797799+0000","last_clean":"2024-05-08T23:18:42.797799+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:42.797799+0000","last_undegraded":"2024-05-08T23:18:42.797799+0000","last_fullsized":"2024-05-08T23:18:42.797799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799837+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:18:42.799837+0000","last_peered":"2024-05-08T23:18:42.799837+0000","last_clean":"2024-05-08T23:18:42.799837+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:18:42.799837+0000","last_undegraded":"2024-05-08T23:18:42.799837+0000","last_fullsized":"2024-05-08T23:18:42.799837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795769+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:42.795769+0000","last_peered":"2024-05-08T23:18:42.795769+0000","last_clean":"2024-05-08T23:18:42.795769+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:42.795769+0000","last_undegraded":"2024-05-08T23:18:42.795769+0000","last_fullsized":"2024-05-08T23:18:42.795769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793582+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:18:42.793582+0000","last_peered":"2024-05-08T23:18:42.793582+0000","last_clean":"2024-05-08T23:18:42.793582+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:18:42.793582+0000","last_undegraded":"2024-05-08T23:18:42.793582+0000","last_fullsized":"2024-05-08T23:18:42.793582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797388+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:18:42.797388+0000","last_peered":"2024-05-08T23:18:42.797388+0000","last_clean":"2024-05-08T23:18:42.797388+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:18:42.797388+0000","last_undegraded":"2024-05-08T23:18:42.797388+0000","last_fullsized":"2024-05-08T23:18:42.797388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797897+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:42.797897+0000","last_peered":"2024-05-08T23:18:42.797897+0000","last_clean":"2024-05-08T23:18:42.797897+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:42.797897+0000","last_undegraded":"2024-05-08T23:18:42.797897+0000","last_fullsized":"2024-05-08T23:18:42.797897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793553+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:42.793553+0000","last_peered":"2024-05-08T23:18:42.793553+0000","last_clean":"2024-05-08T23:18:42.793553+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:42.793553+0000","last_undegraded":"2024-05-08T23:18:42.793553+0000","last_fullsized":"2024-05-08T23:18:42.793553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850431+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:18:42.850431+0000","last_peered":"2024-05-08T23:18:42.850431+0000","last_clean":"2024-05-08T23:18:42.850431+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:18:42.850431+0000","last_undegraded":"2024-05-08T23:18:42.850431+0000","last_fullsized":"2024-05-08T23:18:42.850431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793223+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:42.793223+0000","last_peered":"2024-05-08T23:18:42.793223+0000","last_clean":"2024-05-08T23:18:42.793223+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:42.793223+0000","last_undegraded":"2024-05-08T23:18:42.793223+0000","last_fullsized":"2024-05-08T23:18:42.793223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799949+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:42.799949+0000","last_peered":"2024-05-08T23:18:42.799949+0000","last_clean":"2024-05-08T23:18:42.799949+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:42.799949+0000","last_undegraded":"2024-05-08T23:18:42.799949+0000","last_fullsized":"2024-05-08T23:18:42.799949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796359+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:18:42.796359+0000","last_peered":"2024-05-08T23:18:42.796359+0000","last_clean":"2024-05-08T23:18:42.796359+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:18:42.796359+0000","last_undegraded":"2024-05-08T23:18:42.796359+0000","last_fullsized":"2024-05-08T23:18:42.796359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793663+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:42.793663+0000","last_peered":"2024-05-08T23:18:42.793663+0000","last_clean":"2024-05-08T23:18:42.793663+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:42.793663+0000","last_undegraded":"2024-05-08T23:18:42.793663+0000","last_fullsized":"2024-05-08T23:18:42.793663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796095+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:42.796095+0000","last_peered":"2024-05-08T23:18:42.796095+0000","last_clean":"2024-05-08T23:18:42.796095+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:42.796095+0000","last_undegraded":"2024-05-08T23:18:42.796095+0000","last_fullsized":"2024-05-08T23:18:42.796095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801296+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:18:42.801296+0000","last_peered":"2024-05-08T23:18:42.801296+0000","last_clean":"2024-05-08T23:18:42.801296+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:18:42.801296+0000","last_undegraded":"2024-05-08T23:18:42.801296+0000","last_fullsized":"2024-05-08T23:18:42.801296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792930+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:42.792930+0000","last_peered":"2024-05-08T23:18:42.792930+0000","last_clean":"2024-05-08T23:18:42.792930+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:42.792930+0000","last_undegraded":"2024-05-08T23:18:42.792930+0000","last_fullsized":"2024-05-08T23:18:42.792930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799130+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:42.799130+0000","last_peered":"2024-05-08T23:18:42.799130+0000","last_clean":"2024-05-08T23:18:42.799130+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:42.799130+0000","last_undegraded":"2024-05-08T23:18:42.799130+0000","last_fullsized":"2024-05-08T23:18:42.799130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796367+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:18:42.796367+0000","last_peered":"2024-05-08T23:18:42.796367+0000","last_clean":"2024-05-08T23:18:42.796367+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:18:42.796367+0000","last_undegraded":"2024-05-08T23:18:42.796367+0000","last_fullsized":"2024-05-08T23:18:42.796367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793402+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:42.793402+0000","last_peered":"2024-05-08T23:18:42.793402+0000","last_clean":"2024-05-08T23:18:42.793402+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:42.793402+0000","last_undegraded":"2024-05-08T23:18:42.793402+0000","last_fullsized":"2024-05-08T23:18:42.793402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.808122+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:43.808122+0000","last_peered":"2024-05-08T23:18:43.808122+0000","last_clean":"2024-05-08T23:18:43.808122+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:43.808122+0000","last_undegraded":"2024-05-08T23:18:43.808122+0000","last_fullsized":"2024-05-08T23:18:43.808122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679645+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:17:50.679645+0000","last_peered":"2024-05-08T23:17:50.679645+0000","last_clean":"2024-05-08T23:17:50.679645+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:17:50.679645+0000","last_undegraded":"2024-05-08T23:17:50.679645+0000","last_fullsized":"2024-05-08T23:17:50.679645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800112+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:42.800112+0000","last_peered":"2024-05-08T23:18:42.800112+0000","last_clean":"2024-05-08T23:18:42.800112+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:42.800112+0000","last_undegraded":"2024-05-08T23:18:42.800112+0000","last_fullsized":"2024-05-08T23:18:42.800112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800360+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:42.800360+0000","last_peered":"2024-05-08T23:18:42.800360+0000","last_clean":"2024-05-08T23:18:42.800360+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:42.800360+0000","last_undegraded":"2024-05-08T23:18:42.800360+0000","last_fullsized":"2024-05-08T23:18:42.800360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680713+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:17:50.680713+0000","last_peered":"2024-05-08T23:17:50.680713+0000","last_clean":"2024-05-08T23:17:50.680713+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:17:50.680713+0000","last_undegraded":"2024-05-08T23:17:50.680713+0000","last_fullsized":"2024-05-08T23:17:50.680713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849917+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:42.849917+0000","last_peered":"2024-05-08T23:18:42.849917+0000","last_clean":"2024-05-08T23:18:42.849917+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:42.849917+0000","last_undegraded":"2024-05-08T23:18:42.849917+0000","last_fullsized":"2024-05-08T23:18:42.849917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:12.700525+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:18:12.700525+0000","last_peered":"2024-05-08T23:18:12.700525+0000","last_clean":"2024-05-08T23:18:12.700525+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:18:12.700525+0000","last_undegraded":"2024-05-08T23:18:12.700525+0000","last_fullsized":"2024-05-08T23:18:12.700525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679646+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:17:50.679646+0000","last_peered":"2024-05-08T23:17:50.679646+0000","last_clean":"2024-05-08T23:17:50.679646+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:17:50.679646+0000","last_undegraded":"2024-05-08T23:17:50.679646+0000","last_fullsized":"2024-05-08T23:17:50.679646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616381+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:17:53.616381+0000","last_peered":"2024-05-08T23:17:53.616381+0000","last_clean":"2024-05-08T23:17:53.616381+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:17:53.616381+0000","last_undegraded":"2024-05-08T23:17:53.616381+0000","last_fullsized":"2024-05-08T23:17:53.616381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931402+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931402+0000","last_peered":"2024-05-08T23:18:43.931402+0000","last_clean":"2024-05-08T23:18:43.931402+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931402+0000","last_undegraded":"2024-05-08T23:18:43.931402+0000","last_fullsized":"2024-05-08T23:18:43.931402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680144+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:17:50.680144+0000","last_peered":"2024-05-08T23:17:50.680144+0000","last_clean":"2024-05-08T23:17:50.680144+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:17:50.680144+0000","last_undegraded":"2024-05-08T23:17:50.680144+0000","last_fullsized":"2024-05-08T23:17:50.680144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801392+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:18:42.801392+0000","last_peered":"2024-05-08T23:18:42.801392+0000","last_clean":"2024-05-08T23:18:42.801392+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:18:42.801392+0000","last_undegraded":"2024-05-08T23:18:42.801392+0000","last_fullsized":"2024-05-08T23:18:42.801392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034679+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:18:44.034679+0000","last_peered":"2024-05-08T23:18:44.034679+0000","last_clean":"2024-05-08T23:18:44.034679+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:18:44.034679+0000","last_undegraded":"2024-05-08T23:18:44.034679+0000","last_fullsized":"2024-05-08T23:18:44.034679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799239+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:18:42.799239+0000","last_peered":"2024-05-08T23:18:42.799239+0000","last_clean":"2024-05-08T23:18:42.799239+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:18:42.799239+0000","last_undegraded":"2024-05-08T23:18:42.799239+0000","last_fullsized":"2024-05-08T23:18:42.799239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850099+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:18:42.850099+0000","last_peered":"2024-05-08T23:18:42.850099+0000","last_clean":"2024-05-08T23:18:42.850099+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:18:42.850099+0000","last_undegraded":"2024-05-08T23:18:42.850099+0000","last_fullsized":"2024-05-08T23:18:42.850099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799941+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:18:42.799941+0000","last_peered":"2024-05-08T23:18:42.799941+0000","last_clean":"2024-05-08T23:18:42.799941+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:18:42.799941+0000","last_undegraded":"2024-05-08T23:18:42.799941+0000","last_fullsized":"2024-05-08T23:18:42.799941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.803347+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:18:42.803347+0000","last_peered":"2024-05-08T23:18:42.803347+0000","last_clean":"2024-05-08T23:18:42.803347+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:18:42.803347+0000","last_undegraded":"2024-05-08T23:18:42.803347+0000","last_fullsized":"2024-05-08T23:18:42.803347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800760+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:18:42.800760+0000","last_peered":"2024-05-08T23:18:42.800760+0000","last_clean":"2024-05-08T23:18:42.800760+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:18:42.800760+0000","last_undegraded":"2024-05-08T23:18:42.800760+0000","last_fullsized":"2024-05-08T23:18:42.800760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797474+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:18:42.797474+0000","last_peered":"2024-05-08T23:18:42.797474+0000","last_clean":"2024-05-08T23:18:42.797474+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:18:42.797474+0000","last_undegraded":"2024-05-08T23:18:42.797474+0000","last_fullsized":"2024-05-08T23:18:42.797474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681194+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:17:50.681194+0000","last_peered":"2024-05-08T23:17:50.681194+0000","last_clean":"2024-05-08T23:17:50.681194+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:17:50.681194+0000","last_undegraded":"2024-05-08T23:17:50.681194+0000","last_fullsized":"2024-05-08T23:17:50.681194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796074+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:18:42.796074+0000","last_peered":"2024-05-08T23:18:42.796074+0000","last_clean":"2024-05-08T23:18:42.796074+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:18:42.796074+0000","last_undegraded":"2024-05-08T23:18:42.796074+0000","last_fullsized":"2024-05-08T23:18:42.796074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800329+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:18:42.800329+0000","last_peered":"2024-05-08T23:18:42.800329+0000","last_clean":"2024-05-08T23:18:42.800329+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:18:42.800329+0000","last_undegraded":"2024-05-08T23:18:42.800329+0000","last_fullsized":"2024-05-08T23:18:42.800329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800055+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:18:42.800055+0000","last_peered":"2024-05-08T23:18:42.800055+0000","last_clean":"2024-05-08T23:18:42.800055+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:18:42.800055+0000","last_undegraded":"2024-05-08T23:18:42.800055+0000","last_fullsized":"2024-05-08T23:18:42.800055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801597+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:18:42.801597+0000","last_peered":"2024-05-08T23:18:42.801597+0000","last_clean":"2024-05-08T23:18:42.801597+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:18:42.801597+0000","last_undegraded":"2024-05-08T23:18:42.801597+0000","last_fullsized":"2024-05-08T23:18:42.801597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":229,"num_read_kb":185,"num_write":156,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645335,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":463111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.439}]},{"osd":5,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52200000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.496},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":9,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78600000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]}]},{"osd":10,"up_from":65,"seq":279172874267,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1617920,"data_stored":1111146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.36399999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.503},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.52600000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.625}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.64600000000000002}]}]},{"osd":9,"up_from":60,"seq":257698037789,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":458188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.505}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.58299999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Wed May 8 23:17:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":8,"up_from":53,"seq":227633266722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86499999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":1.137,"15min":1.137},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92200000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.754},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93500000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81299999999999994}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78500000000000003}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":7,"up_from":46,"seq":197568495652,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36700,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708452,"statfs":{"total":95995035648,"available":95957454848,"internally_reserved":0,"allocated":1523712,"data_stored":1049103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65100000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76100000000000001}]},{"osd":8,"last update":"Wed May 8 23:18:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.71699999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80900000000000005}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.746},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":458523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":1.022},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68400000000000005}]},{"osd":1,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85299999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91600000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.85099999999999998}]},{"osd":3,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.443},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92700000000000005}]},{"osd":5,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71999999999999997}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0049999999999999,"5min":1.177,"15min":1.177},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.046,"5min":1.091,"15min":1.091},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.504}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":978944,"data_stored":510324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.41699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.39000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.72699999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.46800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 8 23:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.50900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76400000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.83899999999999997}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52900000000000003}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68500000000000005,"15min":0.67000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.877}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76400000000000001}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":954368,"data_stored":459911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed May 8 23:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 23:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.624}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57399999999999995}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69999999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.86099999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444087,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":520881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.76500000000000001}]},{"osd":4,"last update":"Wed May 8 23:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.87},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.437,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70299999999999996,"5min":0.872,"15min":0.872},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.753,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.97499999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":1.024}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":1.0089999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":3,"up_from":23,"seq":98784247858,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":925696,"data_stored":449462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.628},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.627,"15min":0.58899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59199999999999997,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.9189999999999996,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.45600000000000002}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.67900000000000005}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.80300000000000005,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.72199999999999998}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.29299999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.373},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.434,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.46500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75800000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.51100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51300000000000001}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83399999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46100000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65200000000000002}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.625},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66800000000000004}]}]},{"osd":5,"up_from":35,"seq":150323855403,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36692,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708460,"statfs":{"total":95995035648,"available":95957463040,"internally_reserved":0,"allocated":1519616,"data_stored":1035602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.52500000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.754,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66900000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42999999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.63100000000000001,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56200000000000006}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66100000000000003}]},{"osd":10,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:18:49.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:18:49.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:18:49.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:18:49.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:48 smithi189 ceph-mon[46919]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:18:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:18:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:48 smithi155 ceph-mon[47105]: from='mgr.14152 172.21.15.16:0/3709363584' entity='mgr.x' 2024-05-08T23:18:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:50.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:49 smithi016 ceph-mon[42541]: pgmap v256: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:50.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:49 smithi016 ceph-mon[42541]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:50.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:49 smithi189 ceph-mon[46919]: pgmap v256: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:50.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:49 smithi189 ceph-mon[46919]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:50.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:49 smithi155 ceph-mon[47105]: pgmap v256: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:50.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:49 smithi155 ceph-mon[47105]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 -- 172.21.15.16:0/1540152452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 msgr2=0x7f2f5c10fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 --2- 172.21.15.16:0/1540152452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c10fe70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400b0a0 tx=0x7f2f5400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 -- 172.21.15.16:0/1540152452 shutdown_connections 2024-05-08T23:18:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 --2- 172.21.15.16:0/1540152452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c10fe70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 --2- 172.21.15.16:0/1540152452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c068950 0x7f2f5c068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 --2- 172.21.15.16:0/1540152452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 0x7f2f5c10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 -- 172.21.15.16:0/1540152452 >> 172.21.15.16:0/1540152452 conn(0x7f2f5c075f80 msgr2=0x7f2f5c076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 -- 172.21.15.16:0/1540152452 shutdown_connections 2024-05-08T23:18:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.375+0000 7f2f63f76700 1 -- 172.21.15.16:0/1540152452 wait complete. 2024-05-08T23:18:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 Processor -- start 2024-05-08T23:18:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 -- start start 2024-05-08T23:18:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c068950 0x7f2f5c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 0x7f2f5c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c19e480 con 0x7f2f5c10da80 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c19e5f0 con 0x7f2f5c068950 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f63f76700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c19e7b0 con 0x7f2f5c069270 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50310/0 (socket says 172.21.15.16:50310) 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 -- 172.21.15.16:0/3407792928 learned_addr learned my addr 172.21.15.16:0/3407792928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f61511700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 0x7f2f5c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f61d12700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c068950 0x7f2f5c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 -- 172.21.15.16:0/3407792928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 msgr2=0x7f2f5c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 0x7f2f5c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 -- 172.21.15.16:0/3407792928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c068950 msgr2=0x7f2f5c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c068950 0x7f2f5c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f62513700 1 -- 172.21.15.16:0/3407792928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54009d00 con 0x7f2f5c10da80 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.376+0000 7f2f61511700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 0x7f2f5c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.377+0000 7f2f62513700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c19de70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400b070 tx=0x7f2f54009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.377+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400e040 con 0x7f2f5c10da80 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.377+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f5400ce50 con 0x7f2f5c10da80 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.377+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5401b450 con 0x7f2f5c10da80 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.377+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c19ea30 con 0x7f2f5c10da80 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.377+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c1a62f0 con 0x7f2f5c10da80 2024-05-08T23:18:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.378+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5c108ca0 con 0x7f2f5c10da80 2024-05-08T23:18:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.379+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2f54004c60 con 0x7f2f5c10da80 2024-05-08T23:18:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.380+0000 7f2f52ffd700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2f480a1230 0x7f2f480a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.380+0000 7f2f61d12700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2f480a1230 0x7f2f480a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.380+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2f540ce9f0 con 0x7f2f5c10da80 2024-05-08T23:18:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.380+0000 7f2f61d12700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2f480a1230 0x7f2f480a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800ac50 tx=0x7f2f5800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.383+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2f54097db0 con 0x7f2f5c10da80 2024-05-08T23:18:50.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.591+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f5c19a890 con 0x7f2f480a1230 2024-05-08T23:18:50.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.604+0000 7f2f52ffd700 1 -- 172.21.15.16:0/3407792928 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423260 (secure 0 0 0) 0x7f2f5c19a890 con 0x7f2f480a1230 2024-05-08T23:18:50.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2f480a1230 msgr2=0x7f2f480a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2f480a1230 0x7f2f480a36f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800ac50 tx=0x7f2f5800a380 comp rx=0 tx=0).stop 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 msgr2=0x7f2f5c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c19de70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400b070 tx=0x7f2f54009380 comp rx=0 tx=0).stop 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 shutdown_connections 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f5c10da80 0x7f2f5c19de70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f5c068950 0x7f2f5c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2f480a1230 0x7f2f480a36f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 --2- 172.21.15.16:0/3407792928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5c069270 0x7f2f5c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 >> 172.21.15.16:0/3407792928 conn(0x7f2f5c075f80 msgr2=0x7f2f5c10ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 shutdown_connections 2024-05-08T23:18:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:50.608+0000 7f2f63f76700 1 -- 172.21.15.16:0/3407792928 wait complete. 2024-05-08T23:18:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:18:50.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-05-08T23:18:48.593596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":365,"num_read_kb":304,"num_write":339,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394500,"kb_used_data":12924,"kb_used_omap":18,"kb_used_meta":381421,"kb_avail":1124547324,"statfs":{"total":1151940427776,"available":1151536459776,"internally_reserved":0,"allocated":13234176,"data_stored":7442643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":390575432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002200"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802073+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:18:42.802073+0000","last_peered":"2024-05-08T23:18:42.802073+0000","last_clean":"2024-05-08T23:18:42.802073+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:18:42.802073+0000","last_undegraded":"2024-05-08T23:18:42.802073+0000","last_fullsized":"2024-05-08T23:18:42.802073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850382+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:18:42.850382+0000","last_peered":"2024-05-08T23:18:42.850382+0000","last_clean":"2024-05-08T23:18:42.850382+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:18:42.850382+0000","last_undegraded":"2024-05-08T23:18:42.850382+0000","last_fullsized":"2024-05-08T23:18:42.850382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850219+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:18:42.850219+0000","last_peered":"2024-05-08T23:18:42.850219+0000","last_clean":"2024-05-08T23:18:42.850219+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:18:42.850219+0000","last_undegraded":"2024-05-08T23:18:42.850219+0000","last_fullsized":"2024-05-08T23:18:42.850219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793298+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:18:42.793298+0000","last_peered":"2024-05-08T23:18:42.793298+0000","last_clean":"2024-05-08T23:18:42.793298+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:18:42.793298+0000","last_undegraded":"2024-05-08T23:18:42.793298+0000","last_fullsized":"2024-05-08T23:18:42.793298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799491+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:18:42.799491+0000","last_peered":"2024-05-08T23:18:42.799491+0000","last_clean":"2024-05-08T23:18:42.799491+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:18:42.799491+0000","last_undegraded":"2024-05-08T23:18:42.799491+0000","last_fullsized":"2024-05-08T23:18:42.799491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796401+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:18:42.796401+0000","last_peered":"2024-05-08T23:18:42.796401+0000","last_clean":"2024-05-08T23:18:42.796401+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:18:42.796401+0000","last_undegraded":"2024-05-08T23:18:42.796401+0000","last_fullsized":"2024-05-08T23:18:42.796401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680125+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:17:50.680125+0000","last_peered":"2024-05-08T23:17:50.680125+0000","last_clean":"2024-05-08T23:17:50.680125+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:17:50.680125+0000","last_undegraded":"2024-05-08T23:17:50.680125+0000","last_fullsized":"2024-05-08T23:17:50.680125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795761+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:18:42.795761+0000","last_peered":"2024-05-08T23:18:42.795761+0000","last_clean":"2024-05-08T23:18:42.795761+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:18:42.795761+0000","last_undegraded":"2024-05-08T23:18:42.795761+0000","last_fullsized":"2024-05-08T23:18:42.795761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800861+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:18:42.800861+0000","last_peered":"2024-05-08T23:18:42.800861+0000","last_clean":"2024-05-08T23:18:42.800861+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:18:42.800861+0000","last_undegraded":"2024-05-08T23:18:42.800861+0000","last_fullsized":"2024-05-08T23:18:42.800861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034666+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:18:44.034666+0000","last_peered":"2024-05-08T23:18:44.034666+0000","last_clean":"2024-05-08T23:18:44.034666+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:18:44.034666+0000","last_undegraded":"2024-05-08T23:18:44.034666+0000","last_fullsized":"2024-05-08T23:18:44.034666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680536+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:17:50.680536+0000","last_peered":"2024-05-08T23:17:50.680536+0000","last_clean":"2024-05-08T23:17:50.680536+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:17:50.680536+0000","last_undegraded":"2024-05-08T23:17:50.680536+0000","last_fullsized":"2024-05-08T23:17:50.680536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793543+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:18:42.793543+0000","last_peered":"2024-05-08T23:18:42.793543+0000","last_clean":"2024-05-08T23:18:42.793543+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:18:42.793543+0000","last_undegraded":"2024-05-08T23:18:42.793543+0000","last_fullsized":"2024-05-08T23:18:42.793543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800158+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:18:42.800158+0000","last_peered":"2024-05-08T23:18:42.800158+0000","last_clean":"2024-05-08T23:18:42.800158+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:18:42.800158+0000","last_undegraded":"2024-05-08T23:18:42.800158+0000","last_fullsized":"2024-05-08T23:18:42.800158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849931+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:18:42.849931+0000","last_peered":"2024-05-08T23:18:42.849931+0000","last_clean":"2024-05-08T23:18:42.849931+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:18:42.849931+0000","last_undegraded":"2024-05-08T23:18:42.849931+0000","last_fullsized":"2024-05-08T23:18:42.849931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849831+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:18:42.849831+0000","last_peered":"2024-05-08T23:18:42.849831+0000","last_clean":"2024-05-08T23:18:42.849831+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:18:42.849831+0000","last_undegraded":"2024-05-08T23:18:42.849831+0000","last_fullsized":"2024-05-08T23:18:42.849831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799862+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:18:42.799862+0000","last_peered":"2024-05-08T23:18:42.799862+0000","last_clean":"2024-05-08T23:18:42.799862+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.799862+0000","last_undegraded":"2024-05-08T23:18:42.799862+0000","last_fullsized":"2024-05-08T23:18:42.799862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801991+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:18:42.801991+0000","last_peered":"2024-05-08T23:18:42.801991+0000","last_clean":"2024-05-08T23:18:42.801991+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:18:42.801991+0000","last_undegraded":"2024-05-08T23:18:42.801991+0000","last_fullsized":"2024-05-08T23:18:42.801991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793563+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:18:42.793563+0000","last_peered":"2024-05-08T23:18:42.793563+0000","last_clean":"2024-05-08T23:18:42.793563+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:18:42.793563+0000","last_undegraded":"2024-05-08T23:18:42.793563+0000","last_fullsized":"2024-05-08T23:18:42.793563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799013+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:18:42.799013+0000","last_peered":"2024-05-08T23:18:42.799013+0000","last_clean":"2024-05-08T23:18:42.799013+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:18:42.799013+0000","last_undegraded":"2024-05-08T23:18:42.799013+0000","last_fullsized":"2024-05-08T23:18:42.799013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800896+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:42.800896+0000","last_peered":"2024-05-08T23:18:42.800896+0000","last_clean":"2024-05-08T23:18:42.800896+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:42.800896+0000","last_undegraded":"2024-05-08T23:18:42.800896+0000","last_fullsized":"2024-05-08T23:18:42.800896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793673+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:42.793673+0000","last_peered":"2024-05-08T23:18:42.793673+0000","last_clean":"2024-05-08T23:18:42.793673+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:42.793673+0000","last_undegraded":"2024-05-08T23:18:42.793673+0000","last_fullsized":"2024-05-08T23:18:42.793673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797273+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:18:42.797273+0000","last_peered":"2024-05-08T23:18:42.797273+0000","last_clean":"2024-05-08T23:18:42.797273+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:18:42.797273+0000","last_undegraded":"2024-05-08T23:18:42.797273+0000","last_fullsized":"2024-05-08T23:18:42.797273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792847+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:42.792847+0000","last_peered":"2024-05-08T23:18:42.792847+0000","last_clean":"2024-05-08T23:18:42.792847+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:42.792847+0000","last_undegraded":"2024-05-08T23:18:42.792847+0000","last_fullsized":"2024-05-08T23:18:42.792847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850495+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:18:42.850495+0000","last_peered":"2024-05-08T23:18:42.850495+0000","last_clean":"2024-05-08T23:18:42.850495+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:18:42.850495+0000","last_undegraded":"2024-05-08T23:18:42.850495+0000","last_fullsized":"2024-05-08T23:18:42.850495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803307+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803307+0000","last_peered":"2024-05-08T23:18:43.803307+0000","last_clean":"2024-05-08T23:18:43.803307+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803307+0000","last_undegraded":"2024-05-08T23:18:43.803307+0000","last_fullsized":"2024-05-08T23:18:43.803307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.813513+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:43.813513+0000","last_peered":"2024-05-08T23:18:43.813513+0000","last_clean":"2024-05-08T23:18:43.813513+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:43.813513+0000","last_undegraded":"2024-05-08T23:18:43.813513+0000","last_fullsized":"2024-05-08T23:18:43.813513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800094+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:18:42.800094+0000","last_peered":"2024-05-08T23:18:42.800094+0000","last_clean":"2024-05-08T23:18:42.800094+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:18:42.800094+0000","last_undegraded":"2024-05-08T23:18:42.800094+0000","last_fullsized":"2024-05-08T23:18:42.800094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792881+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:42.792881+0000","last_peered":"2024-05-08T23:18:42.792881+0000","last_clean":"2024-05-08T23:18:42.792881+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:42.792881+0000","last_undegraded":"2024-05-08T23:18:42.792881+0000","last_fullsized":"2024-05-08T23:18:42.792881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797321+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:42.797321+0000","last_peered":"2024-05-08T23:18:42.797321+0000","last_clean":"2024-05-08T23:18:42.797321+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:42.797321+0000","last_undegraded":"2024-05-08T23:18:42.797321+0000","last_fullsized":"2024-05-08T23:18:42.797321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680608+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:17:50.680608+0000","last_peered":"2024-05-08T23:17:50.680608+0000","last_clean":"2024-05-08T23:17:50.680608+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:17:50.680608+0000","last_undegraded":"2024-05-08T23:17:50.680608+0000","last_fullsized":"2024-05-08T23:17:50.680608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798260+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:42.798260+0000","last_peered":"2024-05-08T23:18:42.798260+0000","last_clean":"2024-05-08T23:18:42.798260+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:42.798260+0000","last_undegraded":"2024-05-08T23:18:42.798260+0000","last_fullsized":"2024-05-08T23:18:42.798260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.747822+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:42.747822+0000","last_peered":"2024-05-08T23:18:42.747822+0000","last_clean":"2024-05-08T23:18:42.747822+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:42.747822+0000","last_undegraded":"2024-05-08T23:18:42.747822+0000","last_fullsized":"2024-05-08T23:18:42.747822+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796860+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:18:42.796860+0000","last_peered":"2024-05-08T23:18:42.796860+0000","last_clean":"2024-05-08T23:18:42.796860+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:18:42.796860+0000","last_undegraded":"2024-05-08T23:18:42.796860+0000","last_fullsized":"2024-05-08T23:18:42.796860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803558+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803558+0000","last_peered":"2024-05-08T23:18:43.803558+0000","last_clean":"2024-05-08T23:18:43.803558+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803558+0000","last_undegraded":"2024-05-08T23:18:43.803558+0000","last_fullsized":"2024-05-08T23:18:43.803558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796303+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:42.796303+0000","last_peered":"2024-05-08T23:18:42.796303+0000","last_clean":"2024-05-08T23:18:42.796303+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:42.796303+0000","last_undegraded":"2024-05-08T23:18:42.796303+0000","last_fullsized":"2024-05-08T23:18:42.796303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680486+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:17:50.680486+0000","last_peered":"2024-05-08T23:17:50.680486+0000","last_clean":"2024-05-08T23:17:50.680486+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:17:50.680486+0000","last_undegraded":"2024-05-08T23:17:50.680486+0000","last_fullsized":"2024-05-08T23:17:50.680486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801918+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:42.801918+0000","last_peered":"2024-05-08T23:18:42.801918+0000","last_clean":"2024-05-08T23:18:42.801918+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:42.801918+0000","last_undegraded":"2024-05-08T23:18:42.801918+0000","last_fullsized":"2024-05-08T23:18:42.801918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799336+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:42.799336+0000","last_peered":"2024-05-08T23:18:42.799336+0000","last_clean":"2024-05-08T23:18:42.799336+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:42.799336+0000","last_undegraded":"2024-05-08T23:18:42.799336+0000","last_fullsized":"2024-05-08T23:18:42.799336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795875+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:18:42.795875+0000","last_peered":"2024-05-08T23:18:42.795875+0000","last_clean":"2024-05-08T23:18:42.795875+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:18:42.795875+0000","last_undegraded":"2024-05-08T23:18:42.795875+0000","last_fullsized":"2024-05-08T23:18:42.795875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850309+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:42.850309+0000","last_peered":"2024-05-08T23:18:42.850309+0000","last_clean":"2024-05-08T23:18:42.850309+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:42.850309+0000","last_undegraded":"2024-05-08T23:18:42.850309+0000","last_fullsized":"2024-05-08T23:18:42.850309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.688533+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:10.688533+0000","last_peered":"2024-05-08T23:18:10.688533+0000","last_clean":"2024-05-08T23:18:10.688533+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:10.688533+0000","last_undegraded":"2024-05-08T23:18:10.688533+0000","last_fullsized":"2024-05-08T23:18:10.688533+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792376+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:18:42.792376+0000","last_peered":"2024-05-08T23:18:42.792376+0000","last_clean":"2024-05-08T23:18:42.792376+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:18:42.792376+0000","last_undegraded":"2024-05-08T23:18:42.792376+0000","last_fullsized":"2024-05-08T23:18:42.792376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795976+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:42.795976+0000","last_peered":"2024-05-08T23:18:42.795976+0000","last_clean":"2024-05-08T23:18:42.795976+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:42.795976+0000","last_undegraded":"2024-05-08T23:18:42.795976+0000","last_fullsized":"2024-05-08T23:18:42.795976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.750725+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:45.750725+0000","last_peered":"2024-05-08T23:18:45.750725+0000","last_clean":"2024-05-08T23:18:45.750725+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:45.750725+0000","last_undegraded":"2024-05-08T23:18:45.750725+0000","last_fullsized":"2024-05-08T23:18:45.750725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800494+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:18:42.800494+0000","last_peered":"2024-05-08T23:18:42.800494+0000","last_clean":"2024-05-08T23:18:42.800494+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:18:42.800494+0000","last_undegraded":"2024-05-08T23:18:42.800494+0000","last_fullsized":"2024-05-08T23:18:42.800494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792335+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:42.792335+0000","last_peered":"2024-05-08T23:18:42.792335+0000","last_clean":"2024-05-08T23:18:42.792335+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:42.792335+0000","last_undegraded":"2024-05-08T23:18:42.792335+0000","last_fullsized":"2024-05-08T23:18:42.792335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689757+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:10.689757+0000","last_peered":"2024-05-08T23:18:10.689757+0000","last_clean":"2024-05-08T23:18:10.689757+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:10.689757+0000","last_undegraded":"2024-05-08T23:18:10.689757+0000","last_fullsized":"2024-05-08T23:18:10.689757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850542+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:18:42.850542+0000","last_peered":"2024-05-08T23:18:42.850542+0000","last_clean":"2024-05-08T23:18:42.850542+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:18:42.850542+0000","last_undegraded":"2024-05-08T23:18:42.850542+0000","last_fullsized":"2024-05-08T23:18:42.850542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793231+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:42.793231+0000","last_peered":"2024-05-08T23:18:42.793231+0000","last_clean":"2024-05-08T23:18:42.793231+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:42.793231+0000","last_undegraded":"2024-05-08T23:18:42.793231+0000","last_fullsized":"2024-05-08T23:18:42.793231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:12.703676+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:12.703676+0000","last_peered":"2024-05-08T23:18:12.703676+0000","last_clean":"2024-05-08T23:18:12.703676+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:12.703676+0000","last_undegraded":"2024-05-08T23:18:12.703676+0000","last_fullsized":"2024-05-08T23:18:12.703676+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800205+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:18:42.800205+0000","last_peered":"2024-05-08T23:18:42.800205+0000","last_clean":"2024-05-08T23:18:42.800205+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:18:42.800205+0000","last_undegraded":"2024-05-08T23:18:42.800205+0000","last_fullsized":"2024-05-08T23:18:42.800205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792502+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:42.792502+0000","last_peered":"2024-05-08T23:18:42.792502+0000","last_clean":"2024-05-08T23:18:42.792502+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:42.792502+0000","last_undegraded":"2024-05-08T23:18:42.792502+0000","last_fullsized":"2024-05-08T23:18:42.792502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:46.751917+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:46.751917+0000","last_peered":"2024-05-08T23:18:46.751917+0000","last_clean":"2024-05-08T23:18:46.751917+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:46.751917+0000","last_undegraded":"2024-05-08T23:18:46.751917+0000","last_fullsized":"2024-05-08T23:18:46.751917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61146,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":78,"num_read_kb":99,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797717+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:18:42.797717+0000","last_peered":"2024-05-08T23:18:42.797717+0000","last_clean":"2024-05-08T23:18:42.797717+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:18:42.797717+0000","last_undegraded":"2024-05-08T23:18:42.797717+0000","last_fullsized":"2024-05-08T23:18:42.797717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616558+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:17:53.616558+0000","last_peered":"2024-05-08T23:17:53.616558+0000","last_clean":"2024-05-08T23:17:53.616558+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:17:53.616558+0000","last_undegraded":"2024-05-08T23:17:53.616558+0000","last_fullsized":"2024-05-08T23:17:53.616558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798509+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:42.798509+0000","last_peered":"2024-05-08T23:18:42.798509+0000","last_clean":"2024-05-08T23:18:42.798509+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:42.798509+0000","last_undegraded":"2024-05-08T23:18:42.798509+0000","last_fullsized":"2024-05-08T23:18:42.798509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799523+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:18:42.799523+0000","last_peered":"2024-05-08T23:18:42.799523+0000","last_clean":"2024-05-08T23:18:42.799523+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:18:42.799523+0000","last_undegraded":"2024-05-08T23:18:42.799523+0000","last_fullsized":"2024-05-08T23:18:42.799523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801502+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:42.801502+0000","last_peered":"2024-05-08T23:18:42.801502+0000","last_clean":"2024-05-08T23:18:42.801502+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:42.801502+0000","last_undegraded":"2024-05-08T23:18:42.801502+0000","last_fullsized":"2024-05-08T23:18:42.801502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:10.689277+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:10.689277+0000","last_peered":"2024-05-08T23:18:10.689277+0000","last_clean":"2024-05-08T23:18:10.689277+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:10.689277+0000","last_undegraded":"2024-05-08T23:18:10.689277+0000","last_fullsized":"2024-05-08T23:18:10.689277+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"peering","last_fresh":"2024-05-08T23:18:42.796388+0000","last_change":"2024-05-08T23:18:42.796590+0000","last_active":"2024-05-08T23:18:42.796388+0000","last_peered":"2024-05-08T23:17:50.679860+0000","last_clean":"2024-05-08T23:17:50.679860+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.796388+0000","last_undegraded":"2024-05-08T23:18:42.796388+0000","last_fullsized":"2024-05-08T23:18:42.796388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:52.632483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793500+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:42.793500+0000","last_peered":"2024-05-08T23:18:42.793500+0000","last_clean":"2024-05-08T23:18:42.793500+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:42.793500+0000","last_undegraded":"2024-05-08T23:18:42.793500+0000","last_fullsized":"2024-05-08T23:18:42.793500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.749834+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:44.749834+0000","last_peered":"2024-05-08T23:18:44.749834+0000","last_clean":"2024-05-08T23:18:44.749834+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:44.749834+0000","last_undegraded":"2024-05-08T23:18:44.749834+0000","last_fullsized":"2024-05-08T23:18:44.749834+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679597+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:17:50.679597+0000","last_peered":"2024-05-08T23:17:50.679597+0000","last_clean":"2024-05-08T23:17:50.679597+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:17:50.679597+0000","last_undegraded":"2024-05-08T23:17:50.679597+0000","last_fullsized":"2024-05-08T23:17:50.679597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800355+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:42.800355+0000","last_peered":"2024-05-08T23:18:42.800355+0000","last_clean":"2024-05-08T23:18:42.800355+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:42.800355+0000","last_undegraded":"2024-05-08T23:18:42.800355+0000","last_fullsized":"2024-05-08T23:18:42.800355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796252+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:42.796252+0000","last_peered":"2024-05-08T23:18:42.796252+0000","last_clean":"2024-05-08T23:18:42.796252+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:42.796252+0000","last_undegraded":"2024-05-08T23:18:42.796252+0000","last_fullsized":"2024-05-08T23:18:42.796252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"peering","last_fresh":"2024-05-08T23:18:42.796575+0000","last_change":"2024-05-08T23:18:42.796719+0000","last_active":"2024-05-08T23:18:42.796575+0000","last_peered":"2024-05-08T23:17:50.681270+0000","last_clean":"2024-05-08T23:17:50.681270+0000","last_became_active":"2024-05-08T23:17:46.106950+0000","last_became_peered":"2024-05-08T23:17:46.106950+0000","last_unstale":"2024-05-08T23:18:42.796575+0000","last_undegraded":"2024-05-08T23:18:42.796575+0000","last_fullsized":"2024-05-08T23:18:42.796575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:55:23.833341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616453+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:17:53.616453+0000","last_peered":"2024-05-08T23:17:53.616453+0000","last_clean":"2024-05-08T23:17:53.616453+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:17:53.616453+0000","last_undegraded":"2024-05-08T23:17:53.616453+0000","last_fullsized":"2024-05-08T23:17:53.616453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798692+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:42.798692+0000","last_peered":"2024-05-08T23:18:42.798692+0000","last_clean":"2024-05-08T23:18:42.798692+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:42.798692+0000","last_undegraded":"2024-05-08T23:18:42.798692+0000","last_fullsized":"2024-05-08T23:18:42.798692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802290+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:18:42.802290+0000","last_peered":"2024-05-08T23:18:42.802290+0000","last_clean":"2024-05-08T23:18:42.802290+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:18:42.802290+0000","last_undegraded":"2024-05-08T23:18:42.802290+0000","last_fullsized":"2024-05-08T23:18:42.802290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796770+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:42.796770+0000","last_peered":"2024-05-08T23:18:42.796770+0000","last_clean":"2024-05-08T23:18:42.796770+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:42.796770+0000","last_undegraded":"2024-05-08T23:18:42.796770+0000","last_fullsized":"2024-05-08T23:18:42.796770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792981+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:42.792981+0000","last_peered":"2024-05-08T23:18:42.792981+0000","last_clean":"2024-05-08T23:18:42.792981+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:42.792981+0000","last_undegraded":"2024-05-08T23:18:42.792981+0000","last_fullsized":"2024-05-08T23:18:42.792981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796821+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:18:42.796821+0000","last_peered":"2024-05-08T23:18:42.796821+0000","last_clean":"2024-05-08T23:18:42.796821+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:18:42.796821+0000","last_undegraded":"2024-05-08T23:18:42.796821+0000","last_fullsized":"2024-05-08T23:18:42.796821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799638+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:42.799638+0000","last_peered":"2024-05-08T23:18:42.799638+0000","last_clean":"2024-05-08T23:18:42.799638+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:42.799638+0000","last_undegraded":"2024-05-08T23:18:42.799638+0000","last_fullsized":"2024-05-08T23:18:42.799638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694313+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:11.694313+0000","last_peered":"2024-05-08T23:18:11.694313+0000","last_clean":"2024-05-08T23:18:11.694313+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:11.694313+0000","last_undegraded":"2024-05-08T23:18:11.694313+0000","last_fullsized":"2024-05-08T23:18:11.694313+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796378+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:18:42.796378+0000","last_peered":"2024-05-08T23:18:42.796378+0000","last_clean":"2024-05-08T23:18:42.796378+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:18:42.796378+0000","last_undegraded":"2024-05-08T23:18:42.796378+0000","last_fullsized":"2024-05-08T23:18:42.796378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799747+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:42.799747+0000","last_peered":"2024-05-08T23:18:42.799747+0000","last_clean":"2024-05-08T23:18:42.799747+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:42.799747+0000","last_undegraded":"2024-05-08T23:18:42.799747+0000","last_fullsized":"2024-05-08T23:18:42.799747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681301+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:17:50.681301+0000","last_peered":"2024-05-08T23:17:50.681301+0000","last_clean":"2024-05-08T23:17:50.681301+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:17:50.681301+0000","last_undegraded":"2024-05-08T23:17:50.681301+0000","last_fullsized":"2024-05-08T23:17:50.681301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.750437+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:45.750437+0000","last_peered":"2024-05-08T23:18:45.750437+0000","last_clean":"2024-05-08T23:18:45.750437+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:45.750437+0000","last_undegraded":"2024-05-08T23:18:45.750437+0000","last_fullsized":"2024-05-08T23:18:45.750437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:13.588999+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:13.588999+0000","last_peered":"2024-05-08T23:18:13.588999+0000","last_clean":"2024-05-08T23:18:13.588999+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:13.588999+0000","last_undegraded":"2024-05-08T23:18:13.588999+0000","last_fullsized":"2024-05-08T23:18:13.588999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796295+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:18:42.796295+0000","last_peered":"2024-05-08T23:18:42.796295+0000","last_clean":"2024-05-08T23:18:42.796295+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:18:42.796295+0000","last_undegraded":"2024-05-08T23:18:42.796295+0000","last_fullsized":"2024-05-08T23:18:42.796295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793103+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:42.793103+0000","last_peered":"2024-05-08T23:18:42.793103+0000","last_clean":"2024-05-08T23:18:42.793103+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:42.793103+0000","last_undegraded":"2024-05-08T23:18:42.793103+0000","last_fullsized":"2024-05-08T23:18:42.793103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:03.628232+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:03.628232+0000","last_peered":"2024-05-08T23:18:03.628232+0000","last_clean":"2024-05-08T23:18:03.628232+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:03.628232+0000","last_undegraded":"2024-05-08T23:18:03.628232+0000","last_fullsized":"2024-05-08T23:18:03.628232+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616580+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:17:53.616580+0000","last_peered":"2024-05-08T23:17:53.616580+0000","last_clean":"2024-05-08T23:17:53.616580+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:17:53.616580+0000","last_undegraded":"2024-05-08T23:17:53.616580+0000","last_fullsized":"2024-05-08T23:17:53.616580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799236+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:18:42.799236+0000","last_peered":"2024-05-08T23:18:42.799236+0000","last_clean":"2024-05-08T23:18:42.799236+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:18:42.799236+0000","last_undegraded":"2024-05-08T23:18:42.799236+0000","last_fullsized":"2024-05-08T23:18:42.799236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:11.694199+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:11.694199+0000","last_peered":"2024-05-08T23:18:11.694199+0000","last_clean":"2024-05-08T23:18:11.694199+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:11.694199+0000","last_undegraded":"2024-05-08T23:18:11.694199+0000","last_fullsized":"2024-05-08T23:18:11.694199+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802361+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:42.802361+0000","last_peered":"2024-05-08T23:18:42.802361+0000","last_clean":"2024-05-08T23:18:42.802361+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:42.802361+0000","last_undegraded":"2024-05-08T23:18:42.802361+0000","last_fullsized":"2024-05-08T23:18:42.802361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797799+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:42.797799+0000","last_peered":"2024-05-08T23:18:42.797799+0000","last_clean":"2024-05-08T23:18:42.797799+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:42.797799+0000","last_undegraded":"2024-05-08T23:18:42.797799+0000","last_fullsized":"2024-05-08T23:18:42.797799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799837+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:18:42.799837+0000","last_peered":"2024-05-08T23:18:42.799837+0000","last_clean":"2024-05-08T23:18:42.799837+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:18:42.799837+0000","last_undegraded":"2024-05-08T23:18:42.799837+0000","last_fullsized":"2024-05-08T23:18:42.799837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795769+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:42.795769+0000","last_peered":"2024-05-08T23:18:42.795769+0000","last_clean":"2024-05-08T23:18:42.795769+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:42.795769+0000","last_undegraded":"2024-05-08T23:18:42.795769+0000","last_fullsized":"2024-05-08T23:18:42.795769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793582+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:18:42.793582+0000","last_peered":"2024-05-08T23:18:42.793582+0000","last_clean":"2024-05-08T23:18:42.793582+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:18:42.793582+0000","last_undegraded":"2024-05-08T23:18:42.793582+0000","last_fullsized":"2024-05-08T23:18:42.793582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797388+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:18:42.797388+0000","last_peered":"2024-05-08T23:18:42.797388+0000","last_clean":"2024-05-08T23:18:42.797388+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:18:42.797388+0000","last_undegraded":"2024-05-08T23:18:42.797388+0000","last_fullsized":"2024-05-08T23:18:42.797388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797897+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:42.797897+0000","last_peered":"2024-05-08T23:18:42.797897+0000","last_clean":"2024-05-08T23:18:42.797897+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:42.797897+0000","last_undegraded":"2024-05-08T23:18:42.797897+0000","last_fullsized":"2024-05-08T23:18:42.797897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793553+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:42.793553+0000","last_peered":"2024-05-08T23:18:42.793553+0000","last_clean":"2024-05-08T23:18:42.793553+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:42.793553+0000","last_undegraded":"2024-05-08T23:18:42.793553+0000","last_fullsized":"2024-05-08T23:18:42.793553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850431+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:18:42.850431+0000","last_peered":"2024-05-08T23:18:42.850431+0000","last_clean":"2024-05-08T23:18:42.850431+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:18:42.850431+0000","last_undegraded":"2024-05-08T23:18:42.850431+0000","last_fullsized":"2024-05-08T23:18:42.850431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793223+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:42.793223+0000","last_peered":"2024-05-08T23:18:42.793223+0000","last_clean":"2024-05-08T23:18:42.793223+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:42.793223+0000","last_undegraded":"2024-05-08T23:18:42.793223+0000","last_fullsized":"2024-05-08T23:18:42.793223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799949+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:42.799949+0000","last_peered":"2024-05-08T23:18:42.799949+0000","last_clean":"2024-05-08T23:18:42.799949+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:42.799949+0000","last_undegraded":"2024-05-08T23:18:42.799949+0000","last_fullsized":"2024-05-08T23:18:42.799949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796359+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:18:42.796359+0000","last_peered":"2024-05-08T23:18:42.796359+0000","last_clean":"2024-05-08T23:18:42.796359+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:18:42.796359+0000","last_undegraded":"2024-05-08T23:18:42.796359+0000","last_fullsized":"2024-05-08T23:18:42.796359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793663+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:42.793663+0000","last_peered":"2024-05-08T23:18:42.793663+0000","last_clean":"2024-05-08T23:18:42.793663+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:42.793663+0000","last_undegraded":"2024-05-08T23:18:42.793663+0000","last_fullsized":"2024-05-08T23:18:42.793663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796095+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:42.796095+0000","last_peered":"2024-05-08T23:18:42.796095+0000","last_clean":"2024-05-08T23:18:42.796095+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:42.796095+0000","last_undegraded":"2024-05-08T23:18:42.796095+0000","last_fullsized":"2024-05-08T23:18:42.796095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801296+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:18:42.801296+0000","last_peered":"2024-05-08T23:18:42.801296+0000","last_clean":"2024-05-08T23:18:42.801296+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:18:42.801296+0000","last_undegraded":"2024-05-08T23:18:42.801296+0000","last_fullsized":"2024-05-08T23:18:42.801296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792930+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:42.792930+0000","last_peered":"2024-05-08T23:18:42.792930+0000","last_clean":"2024-05-08T23:18:42.792930+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:42.792930+0000","last_undegraded":"2024-05-08T23:18:42.792930+0000","last_fullsized":"2024-05-08T23:18:42.792930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799130+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:42.799130+0000","last_peered":"2024-05-08T23:18:42.799130+0000","last_clean":"2024-05-08T23:18:42.799130+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:42.799130+0000","last_undegraded":"2024-05-08T23:18:42.799130+0000","last_fullsized":"2024-05-08T23:18:42.799130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796367+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:18:42.796367+0000","last_peered":"2024-05-08T23:18:42.796367+0000","last_clean":"2024-05-08T23:18:42.796367+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:18:42.796367+0000","last_undegraded":"2024-05-08T23:18:42.796367+0000","last_fullsized":"2024-05-08T23:18:42.796367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793402+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:42.793402+0000","last_peered":"2024-05-08T23:18:42.793402+0000","last_clean":"2024-05-08T23:18:42.793402+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:42.793402+0000","last_undegraded":"2024-05-08T23:18:42.793402+0000","last_fullsized":"2024-05-08T23:18:42.793402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.808122+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:43.808122+0000","last_peered":"2024-05-08T23:18:43.808122+0000","last_clean":"2024-05-08T23:18:43.808122+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:43.808122+0000","last_undegraded":"2024-05-08T23:18:43.808122+0000","last_fullsized":"2024-05-08T23:18:43.808122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679645+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:17:50.679645+0000","last_peered":"2024-05-08T23:17:50.679645+0000","last_clean":"2024-05-08T23:17:50.679645+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:17:50.679645+0000","last_undegraded":"2024-05-08T23:17:50.679645+0000","last_fullsized":"2024-05-08T23:17:50.679645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800112+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:42.800112+0000","last_peered":"2024-05-08T23:18:42.800112+0000","last_clean":"2024-05-08T23:18:42.800112+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:42.800112+0000","last_undegraded":"2024-05-08T23:18:42.800112+0000","last_fullsized":"2024-05-08T23:18:42.800112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800360+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:42.800360+0000","last_peered":"2024-05-08T23:18:42.800360+0000","last_clean":"2024-05-08T23:18:42.800360+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:42.800360+0000","last_undegraded":"2024-05-08T23:18:42.800360+0000","last_fullsized":"2024-05-08T23:18:42.800360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680713+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:17:50.680713+0000","last_peered":"2024-05-08T23:17:50.680713+0000","last_clean":"2024-05-08T23:17:50.680713+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:17:50.680713+0000","last_undegraded":"2024-05-08T23:17:50.680713+0000","last_fullsized":"2024-05-08T23:17:50.680713+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849917+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:42.849917+0000","last_peered":"2024-05-08T23:18:42.849917+0000","last_clean":"2024-05-08T23:18:42.849917+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:42.849917+0000","last_undegraded":"2024-05-08T23:18:42.849917+0000","last_fullsized":"2024-05-08T23:18:42.849917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:18:12.700525+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:18:12.700525+0000","last_peered":"2024-05-08T23:18:12.700525+0000","last_clean":"2024-05-08T23:18:12.700525+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:18:12.700525+0000","last_undegraded":"2024-05-08T23:18:12.700525+0000","last_fullsized":"2024-05-08T23:18:12.700525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.679646+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:17:50.679646+0000","last_peered":"2024-05-08T23:17:50.679646+0000","last_clean":"2024-05-08T23:17:50.679646+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:17:50.679646+0000","last_undegraded":"2024-05-08T23:17:50.679646+0000","last_fullsized":"2024-05-08T23:17:50.679646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:53.616381+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:17:53.616381+0000","last_peered":"2024-05-08T23:17:53.616381+0000","last_clean":"2024-05-08T23:17:53.616381+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:17:53.616381+0000","last_undegraded":"2024-05-08T23:17:53.616381+0000","last_fullsized":"2024-05-08T23:17:53.616381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931402+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931402+0000","last_peered":"2024-05-08T23:18:43.931402+0000","last_clean":"2024-05-08T23:18:43.931402+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931402+0000","last_undegraded":"2024-05-08T23:18:43.931402+0000","last_fullsized":"2024-05-08T23:18:43.931402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.680144+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:17:50.680144+0000","last_peered":"2024-05-08T23:17:50.680144+0000","last_clean":"2024-05-08T23:17:50.680144+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:17:50.680144+0000","last_undegraded":"2024-05-08T23:17:50.680144+0000","last_fullsized":"2024-05-08T23:17:50.680144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801392+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:18:42.801392+0000","last_peered":"2024-05-08T23:18:42.801392+0000","last_clean":"2024-05-08T23:18:42.801392+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:18:42.801392+0000","last_undegraded":"2024-05-08T23:18:42.801392+0000","last_fullsized":"2024-05-08T23:18:42.801392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034679+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:18:44.034679+0000","last_peered":"2024-05-08T23:18:44.034679+0000","last_clean":"2024-05-08T23:18:44.034679+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:18:44.034679+0000","last_undegraded":"2024-05-08T23:18:44.034679+0000","last_fullsized":"2024-05-08T23:18:44.034679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799239+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:18:42.799239+0000","last_peered":"2024-05-08T23:18:42.799239+0000","last_clean":"2024-05-08T23:18:42.799239+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:18:42.799239+0000","last_undegraded":"2024-05-08T23:18:42.799239+0000","last_fullsized":"2024-05-08T23:18:42.799239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850099+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:18:42.850099+0000","last_peered":"2024-05-08T23:18:42.850099+0000","last_clean":"2024-05-08T23:18:42.850099+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:18:42.850099+0000","last_undegraded":"2024-05-08T23:18:42.850099+0000","last_fullsized":"2024-05-08T23:18:42.850099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799941+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:18:42.799941+0000","last_peered":"2024-05-08T23:18:42.799941+0000","last_clean":"2024-05-08T23:18:42.799941+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:18:42.799941+0000","last_undegraded":"2024-05-08T23:18:42.799941+0000","last_fullsized":"2024-05-08T23:18:42.799941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.803347+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:18:42.803347+0000","last_peered":"2024-05-08T23:18:42.803347+0000","last_clean":"2024-05-08T23:18:42.803347+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:18:42.803347+0000","last_undegraded":"2024-05-08T23:18:42.803347+0000","last_fullsized":"2024-05-08T23:18:42.803347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800760+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:18:42.800760+0000","last_peered":"2024-05-08T23:18:42.800760+0000","last_clean":"2024-05-08T23:18:42.800760+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:18:42.800760+0000","last_undegraded":"2024-05-08T23:18:42.800760+0000","last_fullsized":"2024-05-08T23:18:42.800760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797474+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:18:42.797474+0000","last_peered":"2024-05-08T23:18:42.797474+0000","last_clean":"2024-05-08T23:18:42.797474+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:18:42.797474+0000","last_undegraded":"2024-05-08T23:18:42.797474+0000","last_fullsized":"2024-05-08T23:18:42.797474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-08T23:17:50.681194+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:17:50.681194+0000","last_peered":"2024-05-08T23:17:50.681194+0000","last_clean":"2024-05-08T23:17:50.681194+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:17:50.681194+0000","last_undegraded":"2024-05-08T23:17:50.681194+0000","last_fullsized":"2024-05-08T23:17:50.681194+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796074+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:18:42.796074+0000","last_peered":"2024-05-08T23:18:42.796074+0000","last_clean":"2024-05-08T23:18:42.796074+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:18:42.796074+0000","last_undegraded":"2024-05-08T23:18:42.796074+0000","last_fullsized":"2024-05-08T23:18:42.796074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800329+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:18:42.800329+0000","last_peered":"2024-05-08T23:18:42.800329+0000","last_clean":"2024-05-08T23:18:42.800329+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:18:42.800329+0000","last_undegraded":"2024-05-08T23:18:42.800329+0000","last_fullsized":"2024-05-08T23:18:42.800329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800055+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:18:42.800055+0000","last_peered":"2024-05-08T23:18:42.800055+0000","last_clean":"2024-05-08T23:18:42.800055+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:18:42.800055+0000","last_undegraded":"2024-05-08T23:18:42.800055+0000","last_fullsized":"2024-05-08T23:18:42.800055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801597+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:18:42.801597+0000","last_peered":"2024-05-08T23:18:42.801597+0000","last_clean":"2024-05-08T23:18:42.801597+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:18:42.801597+0000","last_undegraded":"2024-05-08T23:18:42.801597+0000","last_fullsized":"2024-05-08T23:18:42.801597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":229,"num_read_kb":185,"num_write":156,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645335,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":463111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.439}]},{"osd":5,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52200000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.496},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":9,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78600000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]}]},{"osd":10,"up_from":65,"seq":279172874267,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1617920,"data_stored":1111146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.36399999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.503},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.52600000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.625}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.64600000000000002}]}]},{"osd":9,"up_from":60,"seq":257698037789,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":458188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65400000000000003}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.505}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.58299999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Wed May 8 23:17:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":8,"up_from":53,"seq":227633266722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86499999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":1.137,"15min":1.137},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92200000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.754},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93500000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81299999999999994}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78500000000000003}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":7,"up_from":46,"seq":197568495652,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36700,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708452,"statfs":{"total":95995035648,"available":95957454848,"internally_reserved":0,"allocated":1523712,"data_stored":1049103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65100000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76100000000000001}]},{"osd":8,"last update":"Wed May 8 23:18:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.71699999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80900000000000005}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.746},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":458523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":1.022},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68400000000000005}]},{"osd":1,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85299999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91600000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.85099999999999998}]},{"osd":3,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.443},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92700000000000005}]},{"osd":5,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71999999999999997}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0049999999999999,"5min":1.177,"15min":1.177},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.046,"5min":1.091,"15min":1.091},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.504}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36168,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708984,"statfs":{"total":95995035648,"available":95957999616,"internally_reserved":0,"allocated":978944,"data_stored":510324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.41699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.39000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.72699999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.60899999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.46800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 8 23:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.50900000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80400000000000005,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59299999999999997,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76400000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.83899999999999997}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52900000000000003}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68500000000000005,"15min":0.67000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.877}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76400000000000001}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.84999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":954368,"data_stored":459911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed May 8 23:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 23:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.624}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57399999999999995}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69999999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.86099999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444087,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":520881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.76500000000000001}]},{"osd":4,"last update":"Wed May 8 23:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.87},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.437,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70299999999999996,"5min":0.872,"15min":0.872},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.753,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.97499999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":1.024}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":1.0089999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":3,"up_from":23,"seq":98784247858,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":925696,"data_stored":449462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.53500000000000003}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.628},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.627,"15min":0.58899999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.0869999999999997,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.59199999999999997,"15min":0.56299999999999994},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.9189999999999996,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.45600000000000002}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.67900000000000005}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.80300000000000005,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.72199999999999998}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.29299999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.373},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.434,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.46500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75800000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.51100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51300000000000001}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83399999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46100000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65200000000000002}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.625},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66800000000000004}]}]},{"osd":5,"up_from":35,"seq":150323855403,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36692,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708460,"statfs":{"total":95995035648,"available":95957463040,"internally_reserved":0,"allocated":1519616,"data_stored":1035602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.52500000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.754,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66900000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42999999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.63100000000000001,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53600000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56200000000000006}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66100000000000003}]},{"osd":10,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:18:50.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:18:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:52.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:51 smithi016 ceph-mon[42541]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:52.085 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:51 smithi016 ceph-mon[42541]: pgmap v257: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:52.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:51 smithi189 ceph-mon[46919]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:52.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:51 smithi189 ceph-mon[46919]: pgmap v257: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:51 smithi155 ceph-mon[47105]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:51 smithi155 ceph-mon[47105]: pgmap v257: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 -- 172.21.15.16:0/270811336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690068950 msgr2=0x7fa690068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 --2- 172.21.15.16:0/270811336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690068950 0x7fa690068d30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009990 tx=0x7fa68c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 -- 172.21.15.16:0/270811336 shutdown_connections 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 --2- 172.21.15.16:0/270811336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa69010da80 0x7fa69010fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 --2- 172.21.15.16:0/270811336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa690068950 0x7fa690068d30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 --2- 172.21.15.16:0/270811336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690069270 0x7fa69010d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 -- 172.21.15.16:0/270811336 >> 172.21.15.16:0/270811336 conn(0x7fa690075f80 msgr2=0x7fa690076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:52.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 -- 172.21.15.16:0/270811336 shutdown_connections 2024-05-08T23:18:52.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.290+0000 7fa6977f4700 1 -- 172.21.15.16:0/270811336 wait complete. 2024-05-08T23:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 Processor -- start 2024-05-08T23:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 -- start start 2024-05-08T23:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690068950 0x7fa6901995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa690069270 0x7fa690199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 0x7fa69019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69019e4c0 con 0x7fa69010da80 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69019e630 con 0x7fa690068950 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa6977f4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69019e7f0 con 0x7fa690069270 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa695d91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 0x7fa69019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.291+0000 7fa695d91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 0x7fa69019deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50334/0 (socket says 172.21.15.16:50334) 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 -- 172.21.15.16:0/2751220236 learned_addr learned my addr 172.21.15.16:0/2751220236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695590700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690068950 0x7fa6901995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa694d8f700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa690069270 0x7fa690199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 -- 172.21.15.16:0/2751220236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa690069270 msgr2=0x7fa690199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa690069270 0x7fa690199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 -- 172.21.15.16:0/2751220236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690068950 msgr2=0x7fa6901995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690068950 0x7fa6901995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 -- 172.21.15.16:0/2751220236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0095f0 con 0x7fa69010da80 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695590700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690068950 0x7fa6901995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa695d91700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 0x7fa69019deb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa68800ada0 tx=0x7fa68800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800f070 con 0x7fa69010da80 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69019ead0 con 0x7fa69010da80 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa688007b00 con 0x7fa69010da80 2024-05-08T23:18:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688004ea0 con 0x7fa69010da80 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.292+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6901a6330 con 0x7fa69010da80 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.293+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa690108ca0 con 0x7fa69010da80 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.295+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa68800a090 con 0x7fa69010da80 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.295+0000 7fa6867fc700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa67c0a11b0 0x7fa67c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.295+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa6880cc740 con 0x7fa69010da80 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.295+0000 7fa695590700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa67c0a11b0 0x7fa67c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:52.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.296+0000 7fa695590700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa67c0a11b0 0x7fa67c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b5c0 tx=0x7fa68c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:52.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.299+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa688095b00 con 0x7fa69010da80 2024-05-08T23:18:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.506+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa69019a8d0 con 0x7fa67c0a11b0 2024-05-08T23:18:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.519+0000 7fa6867fc700 1 -- 172.21.15.16:0/2751220236 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424475 (secure 0 0 0) 0x7fa69019a8d0 con 0x7fa67c0a11b0 2024-05-08T23:18:52.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa67c0a11b0 msgr2=0x7fa67c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa67c0a11b0 0x7fa67c0a3670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00b5c0 tx=0x7fa68c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 msgr2=0x7fa69019deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 0x7fa69019deb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa68800ada0 tx=0x7fa68800c330 comp rx=0 tx=0).stop 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 shutdown_connections 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa69010da80 0x7fa69019deb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa690068950 0x7fa6901995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa67c0a11b0 0x7fa67c0a3670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 --2- 172.21.15.16:0/2751220236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa690069270 0x7fa690199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 >> 172.21.15.16:0/2751220236 conn(0x7fa690075f80 msgr2=0x7fa69010f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 shutdown_connections 2024-05-08T23:18:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:52.524+0000 7fa6977f4700 1 -- 172.21.15.16:0/2751220236 wait complete. 2024-05-08T23:18:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:18:52.903 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-05-08T23:18:50.605265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":370,"num_read_kb":304,"num_write":339,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394608,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":381421,"kb_avail":1124547216,"statfs":{"total":1151940427776,"available":1151536349184,"internally_reserved":0,"allocated":13332480,"data_stored":7498386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":390575432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.013143"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802073+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:18:42.802073+0000","last_peered":"2024-05-08T23:18:42.802073+0000","last_clean":"2024-05-08T23:18:42.802073+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:18:42.802073+0000","last_undegraded":"2024-05-08T23:18:42.802073+0000","last_fullsized":"2024-05-08T23:18:42.802073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850382+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:18:42.850382+0000","last_peered":"2024-05-08T23:18:42.850382+0000","last_clean":"2024-05-08T23:18:42.850382+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:18:42.850382+0000","last_undegraded":"2024-05-08T23:18:42.850382+0000","last_fullsized":"2024-05-08T23:18:42.850382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850219+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:18:42.850219+0000","last_peered":"2024-05-08T23:18:42.850219+0000","last_clean":"2024-05-08T23:18:42.850219+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:18:42.850219+0000","last_undegraded":"2024-05-08T23:18:42.850219+0000","last_fullsized":"2024-05-08T23:18:42.850219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793298+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:18:42.793298+0000","last_peered":"2024-05-08T23:18:42.793298+0000","last_clean":"2024-05-08T23:18:42.793298+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:18:42.793298+0000","last_undegraded":"2024-05-08T23:18:42.793298+0000","last_fullsized":"2024-05-08T23:18:42.793298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799491+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:18:42.799491+0000","last_peered":"2024-05-08T23:18:42.799491+0000","last_clean":"2024-05-08T23:18:42.799491+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:18:42.799491+0000","last_undegraded":"2024-05-08T23:18:42.799491+0000","last_fullsized":"2024-05-08T23:18:42.799491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796401+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:18:42.796401+0000","last_peered":"2024-05-08T23:18:42.796401+0000","last_clean":"2024-05-08T23:18:42.796401+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:18:42.796401+0000","last_undegraded":"2024-05-08T23:18:42.796401+0000","last_fullsized":"2024-05-08T23:18:42.796401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797424+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:18:42.797424+0000","last_peered":"2024-05-08T23:18:42.797424+0000","last_clean":"2024-05-08T23:18:42.797424+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:18:42.797424+0000","last_undegraded":"2024-05-08T23:18:42.797424+0000","last_fullsized":"2024-05-08T23:18:42.797424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795761+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:18:42.795761+0000","last_peered":"2024-05-08T23:18:42.795761+0000","last_clean":"2024-05-08T23:18:42.795761+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:18:42.795761+0000","last_undegraded":"2024-05-08T23:18:42.795761+0000","last_fullsized":"2024-05-08T23:18:42.795761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800861+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:18:42.800861+0000","last_peered":"2024-05-08T23:18:42.800861+0000","last_clean":"2024-05-08T23:18:42.800861+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:18:42.800861+0000","last_undegraded":"2024-05-08T23:18:42.800861+0000","last_fullsized":"2024-05-08T23:18:42.800861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034666+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:18:44.034666+0000","last_peered":"2024-05-08T23:18:44.034666+0000","last_clean":"2024-05-08T23:18:44.034666+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:18:44.034666+0000","last_undegraded":"2024-05-08T23:18:44.034666+0000","last_fullsized":"2024-05-08T23:18:44.034666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797414+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:18:42.797414+0000","last_peered":"2024-05-08T23:18:42.797414+0000","last_clean":"2024-05-08T23:18:42.797414+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:18:42.797414+0000","last_undegraded":"2024-05-08T23:18:42.797414+0000","last_fullsized":"2024-05-08T23:18:42.797414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793543+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:18:42.793543+0000","last_peered":"2024-05-08T23:18:42.793543+0000","last_clean":"2024-05-08T23:18:42.793543+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:18:42.793543+0000","last_undegraded":"2024-05-08T23:18:42.793543+0000","last_fullsized":"2024-05-08T23:18:42.793543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800158+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:18:42.800158+0000","last_peered":"2024-05-08T23:18:42.800158+0000","last_clean":"2024-05-08T23:18:42.800158+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:18:42.800158+0000","last_undegraded":"2024-05-08T23:18:42.800158+0000","last_fullsized":"2024-05-08T23:18:42.800158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849931+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:18:42.849931+0000","last_peered":"2024-05-08T23:18:42.849931+0000","last_clean":"2024-05-08T23:18:42.849931+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:18:42.849931+0000","last_undegraded":"2024-05-08T23:18:42.849931+0000","last_fullsized":"2024-05-08T23:18:42.849931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849831+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:18:42.849831+0000","last_peered":"2024-05-08T23:18:42.849831+0000","last_clean":"2024-05-08T23:18:42.849831+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:18:42.849831+0000","last_undegraded":"2024-05-08T23:18:42.849831+0000","last_fullsized":"2024-05-08T23:18:42.849831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799862+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:18:42.799862+0000","last_peered":"2024-05-08T23:18:42.799862+0000","last_clean":"2024-05-08T23:18:42.799862+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.799862+0000","last_undegraded":"2024-05-08T23:18:42.799862+0000","last_fullsized":"2024-05-08T23:18:42.799862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801991+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:18:42.801991+0000","last_peered":"2024-05-08T23:18:42.801991+0000","last_clean":"2024-05-08T23:18:42.801991+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:18:42.801991+0000","last_undegraded":"2024-05-08T23:18:42.801991+0000","last_fullsized":"2024-05-08T23:18:42.801991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793563+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:18:42.793563+0000","last_peered":"2024-05-08T23:18:42.793563+0000","last_clean":"2024-05-08T23:18:42.793563+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:18:42.793563+0000","last_undegraded":"2024-05-08T23:18:42.793563+0000","last_fullsized":"2024-05-08T23:18:42.793563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799013+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:18:42.799013+0000","last_peered":"2024-05-08T23:18:42.799013+0000","last_clean":"2024-05-08T23:18:42.799013+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:18:42.799013+0000","last_undegraded":"2024-05-08T23:18:42.799013+0000","last_fullsized":"2024-05-08T23:18:42.799013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800896+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:42.800896+0000","last_peered":"2024-05-08T23:18:42.800896+0000","last_clean":"2024-05-08T23:18:42.800896+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:42.800896+0000","last_undegraded":"2024-05-08T23:18:42.800896+0000","last_fullsized":"2024-05-08T23:18:42.800896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793673+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:42.793673+0000","last_peered":"2024-05-08T23:18:42.793673+0000","last_clean":"2024-05-08T23:18:42.793673+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:42.793673+0000","last_undegraded":"2024-05-08T23:18:42.793673+0000","last_fullsized":"2024-05-08T23:18:42.793673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797273+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:18:42.797273+0000","last_peered":"2024-05-08T23:18:42.797273+0000","last_clean":"2024-05-08T23:18:42.797273+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:18:42.797273+0000","last_undegraded":"2024-05-08T23:18:42.797273+0000","last_fullsized":"2024-05-08T23:18:42.797273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792847+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:42.792847+0000","last_peered":"2024-05-08T23:18:42.792847+0000","last_clean":"2024-05-08T23:18:42.792847+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:42.792847+0000","last_undegraded":"2024-05-08T23:18:42.792847+0000","last_fullsized":"2024-05-08T23:18:42.792847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850495+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:18:42.850495+0000","last_peered":"2024-05-08T23:18:42.850495+0000","last_clean":"2024-05-08T23:18:42.850495+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:18:42.850495+0000","last_undegraded":"2024-05-08T23:18:42.850495+0000","last_fullsized":"2024-05-08T23:18:42.850495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803307+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803307+0000","last_peered":"2024-05-08T23:18:43.803307+0000","last_clean":"2024-05-08T23:18:43.803307+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803307+0000","last_undegraded":"2024-05-08T23:18:43.803307+0000","last_fullsized":"2024-05-08T23:18:43.803307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.813513+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:43.813513+0000","last_peered":"2024-05-08T23:18:43.813513+0000","last_clean":"2024-05-08T23:18:43.813513+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:43.813513+0000","last_undegraded":"2024-05-08T23:18:43.813513+0000","last_fullsized":"2024-05-08T23:18:43.813513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800094+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:18:42.800094+0000","last_peered":"2024-05-08T23:18:42.800094+0000","last_clean":"2024-05-08T23:18:42.800094+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:18:42.800094+0000","last_undegraded":"2024-05-08T23:18:42.800094+0000","last_fullsized":"2024-05-08T23:18:42.800094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792881+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:42.792881+0000","last_peered":"2024-05-08T23:18:42.792881+0000","last_clean":"2024-05-08T23:18:42.792881+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:42.792881+0000","last_undegraded":"2024-05-08T23:18:42.792881+0000","last_fullsized":"2024-05-08T23:18:42.792881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797321+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:42.797321+0000","last_peered":"2024-05-08T23:18:42.797321+0000","last_clean":"2024-05-08T23:18:42.797321+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:42.797321+0000","last_undegraded":"2024-05-08T23:18:42.797321+0000","last_fullsized":"2024-05-08T23:18:42.797321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797301+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:18:42.797301+0000","last_peered":"2024-05-08T23:18:42.797301+0000","last_clean":"2024-05-08T23:18:42.797301+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:18:42.797301+0000","last_undegraded":"2024-05-08T23:18:42.797301+0000","last_fullsized":"2024-05-08T23:18:42.797301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798260+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:42.798260+0000","last_peered":"2024-05-08T23:18:42.798260+0000","last_clean":"2024-05-08T23:18:42.798260+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:42.798260+0000","last_undegraded":"2024-05-08T23:18:42.798260+0000","last_fullsized":"2024-05-08T23:18:42.798260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:46.752665+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:46.752665+0000","last_peered":"2024-05-08T23:18:46.752665+0000","last_clean":"2024-05-08T23:18:46.752665+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:46.752665+0000","last_undegraded":"2024-05-08T23:18:46.752665+0000","last_fullsized":"2024-05-08T23:18:46.752665+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796860+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:18:42.796860+0000","last_peered":"2024-05-08T23:18:42.796860+0000","last_clean":"2024-05-08T23:18:42.796860+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:18:42.796860+0000","last_undegraded":"2024-05-08T23:18:42.796860+0000","last_fullsized":"2024-05-08T23:18:42.796860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803558+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803558+0000","last_peered":"2024-05-08T23:18:43.803558+0000","last_clean":"2024-05-08T23:18:43.803558+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803558+0000","last_undegraded":"2024-05-08T23:18:43.803558+0000","last_fullsized":"2024-05-08T23:18:43.803558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796303+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:42.796303+0000","last_peered":"2024-05-08T23:18:42.796303+0000","last_clean":"2024-05-08T23:18:42.796303+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:42.796303+0000","last_undegraded":"2024-05-08T23:18:42.796303+0000","last_fullsized":"2024-05-08T23:18:42.796303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798231+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:18:42.798231+0000","last_peered":"2024-05-08T23:18:42.798231+0000","last_clean":"2024-05-08T23:18:42.798231+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:18:42.798231+0000","last_undegraded":"2024-05-08T23:18:42.798231+0000","last_fullsized":"2024-05-08T23:18:42.798231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801918+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:42.801918+0000","last_peered":"2024-05-08T23:18:42.801918+0000","last_clean":"2024-05-08T23:18:42.801918+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:42.801918+0000","last_undegraded":"2024-05-08T23:18:42.801918+0000","last_fullsized":"2024-05-08T23:18:42.801918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799336+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:42.799336+0000","last_peered":"2024-05-08T23:18:42.799336+0000","last_clean":"2024-05-08T23:18:42.799336+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:42.799336+0000","last_undegraded":"2024-05-08T23:18:42.799336+0000","last_fullsized":"2024-05-08T23:18:42.799336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795875+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:18:42.795875+0000","last_peered":"2024-05-08T23:18:42.795875+0000","last_clean":"2024-05-08T23:18:42.795875+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:18:42.795875+0000","last_undegraded":"2024-05-08T23:18:42.795875+0000","last_fullsized":"2024-05-08T23:18:42.795875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850309+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:42.850309+0000","last_peered":"2024-05-08T23:18:42.850309+0000","last_clean":"2024-05-08T23:18:42.850309+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:42.850309+0000","last_undegraded":"2024-05-08T23:18:42.850309+0000","last_fullsized":"2024-05-08T23:18:42.850309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797842+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:42.797842+0000","last_peered":"2024-05-08T23:18:42.797842+0000","last_clean":"2024-05-08T23:18:42.797842+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:42.797842+0000","last_undegraded":"2024-05-08T23:18:42.797842+0000","last_fullsized":"2024-05-08T23:18:42.797842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792376+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:18:42.792376+0000","last_peered":"2024-05-08T23:18:42.792376+0000","last_clean":"2024-05-08T23:18:42.792376+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:18:42.792376+0000","last_undegraded":"2024-05-08T23:18:42.792376+0000","last_fullsized":"2024-05-08T23:18:42.792376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795976+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:42.795976+0000","last_peered":"2024-05-08T23:18:42.795976+0000","last_clean":"2024-05-08T23:18:42.795976+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:42.795976+0000","last_undegraded":"2024-05-08T23:18:42.795976+0000","last_fullsized":"2024-05-08T23:18:42.795976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.750725+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:45.750725+0000","last_peered":"2024-05-08T23:18:45.750725+0000","last_clean":"2024-05-08T23:18:45.750725+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:45.750725+0000","last_undegraded":"2024-05-08T23:18:45.750725+0000","last_fullsized":"2024-05-08T23:18:45.750725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800494+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:18:42.800494+0000","last_peered":"2024-05-08T23:18:42.800494+0000","last_clean":"2024-05-08T23:18:42.800494+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:18:42.800494+0000","last_undegraded":"2024-05-08T23:18:42.800494+0000","last_fullsized":"2024-05-08T23:18:42.800494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792335+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:42.792335+0000","last_peered":"2024-05-08T23:18:42.792335+0000","last_clean":"2024-05-08T23:18:42.792335+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:42.792335+0000","last_undegraded":"2024-05-08T23:18:42.792335+0000","last_fullsized":"2024-05-08T23:18:42.792335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797044+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:42.797044+0000","last_peered":"2024-05-08T23:18:42.797044+0000","last_clean":"2024-05-08T23:18:42.797044+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:42.797044+0000","last_undegraded":"2024-05-08T23:18:42.797044+0000","last_fullsized":"2024-05-08T23:18:42.797044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850542+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:18:42.850542+0000","last_peered":"2024-05-08T23:18:42.850542+0000","last_clean":"2024-05-08T23:18:42.850542+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:18:42.850542+0000","last_undegraded":"2024-05-08T23:18:42.850542+0000","last_fullsized":"2024-05-08T23:18:42.850542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793231+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:42.793231+0000","last_peered":"2024-05-08T23:18:42.793231+0000","last_clean":"2024-05-08T23:18:42.793231+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:42.793231+0000","last_undegraded":"2024-05-08T23:18:42.793231+0000","last_fullsized":"2024-05-08T23:18:42.793231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797201+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:42.797201+0000","last_peered":"2024-05-08T23:18:42.797201+0000","last_clean":"2024-05-08T23:18:42.797201+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:42.797201+0000","last_undegraded":"2024-05-08T23:18:42.797201+0000","last_fullsized":"2024-05-08T23:18:42.797201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800205+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:18:42.800205+0000","last_peered":"2024-05-08T23:18:42.800205+0000","last_clean":"2024-05-08T23:18:42.800205+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:18:42.800205+0000","last_undegraded":"2024-05-08T23:18:42.800205+0000","last_fullsized":"2024-05-08T23:18:42.800205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792502+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:42.792502+0000","last_peered":"2024-05-08T23:18:42.792502+0000","last_clean":"2024-05-08T23:18:42.792502+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:42.792502+0000","last_undegraded":"2024-05-08T23:18:42.792502+0000","last_fullsized":"2024-05-08T23:18:42.792502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:46.751917+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:46.751917+0000","last_peered":"2024-05-08T23:18:46.751917+0000","last_clean":"2024-05-08T23:18:46.751917+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:46.751917+0000","last_undegraded":"2024-05-08T23:18:46.751917+0000","last_fullsized":"2024-05-08T23:18:46.751917+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61146,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":78,"num_read_kb":99,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797717+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:18:42.797717+0000","last_peered":"2024-05-08T23:18:42.797717+0000","last_clean":"2024-05-08T23:18:42.797717+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:18:42.797717+0000","last_undegraded":"2024-05-08T23:18:42.797717+0000","last_fullsized":"2024-05-08T23:18:42.797717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797660+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:18:42.797660+0000","last_peered":"2024-05-08T23:18:42.797660+0000","last_clean":"2024-05-08T23:18:42.797660+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:18:42.797660+0000","last_undegraded":"2024-05-08T23:18:42.797660+0000","last_fullsized":"2024-05-08T23:18:42.797660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798509+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:42.798509+0000","last_peered":"2024-05-08T23:18:42.798509+0000","last_clean":"2024-05-08T23:18:42.798509+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:42.798509+0000","last_undegraded":"2024-05-08T23:18:42.798509+0000","last_fullsized":"2024-05-08T23:18:42.798509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799523+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:18:42.799523+0000","last_peered":"2024-05-08T23:18:42.799523+0000","last_clean":"2024-05-08T23:18:42.799523+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:18:42.799523+0000","last_undegraded":"2024-05-08T23:18:42.799523+0000","last_fullsized":"2024-05-08T23:18:42.799523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801502+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:42.801502+0000","last_peered":"2024-05-08T23:18:42.801502+0000","last_clean":"2024-05-08T23:18:42.801502+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:42.801502+0000","last_undegraded":"2024-05-08T23:18:42.801502+0000","last_fullsized":"2024-05-08T23:18:42.801502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798399+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:42.798399+0000","last_peered":"2024-05-08T23:18:42.798399+0000","last_clean":"2024-05-08T23:18:42.798399+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:42.798399+0000","last_undegraded":"2024-05-08T23:18:42.798399+0000","last_fullsized":"2024-05-08T23:18:42.798399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034932+0000","last_change":"2024-05-08T23:18:44.034932+0000","last_active":"2024-05-08T23:18:44.034932+0000","last_peered":"2024-05-08T23:18:44.034932+0000","last_clean":"2024-05-08T23:18:44.034932+0000","last_became_active":"2024-05-08T23:18:44.034439+0000","last_became_peered":"2024-05-08T23:18:44.034439+0000","last_unstale":"2024-05-08T23:18:44.034932+0000","last_undegraded":"2024-05-08T23:18:44.034932+0000","last_fullsized":"2024-05-08T23:18:44.034932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.536646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793500+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:42.793500+0000","last_peered":"2024-05-08T23:18:42.793500+0000","last_clean":"2024-05-08T23:18:42.793500+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:42.793500+0000","last_undegraded":"2024-05-08T23:18:42.793500+0000","last_fullsized":"2024-05-08T23:18:42.793500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.749834+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:44.749834+0000","last_peered":"2024-05-08T23:18:44.749834+0000","last_clean":"2024-05-08T23:18:44.749834+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:44.749834+0000","last_undegraded":"2024-05-08T23:18:44.749834+0000","last_fullsized":"2024-05-08T23:18:44.749834+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797290+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:18:42.797290+0000","last_peered":"2024-05-08T23:18:42.797290+0000","last_clean":"2024-05-08T23:18:42.797290+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:18:42.797290+0000","last_undegraded":"2024-05-08T23:18:42.797290+0000","last_fullsized":"2024-05-08T23:18:42.797290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800355+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:42.800355+0000","last_peered":"2024-05-08T23:18:42.800355+0000","last_clean":"2024-05-08T23:18:42.800355+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:42.800355+0000","last_undegraded":"2024-05-08T23:18:42.800355+0000","last_fullsized":"2024-05-08T23:18:42.800355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796252+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:42.796252+0000","last_peered":"2024-05-08T23:18:42.796252+0000","last_clean":"2024-05-08T23:18:42.796252+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:42.796252+0000","last_undegraded":"2024-05-08T23:18:42.796252+0000","last_fullsized":"2024-05-08T23:18:42.796252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.929400+0000","last_change":"2024-05-08T23:18:43.929400+0000","last_active":"2024-05-08T23:18:43.929400+0000","last_peered":"2024-05-08T23:18:43.929400+0000","last_clean":"2024-05-08T23:18:43.929400+0000","last_became_active":"2024-05-08T23:18:43.928980+0000","last_became_peered":"2024-05-08T23:18:43.928980+0000","last_unstale":"2024-05-08T23:18:43.929400+0000","last_undegraded":"2024-05-08T23:18:43.929400+0000","last_fullsized":"2024-05-08T23:18:43.929400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:10.433725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797719+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:18:42.797719+0000","last_peered":"2024-05-08T23:18:42.797719+0000","last_clean":"2024-05-08T23:18:42.797719+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:18:42.797719+0000","last_undegraded":"2024-05-08T23:18:42.797719+0000","last_fullsized":"2024-05-08T23:18:42.797719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798692+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:42.798692+0000","last_peered":"2024-05-08T23:18:42.798692+0000","last_clean":"2024-05-08T23:18:42.798692+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:42.798692+0000","last_undegraded":"2024-05-08T23:18:42.798692+0000","last_fullsized":"2024-05-08T23:18:42.798692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802290+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:18:42.802290+0000","last_peered":"2024-05-08T23:18:42.802290+0000","last_clean":"2024-05-08T23:18:42.802290+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:18:42.802290+0000","last_undegraded":"2024-05-08T23:18:42.802290+0000","last_fullsized":"2024-05-08T23:18:42.802290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796770+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:42.796770+0000","last_peered":"2024-05-08T23:18:42.796770+0000","last_clean":"2024-05-08T23:18:42.796770+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:42.796770+0000","last_undegraded":"2024-05-08T23:18:42.796770+0000","last_fullsized":"2024-05-08T23:18:42.796770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792981+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:42.792981+0000","last_peered":"2024-05-08T23:18:42.792981+0000","last_clean":"2024-05-08T23:18:42.792981+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:42.792981+0000","last_undegraded":"2024-05-08T23:18:42.792981+0000","last_fullsized":"2024-05-08T23:18:42.792981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796821+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:18:42.796821+0000","last_peered":"2024-05-08T23:18:42.796821+0000","last_clean":"2024-05-08T23:18:42.796821+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:18:42.796821+0000","last_undegraded":"2024-05-08T23:18:42.796821+0000","last_fullsized":"2024-05-08T23:18:42.796821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799638+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:42.799638+0000","last_peered":"2024-05-08T23:18:42.799638+0000","last_clean":"2024-05-08T23:18:42.799638+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:42.799638+0000","last_undegraded":"2024-05-08T23:18:42.799638+0000","last_fullsized":"2024-05-08T23:18:42.799638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796478+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:42.796478+0000","last_peered":"2024-05-08T23:18:42.796478+0000","last_clean":"2024-05-08T23:18:42.796478+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:42.796478+0000","last_undegraded":"2024-05-08T23:18:42.796478+0000","last_fullsized":"2024-05-08T23:18:42.796478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796378+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:18:42.796378+0000","last_peered":"2024-05-08T23:18:42.796378+0000","last_clean":"2024-05-08T23:18:42.796378+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:18:42.796378+0000","last_undegraded":"2024-05-08T23:18:42.796378+0000","last_fullsized":"2024-05-08T23:18:42.796378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799747+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:42.799747+0000","last_peered":"2024-05-08T23:18:42.799747+0000","last_clean":"2024-05-08T23:18:42.799747+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:42.799747+0000","last_undegraded":"2024-05-08T23:18:42.799747+0000","last_fullsized":"2024-05-08T23:18:42.799747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797943+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:18:42.797943+0000","last_peered":"2024-05-08T23:18:42.797943+0000","last_clean":"2024-05-08T23:18:42.797943+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:18:42.797943+0000","last_undegraded":"2024-05-08T23:18:42.797943+0000","last_fullsized":"2024-05-08T23:18:42.797943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.750437+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:45.750437+0000","last_peered":"2024-05-08T23:18:45.750437+0000","last_clean":"2024-05-08T23:18:45.750437+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:45.750437+0000","last_undegraded":"2024-05-08T23:18:45.750437+0000","last_fullsized":"2024-05-08T23:18:45.750437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796463+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:42.796463+0000","last_peered":"2024-05-08T23:18:42.796463+0000","last_clean":"2024-05-08T23:18:42.796463+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:42.796463+0000","last_undegraded":"2024-05-08T23:18:42.796463+0000","last_fullsized":"2024-05-08T23:18:42.796463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796295+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:18:42.796295+0000","last_peered":"2024-05-08T23:18:42.796295+0000","last_clean":"2024-05-08T23:18:42.796295+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:18:42.796295+0000","last_undegraded":"2024-05-08T23:18:42.796295+0000","last_fullsized":"2024-05-08T23:18:42.796295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793103+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:42.793103+0000","last_peered":"2024-05-08T23:18:42.793103+0000","last_clean":"2024-05-08T23:18:42.793103+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:42.793103+0000","last_undegraded":"2024-05-08T23:18:42.793103+0000","last_fullsized":"2024-05-08T23:18:42.793103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796517+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:42.796517+0000","last_peered":"2024-05-08T23:18:42.796517+0000","last_clean":"2024-05-08T23:18:42.796517+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:42.796517+0000","last_undegraded":"2024-05-08T23:18:42.796517+0000","last_fullsized":"2024-05-08T23:18:42.796517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796867+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:18:42.796867+0000","last_peered":"2024-05-08T23:18:42.796867+0000","last_clean":"2024-05-08T23:18:42.796867+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:18:42.796867+0000","last_undegraded":"2024-05-08T23:18:42.796867+0000","last_fullsized":"2024-05-08T23:18:42.796867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799236+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:18:42.799236+0000","last_peered":"2024-05-08T23:18:42.799236+0000","last_clean":"2024-05-08T23:18:42.799236+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:18:42.799236+0000","last_undegraded":"2024-05-08T23:18:42.799236+0000","last_fullsized":"2024-05-08T23:18:42.799236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796699+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:42.796699+0000","last_peered":"2024-05-08T23:18:42.796699+0000","last_clean":"2024-05-08T23:18:42.796699+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:42.796699+0000","last_undegraded":"2024-05-08T23:18:42.796699+0000","last_fullsized":"2024-05-08T23:18:42.796699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802361+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:42.802361+0000","last_peered":"2024-05-08T23:18:42.802361+0000","last_clean":"2024-05-08T23:18:42.802361+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:42.802361+0000","last_undegraded":"2024-05-08T23:18:42.802361+0000","last_fullsized":"2024-05-08T23:18:42.802361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797799+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:42.797799+0000","last_peered":"2024-05-08T23:18:42.797799+0000","last_clean":"2024-05-08T23:18:42.797799+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:42.797799+0000","last_undegraded":"2024-05-08T23:18:42.797799+0000","last_fullsized":"2024-05-08T23:18:42.797799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799837+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:18:42.799837+0000","last_peered":"2024-05-08T23:18:42.799837+0000","last_clean":"2024-05-08T23:18:42.799837+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:18:42.799837+0000","last_undegraded":"2024-05-08T23:18:42.799837+0000","last_fullsized":"2024-05-08T23:18:42.799837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795769+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:42.795769+0000","last_peered":"2024-05-08T23:18:42.795769+0000","last_clean":"2024-05-08T23:18:42.795769+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:42.795769+0000","last_undegraded":"2024-05-08T23:18:42.795769+0000","last_fullsized":"2024-05-08T23:18:42.795769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793582+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:18:42.793582+0000","last_peered":"2024-05-08T23:18:42.793582+0000","last_clean":"2024-05-08T23:18:42.793582+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:18:42.793582+0000","last_undegraded":"2024-05-08T23:18:42.793582+0000","last_fullsized":"2024-05-08T23:18:42.793582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797388+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:18:42.797388+0000","last_peered":"2024-05-08T23:18:42.797388+0000","last_clean":"2024-05-08T23:18:42.797388+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:18:42.797388+0000","last_undegraded":"2024-05-08T23:18:42.797388+0000","last_fullsized":"2024-05-08T23:18:42.797388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797897+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:42.797897+0000","last_peered":"2024-05-08T23:18:42.797897+0000","last_clean":"2024-05-08T23:18:42.797897+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:42.797897+0000","last_undegraded":"2024-05-08T23:18:42.797897+0000","last_fullsized":"2024-05-08T23:18:42.797897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793553+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:42.793553+0000","last_peered":"2024-05-08T23:18:42.793553+0000","last_clean":"2024-05-08T23:18:42.793553+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:42.793553+0000","last_undegraded":"2024-05-08T23:18:42.793553+0000","last_fullsized":"2024-05-08T23:18:42.793553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850431+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:18:42.850431+0000","last_peered":"2024-05-08T23:18:42.850431+0000","last_clean":"2024-05-08T23:18:42.850431+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:18:42.850431+0000","last_undegraded":"2024-05-08T23:18:42.850431+0000","last_fullsized":"2024-05-08T23:18:42.850431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793223+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:42.793223+0000","last_peered":"2024-05-08T23:18:42.793223+0000","last_clean":"2024-05-08T23:18:42.793223+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:42.793223+0000","last_undegraded":"2024-05-08T23:18:42.793223+0000","last_fullsized":"2024-05-08T23:18:42.793223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799949+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:42.799949+0000","last_peered":"2024-05-08T23:18:42.799949+0000","last_clean":"2024-05-08T23:18:42.799949+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:42.799949+0000","last_undegraded":"2024-05-08T23:18:42.799949+0000","last_fullsized":"2024-05-08T23:18:42.799949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796359+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:18:42.796359+0000","last_peered":"2024-05-08T23:18:42.796359+0000","last_clean":"2024-05-08T23:18:42.796359+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:18:42.796359+0000","last_undegraded":"2024-05-08T23:18:42.796359+0000","last_fullsized":"2024-05-08T23:18:42.796359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793663+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:42.793663+0000","last_peered":"2024-05-08T23:18:42.793663+0000","last_clean":"2024-05-08T23:18:42.793663+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:42.793663+0000","last_undegraded":"2024-05-08T23:18:42.793663+0000","last_fullsized":"2024-05-08T23:18:42.793663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796095+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:42.796095+0000","last_peered":"2024-05-08T23:18:42.796095+0000","last_clean":"2024-05-08T23:18:42.796095+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:42.796095+0000","last_undegraded":"2024-05-08T23:18:42.796095+0000","last_fullsized":"2024-05-08T23:18:42.796095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801296+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:18:42.801296+0000","last_peered":"2024-05-08T23:18:42.801296+0000","last_clean":"2024-05-08T23:18:42.801296+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:18:42.801296+0000","last_undegraded":"2024-05-08T23:18:42.801296+0000","last_fullsized":"2024-05-08T23:18:42.801296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792930+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:42.792930+0000","last_peered":"2024-05-08T23:18:42.792930+0000","last_clean":"2024-05-08T23:18:42.792930+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:42.792930+0000","last_undegraded":"2024-05-08T23:18:42.792930+0000","last_fullsized":"2024-05-08T23:18:42.792930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799130+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:42.799130+0000","last_peered":"2024-05-08T23:18:42.799130+0000","last_clean":"2024-05-08T23:18:42.799130+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:42.799130+0000","last_undegraded":"2024-05-08T23:18:42.799130+0000","last_fullsized":"2024-05-08T23:18:42.799130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796367+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:18:42.796367+0000","last_peered":"2024-05-08T23:18:42.796367+0000","last_clean":"2024-05-08T23:18:42.796367+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:18:42.796367+0000","last_undegraded":"2024-05-08T23:18:42.796367+0000","last_fullsized":"2024-05-08T23:18:42.796367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793402+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:42.793402+0000","last_peered":"2024-05-08T23:18:42.793402+0000","last_clean":"2024-05-08T23:18:42.793402+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:42.793402+0000","last_undegraded":"2024-05-08T23:18:42.793402+0000","last_fullsized":"2024-05-08T23:18:42.793402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.808122+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:43.808122+0000","last_peered":"2024-05-08T23:18:43.808122+0000","last_clean":"2024-05-08T23:18:43.808122+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:43.808122+0000","last_undegraded":"2024-05-08T23:18:43.808122+0000","last_fullsized":"2024-05-08T23:18:43.808122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796533+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:18:42.796533+0000","last_peered":"2024-05-08T23:18:42.796533+0000","last_clean":"2024-05-08T23:18:42.796533+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:18:42.796533+0000","last_undegraded":"2024-05-08T23:18:42.796533+0000","last_fullsized":"2024-05-08T23:18:42.796533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800112+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:42.800112+0000","last_peered":"2024-05-08T23:18:42.800112+0000","last_clean":"2024-05-08T23:18:42.800112+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:42.800112+0000","last_undegraded":"2024-05-08T23:18:42.800112+0000","last_fullsized":"2024-05-08T23:18:42.800112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800360+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:42.800360+0000","last_peered":"2024-05-08T23:18:42.800360+0000","last_clean":"2024-05-08T23:18:42.800360+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:42.800360+0000","last_undegraded":"2024-05-08T23:18:42.800360+0000","last_fullsized":"2024-05-08T23:18:42.800360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797464+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:18:42.797464+0000","last_peered":"2024-05-08T23:18:42.797464+0000","last_clean":"2024-05-08T23:18:42.797464+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:18:42.797464+0000","last_undegraded":"2024-05-08T23:18:42.797464+0000","last_fullsized":"2024-05-08T23:18:42.797464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849917+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:42.849917+0000","last_peered":"2024-05-08T23:18:42.849917+0000","last_clean":"2024-05-08T23:18:42.849917+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:42.849917+0000","last_undegraded":"2024-05-08T23:18:42.849917+0000","last_fullsized":"2024-05-08T23:18:42.849917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797180+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:18:42.797180+0000","last_peered":"2024-05-08T23:18:42.797180+0000","last_clean":"2024-05-08T23:18:42.797180+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:18:42.797180+0000","last_undegraded":"2024-05-08T23:18:42.797180+0000","last_fullsized":"2024-05-08T23:18:42.797180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796081+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:18:42.796081+0000","last_peered":"2024-05-08T23:18:42.796081+0000","last_clean":"2024-05-08T23:18:42.796081+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:18:42.796081+0000","last_undegraded":"2024-05-08T23:18:42.796081+0000","last_fullsized":"2024-05-08T23:18:42.796081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931402+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931402+0000","last_peered":"2024-05-08T23:18:43.931402+0000","last_clean":"2024-05-08T23:18:43.931402+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931402+0000","last_undegraded":"2024-05-08T23:18:43.931402+0000","last_fullsized":"2024-05-08T23:18:43.931402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796911+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:18:42.796911+0000","last_peered":"2024-05-08T23:18:42.796911+0000","last_clean":"2024-05-08T23:18:42.796911+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:18:42.796911+0000","last_undegraded":"2024-05-08T23:18:42.796911+0000","last_fullsized":"2024-05-08T23:18:42.796911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801392+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:18:42.801392+0000","last_peered":"2024-05-08T23:18:42.801392+0000","last_clean":"2024-05-08T23:18:42.801392+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:18:42.801392+0000","last_undegraded":"2024-05-08T23:18:42.801392+0000","last_fullsized":"2024-05-08T23:18:42.801392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034679+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:18:44.034679+0000","last_peered":"2024-05-08T23:18:44.034679+0000","last_clean":"2024-05-08T23:18:44.034679+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:18:44.034679+0000","last_undegraded":"2024-05-08T23:18:44.034679+0000","last_fullsized":"2024-05-08T23:18:44.034679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799239+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:18:42.799239+0000","last_peered":"2024-05-08T23:18:42.799239+0000","last_clean":"2024-05-08T23:18:42.799239+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:18:42.799239+0000","last_undegraded":"2024-05-08T23:18:42.799239+0000","last_fullsized":"2024-05-08T23:18:42.799239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850099+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:18:42.850099+0000","last_peered":"2024-05-08T23:18:42.850099+0000","last_clean":"2024-05-08T23:18:42.850099+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:18:42.850099+0000","last_undegraded":"2024-05-08T23:18:42.850099+0000","last_fullsized":"2024-05-08T23:18:42.850099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799941+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:18:42.799941+0000","last_peered":"2024-05-08T23:18:42.799941+0000","last_clean":"2024-05-08T23:18:42.799941+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:18:42.799941+0000","last_undegraded":"2024-05-08T23:18:42.799941+0000","last_fullsized":"2024-05-08T23:18:42.799941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.803347+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:18:42.803347+0000","last_peered":"2024-05-08T23:18:42.803347+0000","last_clean":"2024-05-08T23:18:42.803347+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:18:42.803347+0000","last_undegraded":"2024-05-08T23:18:42.803347+0000","last_fullsized":"2024-05-08T23:18:42.803347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800760+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:18:42.800760+0000","last_peered":"2024-05-08T23:18:42.800760+0000","last_clean":"2024-05-08T23:18:42.800760+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:18:42.800760+0000","last_undegraded":"2024-05-08T23:18:42.800760+0000","last_fullsized":"2024-05-08T23:18:42.800760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797474+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:18:42.797474+0000","last_peered":"2024-05-08T23:18:42.797474+0000","last_clean":"2024-05-08T23:18:42.797474+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:18:42.797474+0000","last_undegraded":"2024-05-08T23:18:42.797474+0000","last_fullsized":"2024-05-08T23:18:42.797474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797850+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:18:42.797850+0000","last_peered":"2024-05-08T23:18:42.797850+0000","last_clean":"2024-05-08T23:18:42.797850+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:18:42.797850+0000","last_undegraded":"2024-05-08T23:18:42.797850+0000","last_fullsized":"2024-05-08T23:18:42.797850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796074+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:18:42.796074+0000","last_peered":"2024-05-08T23:18:42.796074+0000","last_clean":"2024-05-08T23:18:42.796074+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:18:42.796074+0000","last_undegraded":"2024-05-08T23:18:42.796074+0000","last_fullsized":"2024-05-08T23:18:42.796074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800329+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:18:42.800329+0000","last_peered":"2024-05-08T23:18:42.800329+0000","last_clean":"2024-05-08T23:18:42.800329+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:18:42.800329+0000","last_undegraded":"2024-05-08T23:18:42.800329+0000","last_fullsized":"2024-05-08T23:18:42.800329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800055+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:18:42.800055+0000","last_peered":"2024-05-08T23:18:42.800055+0000","last_clean":"2024-05-08T23:18:42.800055+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:18:42.800055+0000","last_undegraded":"2024-05-08T23:18:42.800055+0000","last_fullsized":"2024-05-08T23:18:42.800055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801597+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:18:42.801597+0000","last_peered":"2024-05-08T23:18:42.801597+0000","last_clean":"2024-05-08T23:18:42.801597+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:18:42.801597+0000","last_undegraded":"2024-05-08T23:18:42.801597+0000","last_fullsized":"2024-05-08T23:18:42.801597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":234,"num_read_kb":185,"num_write":156,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645335,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":463111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.439}]},{"osd":5,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52200000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.496},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.56599999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":9,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78600000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65500000000000003}]}]},{"osd":10,"up_from":65,"seq":279172874267,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1617920,"data_stored":1111146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.36399999999999999}]},{"osd":5,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.503},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.52600000000000002}]},{"osd":9,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.625}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.64600000000000002}]}]},{"osd":9,"up_from":60,"seq":257698037790,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":467443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.498},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.34699999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.67500000000000004}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.53900000000000003}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70899999999999996}]},{"osd":10,"last update":"Wed May 8 23:17:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.65000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":8,"up_from":53,"seq":227633266722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86499999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.39600000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":1.137,"15min":1.137},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92200000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.754},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93500000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81299999999999994}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78500000000000003}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]}]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1540096,"data_stored":1058358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.626},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65400000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40600000000000003}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59099999999999997}]},{"osd":8,"last update":"Wed May 8 23:18:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.68600000000000005}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60199999999999998}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.46600000000000003}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.74299999999999999}]}]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":458523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.27900000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.57299999999999995,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":1.022},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.435,"15min":0.433},"min":{"1min":0.318,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68400000000000005}]},{"osd":1,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.39500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85299999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91600000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.85099999999999998}]},{"osd":3,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.40000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69999999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.443},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.53800000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.92700000000000005}]},{"osd":5,"last update":"Wed May 8 23:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.131},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.68600000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71999999999999997}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0049999999999999,"5min":1.177,"15min":1.177},"last":0.79300000000000004}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.046,"5min":1.091,"15min":1.091},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.81799999999999995}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.504}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":995328,"data_stored":519579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.41699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.39000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69199999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.371}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.46800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.58299999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.621,"15min":0.59299999999999997},"min":{"1min":0.54700000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68500000000000005,"15min":0.67000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71599999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66800000000000004}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":954368,"data_stored":459911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed May 8 23:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48699999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65700000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84399999999999997,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Wed May 8 23:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.624}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57399999999999995}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.69999999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.86099999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444087,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":520881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.60799999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.76500000000000001}]},{"osd":4,"last update":"Wed May 8 23:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.87},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.437,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70299999999999996,"5min":0.872,"15min":0.872},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.753,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed May 8 23:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.60299999999999998,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.90100000000000002}]},{"osd":7,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.97499999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":1.024}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":1.0429999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":1.0089999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":3,"up_from":23,"seq":98784247859,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":942080,"data_stored":458717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.501},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.745}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed May 8 23:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.624,"15min":0.58799999999999997},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91900000000000004,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.55700000000000005}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.625},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.80300000000000005,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.83999999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.93100000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.87}]}]},{"osd":4,"up_from":30,"seq":128849018927,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.29299999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.373},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.434,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.46500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75800000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.51100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51300000000000001}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83399999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46100000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65200000000000002}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.625},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66800000000000004}]}]},{"osd":5,"up_from":35,"seq":150323855404,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1552384,"data_stored":1054325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.504},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45700000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.52500000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.754,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.53600000000000003}]},{"osd":6,"last update":"Wed May 8 23:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.63100000000000001,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.753,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71299999999999997}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.72499999999999998}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.85499999999999998}]},{"osd":10,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:18:52.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph status --format=json 2024-05-08T23:18:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:54.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:53 smithi016 ceph-mon[42541]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:54.027 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:53 smithi016 ceph-mon[42541]: pgmap v258: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:54.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:53 smithi189 ceph-mon[46919]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:53 smithi189 ceph-mon[46919]: pgmap v258: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:53 smithi155 ceph-mon[47105]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:53 smithi155 ceph-mon[47105]: pgmap v258: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 -- 172.21.15.16:0/967380418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538107020 msgr2=0x7f7538107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 --2- 172.21.15.16:0/967380418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538107020 0x7f7538107400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 -- 172.21.15.16:0/967380418 shutdown_connections 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 --2- 172.21.15.16:0/967380418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75380692f0 0x7f753810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 --2- 172.21.15.16:0/967380418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538107020 0x7f7538107400 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 --2- 172.21.15.16:0/967380418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7538068950 0x7f7538068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 -- 172.21.15.16:0/967380418 >> 172.21.15.16:0/967380418 conn(0x7f7538075f80 msgr2=0x7f7538076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 -- 172.21.15.16:0/967380418 shutdown_connections 2024-05-08T23:18:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.224+0000 7f753e452700 1 -- 172.21.15.16:0/967380418 wait complete. 2024-05-08T23:18:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.225+0000 7f753e452700 1 Processor -- start 2024-05-08T23:18:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.225+0000 7f753e452700 1 -- start start 2024-05-08T23:18:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.225+0000 7f753e452700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 0x7f75381972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f753e452700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75380692f0 0x7f7538197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f753e452700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7538107020 0x7f753819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f753e452700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753819c1c0 con 0x7f7538068950 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f753e452700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753819c330 con 0x7f7538107020 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f753e452700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753819c4f0 con 0x7f75380692f0 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 0x7f75381972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 0x7f75381972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50358/0 (socket says 172.21.15.16:50358) 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 -- 172.21.15.16:0/485734587 learned_addr learned my addr 172.21.15.16:0/485734587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f75377fe700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75380692f0 0x7f7538197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f753c9ef700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7538107020 0x7f753819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 -- 172.21.15.16:0/485734587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75380692f0 msgr2=0x7f7538197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75380692f0 0x7f7538197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 -- 172.21.15.16:0/485734587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7538107020 msgr2=0x7f753819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7538107020 0x7f753819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 -- 172.21.15.16:0/485734587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280095f0 con 0x7f7538068950 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f7537fff700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 0x7f75381972c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7528006010 tx=0x7f7528004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752801d070 con 0x7f7538068950 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.226+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f752800fc00 con 0x7f7538068950 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.227+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753819c770 con 0x7f7538068950 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.227+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528022b90 con 0x7f7538068950 2024-05-08T23:18:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.227+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75381a4030 con 0x7f7538068950 2024-05-08T23:18:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.228+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7538109540 con 0x7f7538068950 2024-05-08T23:18:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.229+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f752800f770 con 0x7f7538068950 2024-05-08T23:18:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.230+0000 7f75357fa700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f75240a11b0 0x7f75240a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.230+0000 7f75377fe700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f75240a11b0 0x7f75240a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.230+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f75280ce920 con 0x7f7538068950 2024-05-08T23:18:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.230+0000 7f75377fe700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f75240a11b0 0x7f75240a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7538198820 tx=0x7f75200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:54.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.234+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7528097ce0 con 0x7f7538068950 2024-05-08T23:18:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.537+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f753804ec60 con 0x7f7538068950 2024-05-08T23:18:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.537+0000 7f75357fa700 1 -- 172.21.15.16:0/485734587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2311 (secure 0 0 0) 0x7f752809b300 con 0x7f7538068950 2024-05-08T23:18:54.538 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:54.538 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":325,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1715210230,"num_in_osds":12,"osd_in_since":1715210212,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":655262,"bytes_used":404078592,"bytes_avail":1151536349184,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":1,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"read_bytes_sec":1877,"read_op_per_sec":4,"write_op_per_sec":0},"fsmap":{"epoch":24,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24497},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24517},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24521},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24529},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24550},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24563}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.16:8443/"}},"servicemap":{"epoch":24,"modified":"2024-05-08T23:18:50.605500+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T23:18:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.539+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f75240a11b0 msgr2=0x7f75240a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.539+0000 7f753e452700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f75240a11b0 0x7f75240a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7538198820 tx=0x7f75200073d0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.539+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 msgr2=0x7f75381972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.539+0000 7f753e452700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 0x7f75381972c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7528006010 tx=0x7f7528004730 comp rx=0 tx=0).stop 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 shutdown_connections 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7538107020 0x7f753819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7538068950 0x7f75381972c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f75240a11b0 0x7f75240a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 --2- 172.21.15.16:0/485734587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75380692f0 0x7f7538197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 >> 172.21.15.16:0/485734587 conn(0x7f7538075f80 msgr2=0x7f75380fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 shutdown_connections 2024-05-08T23:18:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:54.540+0000 7f753e452700 1 -- 172.21.15.16:0/485734587 wait complete. 2024-05-08T23:18:54.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:54 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/485734587' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T23:18:54.943 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-05-08T23:18:54.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:18:55.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:54 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/485734587' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T23:18:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:18:55.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:54 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/485734587' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-05-08T23:18:56.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:55 smithi016 ceph-mon[42541]: pgmap v259: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:18:56.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:55 smithi189 ceph-mon[46919]: pgmap v259: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:18:56.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:55 smithi155 ceph-mon[47105]: pgmap v259: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2062185884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f8107020 msgr2=0x7f88f8107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2062185884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f8107020 0x7f88f8107400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009990 tx=0x7f88e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2062185884 shutdown_connections 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2062185884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f80692f0 0x7f88f810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2062185884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f8107020 0x7f88f8107400 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2062185884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88f8068950 0x7f88f8068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2062185884 >> 172.21.15.16:0/2062185884 conn(0x7f88f8075f80 msgr2=0x7f88f8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2062185884 shutdown_connections 2024-05-08T23:18:56.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.248+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2062185884 wait complete. 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 Processor -- start 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 -- start start 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f8068950 0x7f88f80fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 0x7f88f81004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88f8107020 0x7f88f8100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f81a5780 con 0x7f88f80692f0 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f81a58f0 con 0x7f88f8107020 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88fcdc9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f81a5ab0 con 0x7f88f8068950 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 0x7f88f81004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 0x7f88f81004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50370/0 (socket says 172.21.15.16:50370) 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88f5d9b700 1 -- 172.21.15.16:0/2183200955 learned_addr learned my addr 172.21.15.16:0/2183200955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88f659c700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f8068950 0x7f88f80fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.249+0000 7f88f6d9d700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88f8107020 0x7f88f8100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88f5d9b700 1 -- 172.21.15.16:0/2183200955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f8068950 msgr2=0x7f88f80fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88f5d9b700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f8068950 0x7f88f80fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88f5d9b700 1 -- 172.21.15.16:0/2183200955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88f8107020 msgr2=0x7f88f8100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88f5d9b700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88f8107020 0x7f88f8100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88f5d9b700 1 -- 172.21.15.16:0/2183200955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80095f0 con 0x7f88f80692f0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88f5d9b700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 0x7f88f81004e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d950 tx=0x7f88ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00f890 con 0x7f88f80692f0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88ec00fe80 con 0x7f88f80692f0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f81a5d90 con 0x7f88f80692f0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00e3b0 con 0x7f88f80692f0 2024-05-08T23:18:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.250+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f81a6230 con 0x7f88f80692f0 2024-05-08T23:18:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.251+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f8109540 con 0x7f88f80692f0 2024-05-08T23:18:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.252+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f88ec010460 con 0x7f88f80692f0 2024-05-08T23:18:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.253+0000 7f88e77fe700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f88e00a11b0 0x7f88e00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:18:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.253+0000 7f88f659c700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f88e00a11b0 0x7f88e00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:18:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.253+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f88ec0ccef0 con 0x7f88f80692f0 2024-05-08T23:18:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.254+0000 7f88f659c700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f88e00a11b0 0x7f88e00a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88e800b5c0 tx=0x7f88e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:18:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.258+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f88ec0961c0 con 0x7f88f80692f0 2024-05-08T23:18:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.468+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88f81014f0 con 0x7f88e00a11b0 2024-05-08T23:18:56.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.481+0000 7f88e77fe700 1 -- 172.21.15.16:0/2183200955 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426537 (secure 0 0 0) 0x7f88f81014f0 con 0x7f88e00a11b0 2024-05-08T23:18:56.488 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:18:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f88e00a11b0 msgr2=0x7f88e00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f88e00a11b0 0x7f88e00a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88e800b5c0 tx=0x7f88e8005fb0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 msgr2=0x7f88f81004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:18:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 0x7f88f81004e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d950 tx=0x7f88ec00dc60 comp rx=0 tx=0).stop 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 shutdown_connections 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88f8107020 0x7f88f8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f88e00a11b0 0x7f88e00a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f8068950 0x7f88f80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 --2- 172.21.15.16:0/2183200955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f80692f0 0x7f88f81004e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 >> 172.21.15.16:0/2183200955 conn(0x7f88f8075f80 msgr2=0x7f88f80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 shutdown_connections 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:18:56.486+0000 7f88fcdc9700 1 -- 172.21.15.16:0/2183200955 wait complete. 2024-05-08T23:18:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:18:56.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-05-08T23:18:54.606533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":400,"num_read_kb":319,"num_write":339,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394608,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":381421,"kb_avail":1124547216,"statfs":{"total":1151940427776,"available":1151536349184,"internally_reserved":0,"allocated":13332480,"data_stored":7498386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":390575432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.814484"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802073+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:18:42.802073+0000","last_peered":"2024-05-08T23:18:42.802073+0000","last_clean":"2024-05-08T23:18:42.802073+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:18:42.802073+0000","last_undegraded":"2024-05-08T23:18:42.802073+0000","last_fullsized":"2024-05-08T23:18:42.802073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850382+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:18:42.850382+0000","last_peered":"2024-05-08T23:18:42.850382+0000","last_clean":"2024-05-08T23:18:42.850382+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:18:42.850382+0000","last_undegraded":"2024-05-08T23:18:42.850382+0000","last_fullsized":"2024-05-08T23:18:42.850382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850219+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:18:42.850219+0000","last_peered":"2024-05-08T23:18:42.850219+0000","last_clean":"2024-05-08T23:18:42.850219+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:18:42.850219+0000","last_undegraded":"2024-05-08T23:18:42.850219+0000","last_fullsized":"2024-05-08T23:18:42.850219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793298+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:18:42.793298+0000","last_peered":"2024-05-08T23:18:42.793298+0000","last_clean":"2024-05-08T23:18:42.793298+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:18:42.793298+0000","last_undegraded":"2024-05-08T23:18:42.793298+0000","last_fullsized":"2024-05-08T23:18:42.793298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799491+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:18:42.799491+0000","last_peered":"2024-05-08T23:18:42.799491+0000","last_clean":"2024-05-08T23:18:42.799491+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:18:42.799491+0000","last_undegraded":"2024-05-08T23:18:42.799491+0000","last_fullsized":"2024-05-08T23:18:42.799491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796401+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:18:42.796401+0000","last_peered":"2024-05-08T23:18:42.796401+0000","last_clean":"2024-05-08T23:18:42.796401+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:18:42.796401+0000","last_undegraded":"2024-05-08T23:18:42.796401+0000","last_fullsized":"2024-05-08T23:18:42.796401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797424+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:18:42.797424+0000","last_peered":"2024-05-08T23:18:42.797424+0000","last_clean":"2024-05-08T23:18:42.797424+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:18:42.797424+0000","last_undegraded":"2024-05-08T23:18:42.797424+0000","last_fullsized":"2024-05-08T23:18:42.797424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795761+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:18:42.795761+0000","last_peered":"2024-05-08T23:18:42.795761+0000","last_clean":"2024-05-08T23:18:42.795761+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:18:42.795761+0000","last_undegraded":"2024-05-08T23:18:42.795761+0000","last_fullsized":"2024-05-08T23:18:42.795761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800861+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:18:42.800861+0000","last_peered":"2024-05-08T23:18:42.800861+0000","last_clean":"2024-05-08T23:18:42.800861+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:18:42.800861+0000","last_undegraded":"2024-05-08T23:18:42.800861+0000","last_fullsized":"2024-05-08T23:18:42.800861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034666+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:18:44.034666+0000","last_peered":"2024-05-08T23:18:44.034666+0000","last_clean":"2024-05-08T23:18:44.034666+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:18:44.034666+0000","last_undegraded":"2024-05-08T23:18:44.034666+0000","last_fullsized":"2024-05-08T23:18:44.034666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797414+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:18:42.797414+0000","last_peered":"2024-05-08T23:18:42.797414+0000","last_clean":"2024-05-08T23:18:42.797414+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:18:42.797414+0000","last_undegraded":"2024-05-08T23:18:42.797414+0000","last_fullsized":"2024-05-08T23:18:42.797414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793543+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:18:42.793543+0000","last_peered":"2024-05-08T23:18:42.793543+0000","last_clean":"2024-05-08T23:18:42.793543+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:18:42.793543+0000","last_undegraded":"2024-05-08T23:18:42.793543+0000","last_fullsized":"2024-05-08T23:18:42.793543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800158+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:18:42.800158+0000","last_peered":"2024-05-08T23:18:42.800158+0000","last_clean":"2024-05-08T23:18:42.800158+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:18:42.800158+0000","last_undegraded":"2024-05-08T23:18:42.800158+0000","last_fullsized":"2024-05-08T23:18:42.800158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849931+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:18:42.849931+0000","last_peered":"2024-05-08T23:18:42.849931+0000","last_clean":"2024-05-08T23:18:42.849931+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:18:42.849931+0000","last_undegraded":"2024-05-08T23:18:42.849931+0000","last_fullsized":"2024-05-08T23:18:42.849931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849831+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:18:42.849831+0000","last_peered":"2024-05-08T23:18:42.849831+0000","last_clean":"2024-05-08T23:18:42.849831+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:18:42.849831+0000","last_undegraded":"2024-05-08T23:18:42.849831+0000","last_fullsized":"2024-05-08T23:18:42.849831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799862+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:18:42.799862+0000","last_peered":"2024-05-08T23:18:42.799862+0000","last_clean":"2024-05-08T23:18:42.799862+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.799862+0000","last_undegraded":"2024-05-08T23:18:42.799862+0000","last_fullsized":"2024-05-08T23:18:42.799862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801991+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:18:42.801991+0000","last_peered":"2024-05-08T23:18:42.801991+0000","last_clean":"2024-05-08T23:18:42.801991+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:18:42.801991+0000","last_undegraded":"2024-05-08T23:18:42.801991+0000","last_fullsized":"2024-05-08T23:18:42.801991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793563+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:18:42.793563+0000","last_peered":"2024-05-08T23:18:42.793563+0000","last_clean":"2024-05-08T23:18:42.793563+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:18:42.793563+0000","last_undegraded":"2024-05-08T23:18:42.793563+0000","last_fullsized":"2024-05-08T23:18:42.793563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799013+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:18:42.799013+0000","last_peered":"2024-05-08T23:18:42.799013+0000","last_clean":"2024-05-08T23:18:42.799013+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:18:42.799013+0000","last_undegraded":"2024-05-08T23:18:42.799013+0000","last_fullsized":"2024-05-08T23:18:42.799013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800896+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:42.800896+0000","last_peered":"2024-05-08T23:18:42.800896+0000","last_clean":"2024-05-08T23:18:42.800896+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:42.800896+0000","last_undegraded":"2024-05-08T23:18:42.800896+0000","last_fullsized":"2024-05-08T23:18:42.800896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793673+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:42.793673+0000","last_peered":"2024-05-08T23:18:42.793673+0000","last_clean":"2024-05-08T23:18:42.793673+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:42.793673+0000","last_undegraded":"2024-05-08T23:18:42.793673+0000","last_fullsized":"2024-05-08T23:18:42.793673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797273+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:18:42.797273+0000","last_peered":"2024-05-08T23:18:42.797273+0000","last_clean":"2024-05-08T23:18:42.797273+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:18:42.797273+0000","last_undegraded":"2024-05-08T23:18:42.797273+0000","last_fullsized":"2024-05-08T23:18:42.797273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792847+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:42.792847+0000","last_peered":"2024-05-08T23:18:42.792847+0000","last_clean":"2024-05-08T23:18:42.792847+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:42.792847+0000","last_undegraded":"2024-05-08T23:18:42.792847+0000","last_fullsized":"2024-05-08T23:18:42.792847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850495+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:18:42.850495+0000","last_peered":"2024-05-08T23:18:42.850495+0000","last_clean":"2024-05-08T23:18:42.850495+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:18:42.850495+0000","last_undegraded":"2024-05-08T23:18:42.850495+0000","last_fullsized":"2024-05-08T23:18:42.850495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803307+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803307+0000","last_peered":"2024-05-08T23:18:43.803307+0000","last_clean":"2024-05-08T23:18:43.803307+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803307+0000","last_undegraded":"2024-05-08T23:18:43.803307+0000","last_fullsized":"2024-05-08T23:18:43.803307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.813513+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:43.813513+0000","last_peered":"2024-05-08T23:18:43.813513+0000","last_clean":"2024-05-08T23:18:43.813513+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:43.813513+0000","last_undegraded":"2024-05-08T23:18:43.813513+0000","last_fullsized":"2024-05-08T23:18:43.813513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800094+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:18:42.800094+0000","last_peered":"2024-05-08T23:18:42.800094+0000","last_clean":"2024-05-08T23:18:42.800094+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:18:42.800094+0000","last_undegraded":"2024-05-08T23:18:42.800094+0000","last_fullsized":"2024-05-08T23:18:42.800094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792881+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:42.792881+0000","last_peered":"2024-05-08T23:18:42.792881+0000","last_clean":"2024-05-08T23:18:42.792881+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:42.792881+0000","last_undegraded":"2024-05-08T23:18:42.792881+0000","last_fullsized":"2024-05-08T23:18:42.792881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797321+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:42.797321+0000","last_peered":"2024-05-08T23:18:42.797321+0000","last_clean":"2024-05-08T23:18:42.797321+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:42.797321+0000","last_undegraded":"2024-05-08T23:18:42.797321+0000","last_fullsized":"2024-05-08T23:18:42.797321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797301+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:18:42.797301+0000","last_peered":"2024-05-08T23:18:42.797301+0000","last_clean":"2024-05-08T23:18:42.797301+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:18:42.797301+0000","last_undegraded":"2024-05-08T23:18:42.797301+0000","last_fullsized":"2024-05-08T23:18:42.797301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798260+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:42.798260+0000","last_peered":"2024-05-08T23:18:42.798260+0000","last_clean":"2024-05-08T23:18:42.798260+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:42.798260+0000","last_undegraded":"2024-05-08T23:18:42.798260+0000","last_fullsized":"2024-05-08T23:18:42.798260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:51.758863+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:51.758863+0000","last_peered":"2024-05-08T23:18:51.758863+0000","last_clean":"2024-05-08T23:18:51.758863+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:51.758863+0000","last_undegraded":"2024-05-08T23:18:51.758863+0000","last_fullsized":"2024-05-08T23:18:51.758863+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796860+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:18:42.796860+0000","last_peered":"2024-05-08T23:18:42.796860+0000","last_clean":"2024-05-08T23:18:42.796860+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:18:42.796860+0000","last_undegraded":"2024-05-08T23:18:42.796860+0000","last_fullsized":"2024-05-08T23:18:42.796860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803558+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803558+0000","last_peered":"2024-05-08T23:18:43.803558+0000","last_clean":"2024-05-08T23:18:43.803558+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803558+0000","last_undegraded":"2024-05-08T23:18:43.803558+0000","last_fullsized":"2024-05-08T23:18:43.803558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796303+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:42.796303+0000","last_peered":"2024-05-08T23:18:42.796303+0000","last_clean":"2024-05-08T23:18:42.796303+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:42.796303+0000","last_undegraded":"2024-05-08T23:18:42.796303+0000","last_fullsized":"2024-05-08T23:18:42.796303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798231+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:18:42.798231+0000","last_peered":"2024-05-08T23:18:42.798231+0000","last_clean":"2024-05-08T23:18:42.798231+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:18:42.798231+0000","last_undegraded":"2024-05-08T23:18:42.798231+0000","last_fullsized":"2024-05-08T23:18:42.798231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801918+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:42.801918+0000","last_peered":"2024-05-08T23:18:42.801918+0000","last_clean":"2024-05-08T23:18:42.801918+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:42.801918+0000","last_undegraded":"2024-05-08T23:18:42.801918+0000","last_fullsized":"2024-05-08T23:18:42.801918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799336+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:42.799336+0000","last_peered":"2024-05-08T23:18:42.799336+0000","last_clean":"2024-05-08T23:18:42.799336+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:42.799336+0000","last_undegraded":"2024-05-08T23:18:42.799336+0000","last_fullsized":"2024-05-08T23:18:42.799336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795875+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:18:42.795875+0000","last_peered":"2024-05-08T23:18:42.795875+0000","last_clean":"2024-05-08T23:18:42.795875+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:18:42.795875+0000","last_undegraded":"2024-05-08T23:18:42.795875+0000","last_fullsized":"2024-05-08T23:18:42.795875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850309+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:42.850309+0000","last_peered":"2024-05-08T23:18:42.850309+0000","last_clean":"2024-05-08T23:18:42.850309+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:42.850309+0000","last_undegraded":"2024-05-08T23:18:42.850309+0000","last_fullsized":"2024-05-08T23:18:42.850309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797842+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:42.797842+0000","last_peered":"2024-05-08T23:18:42.797842+0000","last_clean":"2024-05-08T23:18:42.797842+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:42.797842+0000","last_undegraded":"2024-05-08T23:18:42.797842+0000","last_fullsized":"2024-05-08T23:18:42.797842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792376+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:18:42.792376+0000","last_peered":"2024-05-08T23:18:42.792376+0000","last_clean":"2024-05-08T23:18:42.792376+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:18:42.792376+0000","last_undegraded":"2024-05-08T23:18:42.792376+0000","last_fullsized":"2024-05-08T23:18:42.792376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795976+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:42.795976+0000","last_peered":"2024-05-08T23:18:42.795976+0000","last_clean":"2024-05-08T23:18:42.795976+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:42.795976+0000","last_undegraded":"2024-05-08T23:18:42.795976+0000","last_fullsized":"2024-05-08T23:18:42.795976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":61,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:50.756977+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:50.756977+0000","last_peered":"2024-05-08T23:18:50.756977+0000","last_clean":"2024-05-08T23:18:50.756977+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:50.756977+0000","last_undegraded":"2024-05-08T23:18:50.756977+0000","last_fullsized":"2024-05-08T23:18:50.756977+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800494+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:18:42.800494+0000","last_peered":"2024-05-08T23:18:42.800494+0000","last_clean":"2024-05-08T23:18:42.800494+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:18:42.800494+0000","last_undegraded":"2024-05-08T23:18:42.800494+0000","last_fullsized":"2024-05-08T23:18:42.800494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792335+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:42.792335+0000","last_peered":"2024-05-08T23:18:42.792335+0000","last_clean":"2024-05-08T23:18:42.792335+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:42.792335+0000","last_undegraded":"2024-05-08T23:18:42.792335+0000","last_fullsized":"2024-05-08T23:18:42.792335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797044+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:42.797044+0000","last_peered":"2024-05-08T23:18:42.797044+0000","last_clean":"2024-05-08T23:18:42.797044+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:42.797044+0000","last_undegraded":"2024-05-08T23:18:42.797044+0000","last_fullsized":"2024-05-08T23:18:42.797044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850542+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:18:42.850542+0000","last_peered":"2024-05-08T23:18:42.850542+0000","last_clean":"2024-05-08T23:18:42.850542+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:18:42.850542+0000","last_undegraded":"2024-05-08T23:18:42.850542+0000","last_fullsized":"2024-05-08T23:18:42.850542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793231+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:42.793231+0000","last_peered":"2024-05-08T23:18:42.793231+0000","last_clean":"2024-05-08T23:18:42.793231+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:42.793231+0000","last_undegraded":"2024-05-08T23:18:42.793231+0000","last_fullsized":"2024-05-08T23:18:42.793231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797201+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:42.797201+0000","last_peered":"2024-05-08T23:18:42.797201+0000","last_clean":"2024-05-08T23:18:42.797201+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:42.797201+0000","last_undegraded":"2024-05-08T23:18:42.797201+0000","last_fullsized":"2024-05-08T23:18:42.797201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800205+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:18:42.800205+0000","last_peered":"2024-05-08T23:18:42.800205+0000","last_clean":"2024-05-08T23:18:42.800205+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:18:42.800205+0000","last_undegraded":"2024-05-08T23:18:42.800205+0000","last_fullsized":"2024-05-08T23:18:42.800205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792502+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:42.792502+0000","last_peered":"2024-05-08T23:18:42.792502+0000","last_clean":"2024-05-08T23:18:42.792502+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:42.792502+0000","last_undegraded":"2024-05-08T23:18:42.792502+0000","last_fullsized":"2024-05-08T23:18:42.792502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:51.758228+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:51.758228+0000","last_peered":"2024-05-08T23:18:51.758228+0000","last_clean":"2024-05-08T23:18:51.758228+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:51.758228+0000","last_undegraded":"2024-05-08T23:18:51.758228+0000","last_fullsized":"2024-05-08T23:18:51.758228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61146,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":88,"num_read_kb":104,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797717+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:18:42.797717+0000","last_peered":"2024-05-08T23:18:42.797717+0000","last_clean":"2024-05-08T23:18:42.797717+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:18:42.797717+0000","last_undegraded":"2024-05-08T23:18:42.797717+0000","last_fullsized":"2024-05-08T23:18:42.797717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797660+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:18:42.797660+0000","last_peered":"2024-05-08T23:18:42.797660+0000","last_clean":"2024-05-08T23:18:42.797660+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:18:42.797660+0000","last_undegraded":"2024-05-08T23:18:42.797660+0000","last_fullsized":"2024-05-08T23:18:42.797660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798509+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:42.798509+0000","last_peered":"2024-05-08T23:18:42.798509+0000","last_clean":"2024-05-08T23:18:42.798509+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:42.798509+0000","last_undegraded":"2024-05-08T23:18:42.798509+0000","last_fullsized":"2024-05-08T23:18:42.798509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799523+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:18:42.799523+0000","last_peered":"2024-05-08T23:18:42.799523+0000","last_clean":"2024-05-08T23:18:42.799523+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:18:42.799523+0000","last_undegraded":"2024-05-08T23:18:42.799523+0000","last_fullsized":"2024-05-08T23:18:42.799523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801502+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:42.801502+0000","last_peered":"2024-05-08T23:18:42.801502+0000","last_clean":"2024-05-08T23:18:42.801502+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:42.801502+0000","last_undegraded":"2024-05-08T23:18:42.801502+0000","last_fullsized":"2024-05-08T23:18:42.801502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798399+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:42.798399+0000","last_peered":"2024-05-08T23:18:42.798399+0000","last_clean":"2024-05-08T23:18:42.798399+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:42.798399+0000","last_undegraded":"2024-05-08T23:18:42.798399+0000","last_fullsized":"2024-05-08T23:18:42.798399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034932+0000","last_change":"2024-05-08T23:18:44.034932+0000","last_active":"2024-05-08T23:18:44.034932+0000","last_peered":"2024-05-08T23:18:44.034932+0000","last_clean":"2024-05-08T23:18:44.034932+0000","last_became_active":"2024-05-08T23:18:44.034439+0000","last_became_peered":"2024-05-08T23:18:44.034439+0000","last_unstale":"2024-05-08T23:18:44.034932+0000","last_undegraded":"2024-05-08T23:18:44.034932+0000","last_fullsized":"2024-05-08T23:18:44.034932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.536646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793500+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:42.793500+0000","last_peered":"2024-05-08T23:18:42.793500+0000","last_clean":"2024-05-08T23:18:42.793500+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:42.793500+0000","last_undegraded":"2024-05-08T23:18:42.793500+0000","last_fullsized":"2024-05-08T23:18:42.793500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:49.756562+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:49.756562+0000","last_peered":"2024-05-08T23:18:49.756562+0000","last_clean":"2024-05-08T23:18:49.756562+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:49.756562+0000","last_undegraded":"2024-05-08T23:18:49.756562+0000","last_fullsized":"2024-05-08T23:18:49.756562+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797290+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:18:42.797290+0000","last_peered":"2024-05-08T23:18:42.797290+0000","last_clean":"2024-05-08T23:18:42.797290+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:18:42.797290+0000","last_undegraded":"2024-05-08T23:18:42.797290+0000","last_fullsized":"2024-05-08T23:18:42.797290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800355+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:42.800355+0000","last_peered":"2024-05-08T23:18:42.800355+0000","last_clean":"2024-05-08T23:18:42.800355+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:42.800355+0000","last_undegraded":"2024-05-08T23:18:42.800355+0000","last_fullsized":"2024-05-08T23:18:42.800355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796252+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:42.796252+0000","last_peered":"2024-05-08T23:18:42.796252+0000","last_clean":"2024-05-08T23:18:42.796252+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:42.796252+0000","last_undegraded":"2024-05-08T23:18:42.796252+0000","last_fullsized":"2024-05-08T23:18:42.796252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.929400+0000","last_change":"2024-05-08T23:18:43.929400+0000","last_active":"2024-05-08T23:18:43.929400+0000","last_peered":"2024-05-08T23:18:43.929400+0000","last_clean":"2024-05-08T23:18:43.929400+0000","last_became_active":"2024-05-08T23:18:43.928980+0000","last_became_peered":"2024-05-08T23:18:43.928980+0000","last_unstale":"2024-05-08T23:18:43.929400+0000","last_undegraded":"2024-05-08T23:18:43.929400+0000","last_fullsized":"2024-05-08T23:18:43.929400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:10.433725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797719+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:18:42.797719+0000","last_peered":"2024-05-08T23:18:42.797719+0000","last_clean":"2024-05-08T23:18:42.797719+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:18:42.797719+0000","last_undegraded":"2024-05-08T23:18:42.797719+0000","last_fullsized":"2024-05-08T23:18:42.797719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798692+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:42.798692+0000","last_peered":"2024-05-08T23:18:42.798692+0000","last_clean":"2024-05-08T23:18:42.798692+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:42.798692+0000","last_undegraded":"2024-05-08T23:18:42.798692+0000","last_fullsized":"2024-05-08T23:18:42.798692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802290+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:18:42.802290+0000","last_peered":"2024-05-08T23:18:42.802290+0000","last_clean":"2024-05-08T23:18:42.802290+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:18:42.802290+0000","last_undegraded":"2024-05-08T23:18:42.802290+0000","last_fullsized":"2024-05-08T23:18:42.802290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796770+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:42.796770+0000","last_peered":"2024-05-08T23:18:42.796770+0000","last_clean":"2024-05-08T23:18:42.796770+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:42.796770+0000","last_undegraded":"2024-05-08T23:18:42.796770+0000","last_fullsized":"2024-05-08T23:18:42.796770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792981+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:42.792981+0000","last_peered":"2024-05-08T23:18:42.792981+0000","last_clean":"2024-05-08T23:18:42.792981+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:42.792981+0000","last_undegraded":"2024-05-08T23:18:42.792981+0000","last_fullsized":"2024-05-08T23:18:42.792981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796821+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:18:42.796821+0000","last_peered":"2024-05-08T23:18:42.796821+0000","last_clean":"2024-05-08T23:18:42.796821+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:18:42.796821+0000","last_undegraded":"2024-05-08T23:18:42.796821+0000","last_fullsized":"2024-05-08T23:18:42.796821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799638+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:42.799638+0000","last_peered":"2024-05-08T23:18:42.799638+0000","last_clean":"2024-05-08T23:18:42.799638+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:42.799638+0000","last_undegraded":"2024-05-08T23:18:42.799638+0000","last_fullsized":"2024-05-08T23:18:42.799638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796478+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:42.796478+0000","last_peered":"2024-05-08T23:18:42.796478+0000","last_clean":"2024-05-08T23:18:42.796478+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:42.796478+0000","last_undegraded":"2024-05-08T23:18:42.796478+0000","last_fullsized":"2024-05-08T23:18:42.796478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796378+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:18:42.796378+0000","last_peered":"2024-05-08T23:18:42.796378+0000","last_clean":"2024-05-08T23:18:42.796378+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:18:42.796378+0000","last_undegraded":"2024-05-08T23:18:42.796378+0000","last_fullsized":"2024-05-08T23:18:42.796378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799747+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:42.799747+0000","last_peered":"2024-05-08T23:18:42.799747+0000","last_clean":"2024-05-08T23:18:42.799747+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:42.799747+0000","last_undegraded":"2024-05-08T23:18:42.799747+0000","last_fullsized":"2024-05-08T23:18:42.799747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797943+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:18:42.797943+0000","last_peered":"2024-05-08T23:18:42.797943+0000","last_clean":"2024-05-08T23:18:42.797943+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:18:42.797943+0000","last_undegraded":"2024-05-08T23:18:42.797943+0000","last_fullsized":"2024-05-08T23:18:42.797943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:50.757317+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:50.757317+0000","last_peered":"2024-05-08T23:18:50.757317+0000","last_clean":"2024-05-08T23:18:50.757317+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:50.757317+0000","last_undegraded":"2024-05-08T23:18:50.757317+0000","last_fullsized":"2024-05-08T23:18:50.757317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796463+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:42.796463+0000","last_peered":"2024-05-08T23:18:42.796463+0000","last_clean":"2024-05-08T23:18:42.796463+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:42.796463+0000","last_undegraded":"2024-05-08T23:18:42.796463+0000","last_fullsized":"2024-05-08T23:18:42.796463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796295+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:18:42.796295+0000","last_peered":"2024-05-08T23:18:42.796295+0000","last_clean":"2024-05-08T23:18:42.796295+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:18:42.796295+0000","last_undegraded":"2024-05-08T23:18:42.796295+0000","last_fullsized":"2024-05-08T23:18:42.796295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793103+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:42.793103+0000","last_peered":"2024-05-08T23:18:42.793103+0000","last_clean":"2024-05-08T23:18:42.793103+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:42.793103+0000","last_undegraded":"2024-05-08T23:18:42.793103+0000","last_fullsized":"2024-05-08T23:18:42.793103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796517+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:42.796517+0000","last_peered":"2024-05-08T23:18:42.796517+0000","last_clean":"2024-05-08T23:18:42.796517+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:42.796517+0000","last_undegraded":"2024-05-08T23:18:42.796517+0000","last_fullsized":"2024-05-08T23:18:42.796517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796867+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:18:42.796867+0000","last_peered":"2024-05-08T23:18:42.796867+0000","last_clean":"2024-05-08T23:18:42.796867+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:18:42.796867+0000","last_undegraded":"2024-05-08T23:18:42.796867+0000","last_fullsized":"2024-05-08T23:18:42.796867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799236+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:18:42.799236+0000","last_peered":"2024-05-08T23:18:42.799236+0000","last_clean":"2024-05-08T23:18:42.799236+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:18:42.799236+0000","last_undegraded":"2024-05-08T23:18:42.799236+0000","last_fullsized":"2024-05-08T23:18:42.799236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796699+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:42.796699+0000","last_peered":"2024-05-08T23:18:42.796699+0000","last_clean":"2024-05-08T23:18:42.796699+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:42.796699+0000","last_undegraded":"2024-05-08T23:18:42.796699+0000","last_fullsized":"2024-05-08T23:18:42.796699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802361+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:42.802361+0000","last_peered":"2024-05-08T23:18:42.802361+0000","last_clean":"2024-05-08T23:18:42.802361+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:42.802361+0000","last_undegraded":"2024-05-08T23:18:42.802361+0000","last_fullsized":"2024-05-08T23:18:42.802361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797799+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:42.797799+0000","last_peered":"2024-05-08T23:18:42.797799+0000","last_clean":"2024-05-08T23:18:42.797799+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:42.797799+0000","last_undegraded":"2024-05-08T23:18:42.797799+0000","last_fullsized":"2024-05-08T23:18:42.797799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799837+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:18:42.799837+0000","last_peered":"2024-05-08T23:18:42.799837+0000","last_clean":"2024-05-08T23:18:42.799837+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:18:42.799837+0000","last_undegraded":"2024-05-08T23:18:42.799837+0000","last_fullsized":"2024-05-08T23:18:42.799837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795769+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:42.795769+0000","last_peered":"2024-05-08T23:18:42.795769+0000","last_clean":"2024-05-08T23:18:42.795769+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:42.795769+0000","last_undegraded":"2024-05-08T23:18:42.795769+0000","last_fullsized":"2024-05-08T23:18:42.795769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793582+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:18:42.793582+0000","last_peered":"2024-05-08T23:18:42.793582+0000","last_clean":"2024-05-08T23:18:42.793582+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:18:42.793582+0000","last_undegraded":"2024-05-08T23:18:42.793582+0000","last_fullsized":"2024-05-08T23:18:42.793582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797388+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:18:42.797388+0000","last_peered":"2024-05-08T23:18:42.797388+0000","last_clean":"2024-05-08T23:18:42.797388+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:18:42.797388+0000","last_undegraded":"2024-05-08T23:18:42.797388+0000","last_fullsized":"2024-05-08T23:18:42.797388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797897+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:42.797897+0000","last_peered":"2024-05-08T23:18:42.797897+0000","last_clean":"2024-05-08T23:18:42.797897+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:42.797897+0000","last_undegraded":"2024-05-08T23:18:42.797897+0000","last_fullsized":"2024-05-08T23:18:42.797897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793553+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:42.793553+0000","last_peered":"2024-05-08T23:18:42.793553+0000","last_clean":"2024-05-08T23:18:42.793553+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:42.793553+0000","last_undegraded":"2024-05-08T23:18:42.793553+0000","last_fullsized":"2024-05-08T23:18:42.793553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850431+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:18:42.850431+0000","last_peered":"2024-05-08T23:18:42.850431+0000","last_clean":"2024-05-08T23:18:42.850431+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:18:42.850431+0000","last_undegraded":"2024-05-08T23:18:42.850431+0000","last_fullsized":"2024-05-08T23:18:42.850431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793223+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:42.793223+0000","last_peered":"2024-05-08T23:18:42.793223+0000","last_clean":"2024-05-08T23:18:42.793223+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:42.793223+0000","last_undegraded":"2024-05-08T23:18:42.793223+0000","last_fullsized":"2024-05-08T23:18:42.793223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799949+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:42.799949+0000","last_peered":"2024-05-08T23:18:42.799949+0000","last_clean":"2024-05-08T23:18:42.799949+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:42.799949+0000","last_undegraded":"2024-05-08T23:18:42.799949+0000","last_fullsized":"2024-05-08T23:18:42.799949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796359+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:18:42.796359+0000","last_peered":"2024-05-08T23:18:42.796359+0000","last_clean":"2024-05-08T23:18:42.796359+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:18:42.796359+0000","last_undegraded":"2024-05-08T23:18:42.796359+0000","last_fullsized":"2024-05-08T23:18:42.796359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793663+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:42.793663+0000","last_peered":"2024-05-08T23:18:42.793663+0000","last_clean":"2024-05-08T23:18:42.793663+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:42.793663+0000","last_undegraded":"2024-05-08T23:18:42.793663+0000","last_fullsized":"2024-05-08T23:18:42.793663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796095+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:42.796095+0000","last_peered":"2024-05-08T23:18:42.796095+0000","last_clean":"2024-05-08T23:18:42.796095+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:42.796095+0000","last_undegraded":"2024-05-08T23:18:42.796095+0000","last_fullsized":"2024-05-08T23:18:42.796095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801296+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:18:42.801296+0000","last_peered":"2024-05-08T23:18:42.801296+0000","last_clean":"2024-05-08T23:18:42.801296+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:18:42.801296+0000","last_undegraded":"2024-05-08T23:18:42.801296+0000","last_fullsized":"2024-05-08T23:18:42.801296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792930+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:42.792930+0000","last_peered":"2024-05-08T23:18:42.792930+0000","last_clean":"2024-05-08T23:18:42.792930+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:42.792930+0000","last_undegraded":"2024-05-08T23:18:42.792930+0000","last_fullsized":"2024-05-08T23:18:42.792930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799130+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:42.799130+0000","last_peered":"2024-05-08T23:18:42.799130+0000","last_clean":"2024-05-08T23:18:42.799130+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:42.799130+0000","last_undegraded":"2024-05-08T23:18:42.799130+0000","last_fullsized":"2024-05-08T23:18:42.799130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796367+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:18:42.796367+0000","last_peered":"2024-05-08T23:18:42.796367+0000","last_clean":"2024-05-08T23:18:42.796367+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:18:42.796367+0000","last_undegraded":"2024-05-08T23:18:42.796367+0000","last_fullsized":"2024-05-08T23:18:42.796367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793402+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:42.793402+0000","last_peered":"2024-05-08T23:18:42.793402+0000","last_clean":"2024-05-08T23:18:42.793402+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:42.793402+0000","last_undegraded":"2024-05-08T23:18:42.793402+0000","last_fullsized":"2024-05-08T23:18:42.793402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.808122+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:43.808122+0000","last_peered":"2024-05-08T23:18:43.808122+0000","last_clean":"2024-05-08T23:18:43.808122+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:43.808122+0000","last_undegraded":"2024-05-08T23:18:43.808122+0000","last_fullsized":"2024-05-08T23:18:43.808122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796533+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:18:42.796533+0000","last_peered":"2024-05-08T23:18:42.796533+0000","last_clean":"2024-05-08T23:18:42.796533+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:18:42.796533+0000","last_undegraded":"2024-05-08T23:18:42.796533+0000","last_fullsized":"2024-05-08T23:18:42.796533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800112+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:42.800112+0000","last_peered":"2024-05-08T23:18:42.800112+0000","last_clean":"2024-05-08T23:18:42.800112+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:42.800112+0000","last_undegraded":"2024-05-08T23:18:42.800112+0000","last_fullsized":"2024-05-08T23:18:42.800112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800360+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:42.800360+0000","last_peered":"2024-05-08T23:18:42.800360+0000","last_clean":"2024-05-08T23:18:42.800360+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:42.800360+0000","last_undegraded":"2024-05-08T23:18:42.800360+0000","last_fullsized":"2024-05-08T23:18:42.800360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797464+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:18:42.797464+0000","last_peered":"2024-05-08T23:18:42.797464+0000","last_clean":"2024-05-08T23:18:42.797464+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:18:42.797464+0000","last_undegraded":"2024-05-08T23:18:42.797464+0000","last_fullsized":"2024-05-08T23:18:42.797464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849917+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:42.849917+0000","last_peered":"2024-05-08T23:18:42.849917+0000","last_clean":"2024-05-08T23:18:42.849917+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:42.849917+0000","last_undegraded":"2024-05-08T23:18:42.849917+0000","last_fullsized":"2024-05-08T23:18:42.849917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797180+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:18:42.797180+0000","last_peered":"2024-05-08T23:18:42.797180+0000","last_clean":"2024-05-08T23:18:42.797180+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:18:42.797180+0000","last_undegraded":"2024-05-08T23:18:42.797180+0000","last_fullsized":"2024-05-08T23:18:42.797180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796081+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:18:42.796081+0000","last_peered":"2024-05-08T23:18:42.796081+0000","last_clean":"2024-05-08T23:18:42.796081+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:18:42.796081+0000","last_undegraded":"2024-05-08T23:18:42.796081+0000","last_fullsized":"2024-05-08T23:18:42.796081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931402+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931402+0000","last_peered":"2024-05-08T23:18:43.931402+0000","last_clean":"2024-05-08T23:18:43.931402+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931402+0000","last_undegraded":"2024-05-08T23:18:43.931402+0000","last_fullsized":"2024-05-08T23:18:43.931402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796911+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:18:42.796911+0000","last_peered":"2024-05-08T23:18:42.796911+0000","last_clean":"2024-05-08T23:18:42.796911+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:18:42.796911+0000","last_undegraded":"2024-05-08T23:18:42.796911+0000","last_fullsized":"2024-05-08T23:18:42.796911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801392+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:18:42.801392+0000","last_peered":"2024-05-08T23:18:42.801392+0000","last_clean":"2024-05-08T23:18:42.801392+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:18:42.801392+0000","last_undegraded":"2024-05-08T23:18:42.801392+0000","last_fullsized":"2024-05-08T23:18:42.801392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034679+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:18:44.034679+0000","last_peered":"2024-05-08T23:18:44.034679+0000","last_clean":"2024-05-08T23:18:44.034679+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:18:44.034679+0000","last_undegraded":"2024-05-08T23:18:44.034679+0000","last_fullsized":"2024-05-08T23:18:44.034679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799239+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:18:42.799239+0000","last_peered":"2024-05-08T23:18:42.799239+0000","last_clean":"2024-05-08T23:18:42.799239+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:18:42.799239+0000","last_undegraded":"2024-05-08T23:18:42.799239+0000","last_fullsized":"2024-05-08T23:18:42.799239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850099+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:18:42.850099+0000","last_peered":"2024-05-08T23:18:42.850099+0000","last_clean":"2024-05-08T23:18:42.850099+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:18:42.850099+0000","last_undegraded":"2024-05-08T23:18:42.850099+0000","last_fullsized":"2024-05-08T23:18:42.850099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799941+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:18:42.799941+0000","last_peered":"2024-05-08T23:18:42.799941+0000","last_clean":"2024-05-08T23:18:42.799941+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:18:42.799941+0000","last_undegraded":"2024-05-08T23:18:42.799941+0000","last_fullsized":"2024-05-08T23:18:42.799941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.803347+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:18:42.803347+0000","last_peered":"2024-05-08T23:18:42.803347+0000","last_clean":"2024-05-08T23:18:42.803347+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:18:42.803347+0000","last_undegraded":"2024-05-08T23:18:42.803347+0000","last_fullsized":"2024-05-08T23:18:42.803347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800760+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:18:42.800760+0000","last_peered":"2024-05-08T23:18:42.800760+0000","last_clean":"2024-05-08T23:18:42.800760+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:18:42.800760+0000","last_undegraded":"2024-05-08T23:18:42.800760+0000","last_fullsized":"2024-05-08T23:18:42.800760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797474+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:18:42.797474+0000","last_peered":"2024-05-08T23:18:42.797474+0000","last_clean":"2024-05-08T23:18:42.797474+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:18:42.797474+0000","last_undegraded":"2024-05-08T23:18:42.797474+0000","last_fullsized":"2024-05-08T23:18:42.797474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797850+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:18:42.797850+0000","last_peered":"2024-05-08T23:18:42.797850+0000","last_clean":"2024-05-08T23:18:42.797850+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:18:42.797850+0000","last_undegraded":"2024-05-08T23:18:42.797850+0000","last_fullsized":"2024-05-08T23:18:42.797850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796074+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:18:42.796074+0000","last_peered":"2024-05-08T23:18:42.796074+0000","last_clean":"2024-05-08T23:18:42.796074+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:18:42.796074+0000","last_undegraded":"2024-05-08T23:18:42.796074+0000","last_fullsized":"2024-05-08T23:18:42.796074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800329+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:18:42.800329+0000","last_peered":"2024-05-08T23:18:42.800329+0000","last_clean":"2024-05-08T23:18:42.800329+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:18:42.800329+0000","last_undegraded":"2024-05-08T23:18:42.800329+0000","last_fullsized":"2024-05-08T23:18:42.800329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800055+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:18:42.800055+0000","last_peered":"2024-05-08T23:18:42.800055+0000","last_clean":"2024-05-08T23:18:42.800055+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:18:42.800055+0000","last_undegraded":"2024-05-08T23:18:42.800055+0000","last_fullsized":"2024-05-08T23:18:42.800055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801597+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:18:42.801597+0000","last_peered":"2024-05-08T23:18:42.801597+0000","last_clean":"2024-05-08T23:18:42.801597+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:18:42.801597+0000","last_undegraded":"2024-05-08T23:18:42.801597+0000","last_fullsized":"2024-05-08T23:18:42.801597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":264,"num_read_kb":200,"num_write":156,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645336,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":463111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70699999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.80400000000000005}]},{"osd":8,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68400000000000005}]},{"osd":9,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.746},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.77300000000000002}]},{"osd":10,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.78800000000000003}]}]},{"osd":10,"up_from":65,"seq":279172874268,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1617920,"data_stored":1111146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48599999999999999}]},{"osd":1,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.378}]},{"osd":2,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65300000000000002}]},{"osd":3,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.72199999999999998}]},{"osd":6,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.74199999999999999}]},{"osd":9,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.61899999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.77700000000000002}]}]},{"osd":9,"up_from":60,"seq":257698037790,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":467443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.498},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.60399999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55900000000000005}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.34699999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.67500000000000004}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.53900000000000003}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.70899999999999996}]},{"osd":10,"last update":"Wed May 8 23:17:51 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.65000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]},{"osd":8,"up_from":53,"seq":227633266723,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86499999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78000000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":1.137,"15min":1.137},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84399999999999997}]},{"osd":5,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.74399999999999999}]},{"osd":6,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92200000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.95499999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93500000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.94599999999999995}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.89700000000000002}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.93600000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.88200000000000001}]}]},{"osd":7,"up_from":46,"seq":197568495654,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1540096,"data_stored":1058358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.495}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.628}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.38}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61199999999999999}]},{"osd":8,"last update":"Wed May 8 23:18:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.65600000000000003}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.72399999999999998}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.754}]}]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":458523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.56599999999999995}]},{"osd":4,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.621}]},{"osd":5,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.752},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69999999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0049999999999999,"5min":1.177,"15min":1.177},"last":0.79200000000000004}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.046,"5min":1.091,"15min":1.091},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65900000000000003}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.68500000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":995328,"data_stored":519579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.41699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.39000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69199999999999995}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.45000000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79300000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.52600000000000002},"min":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76400000000000001,"5min":0.873,"15min":0.873},"last":0.46600000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.371}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.46800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.58299999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.621,"15min":0.59299999999999997},"min":{"1min":0.54700000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57899999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60599999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68500000000000005,"15min":0.67000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71599999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.66800000000000004}]},{"osd":10,"last update":"Wed May 8 23:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":954368,"data_stored":459911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.501}]},{"osd":3,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.437},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65200000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76900000000000002}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.55800000000000005}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.622},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79000000000000004}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.80600000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.751}]}]},{"osd":2,"up_from":17,"seq":73014444088,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":520881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.39100000000000001}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53000000000000003}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.437,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70299999999999996,"5min":0.872,"15min":0.872},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.753,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.439}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.58699999999999997},"min":{"1min":0.52900000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.627}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.748}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65800000000000003}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.58999999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67700000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72899999999999998}]}]},{"osd":3,"up_from":23,"seq":98784247859,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":942080,"data_stored":458717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.501},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.745}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.82299999999999995}]},{"osd":5,"last update":"Wed May 8 23:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.878,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.77500000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.624,"15min":0.58799999999999997},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91900000000000004,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.55700000000000005}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.625},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.80300000000000005,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.83999999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":5.5659999999999998,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.93100000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.87}]}]},{"osd":4,"up_from":30,"seq":128849018928,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.29299999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39800000000000002}]},{"osd":2,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.434,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.46500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75800000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.51100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.45900000000000002}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83399999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70799999999999996}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.61599999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.752},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67600000000000005}]}]},{"osd":5,"up_from":35,"seq":150323855405,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1552384,"data_stored":1054325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47299999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.52500000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.754,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60799999999999998}]},{"osd":6,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.66900000000000004}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56100000000000005}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74399999999999999}]},{"osd":10,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:18:58.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:57 smithi016 ceph-mon[42541]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:57 smithi016 ceph-mon[42541]: pgmap v260: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:18:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:57 smithi189 ceph-mon[46919]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:57 smithi189 ceph-mon[46919]: pgmap v260: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:18:58.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:57 smithi155 ceph-mon[47105]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:18:58.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:57 smithi155 ceph-mon[47105]: pgmap v260: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T23:18:59.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T23:19:00.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:18:59 smithi016 ceph-mon[42541]: pgmap v261: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T23:19:00.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:18:59 smithi189 ceph-mon[46919]: pgmap v261: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T23:19:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:00.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:18:59 smithi155 ceph-mon[47105]: pgmap v261: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 -- 172.21.15.16:0/1560226142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a8105f00 msgr2=0x7fe3a81062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 --2- 172.21.15.16:0/1560226142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a8105f00 0x7fe3a81062e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009990 tx=0x7fe39c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 -- 172.21.15.16:0/1560226142 shutdown_connections 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 --2- 172.21.15.16:0/1560226142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8100890 0x7fe3a810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 --2- 172.21.15.16:0/1560226142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a8105f00 0x7fe3a81062e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 --2- 172.21.15.16:0/1560226142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 -- 172.21.15.16:0/1560226142 >> 172.21.15.16:0/1560226142 conn(0x7fe3a8075f80 msgr2=0x7fe3a8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 -- 172.21.15.16:0/1560226142 shutdown_connections 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.171+0000 7fe3afd70700 1 -- 172.21.15.16:0/1560226142 wait complete. 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 Processor -- start 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 -- start start 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8100890 0x7fe3a8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a8105f00 0x7fe3a819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a819c130 con 0x7fe3a80ffed0 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a819c2a0 con 0x7fe3a8105f00 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.172+0000 7fe3afd70700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a819c460 con 0x7fe3a8100890 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52540/0 (socket says 172.21.15.16:52540) 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 -- 172.21.15.16:0/3127182224 learned_addr learned my addr 172.21.15.16:0/3127182224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3ad30b700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8100890 0x7fe3a8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:01.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3ae30d700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a8105f00 0x7fe3a819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 -- 172.21.15.16:0/3127182224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8100890 msgr2=0x7fe3a8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8100890 0x7fe3a8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 -- 172.21.15.16:0/3127182224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a8105f00 msgr2=0x7fe3a819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a8105f00 0x7fe3a819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 -- 172.21.15.16:0/3127182224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0095f0 con 0x7fe3a80ffed0 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3adb0c700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8197230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe39c006010 tx=0x7fe39c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c01d070 con 0x7fe3a80ffed0 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe39c00fb80 con 0x7fe3a80ffed0 2024-05-08T23:19:01.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a819c6e0 con 0x7fe3a80ffed0 2024-05-08T23:19:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c022b90 con 0x7fe3a80ffed0 2024-05-08T23:19:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.173+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a81a3fa0 con 0x7fe3a80ffed0 2024-05-08T23:19:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.174+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a81015b0 con 0x7fe3a80ffed0 2024-05-08T23:19:01.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.176+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe39c00f6f0 con 0x7fe3a80ffed0 2024-05-08T23:19:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.176+0000 7fe39affd700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3940a11b0 0x7fe3940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.177+0000 7fe3ad30b700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3940a11b0 0x7fe3940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.177+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe39c0ce640 con 0x7fe3a80ffed0 2024-05-08T23:19:01.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.177+0000 7fe3ad30b700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3940a11b0 0x7fe3940a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8198790 tx=0x7fe3a400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:01.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.180+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe39c0979a0 con 0x7fe3a80ffed0 2024-05-08T23:19:01.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.388+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3a8198540 con 0x7fe3940a11b0 2024-05-08T23:19:01.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.401+0000 7fe39affd700 1 -- 172.21.15.16:0/3127182224 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426709 (secure 0 0 0) 0x7fe3a8198540 con 0x7fe3940a11b0 2024-05-08T23:19:01.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3940a11b0 msgr2=0x7fe3940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3940a11b0 0x7fe3940a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8198790 tx=0x7fe3a400a380 comp rx=0 tx=0).stop 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 msgr2=0x7fe3a8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8197230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe39c006010 tx=0x7fe39c004730 comp rx=0 tx=0).stop 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 shutdown_connections 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3a8105f00 0x7fe3a819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3a80ffed0 0x7fe3a8197230 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe3940a11b0 0x7fe3940a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 --2- 172.21.15.16:0/3127182224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3a8100890 0x7fe3a8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 >> 172.21.15.16:0/3127182224 conn(0x7fe3a8075f80 msgr2=0x7fe3a8068bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 shutdown_connections 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:01.405+0000 7fe3afd70700 1 -- 172.21.15.16:0/3127182224 wait complete. 2024-05-08T23:19:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:19:01.738 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-05-08T23:19:00.608873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655262,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":430,"num_read_kb":334,"num_write":339,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394608,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":381421,"kb_avail":1124547216,"statfs":{"total":1151940427776,"available":1151536349184,"internally_reserved":0,"allocated":13332480,"data_stored":7498386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":390575432},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015277"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802073+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:18:42.802073+0000","last_peered":"2024-05-08T23:18:42.802073+0000","last_clean":"2024-05-08T23:18:42.802073+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:18:42.802073+0000","last_undegraded":"2024-05-08T23:18:42.802073+0000","last_fullsized":"2024-05-08T23:18:42.802073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850382+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:18:42.850382+0000","last_peered":"2024-05-08T23:18:42.850382+0000","last_clean":"2024-05-08T23:18:42.850382+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:18:42.850382+0000","last_undegraded":"2024-05-08T23:18:42.850382+0000","last_fullsized":"2024-05-08T23:18:42.850382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850219+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:18:42.850219+0000","last_peered":"2024-05-08T23:18:42.850219+0000","last_clean":"2024-05-08T23:18:42.850219+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:18:42.850219+0000","last_undegraded":"2024-05-08T23:18:42.850219+0000","last_fullsized":"2024-05-08T23:18:42.850219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793298+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:18:42.793298+0000","last_peered":"2024-05-08T23:18:42.793298+0000","last_clean":"2024-05-08T23:18:42.793298+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:18:42.793298+0000","last_undegraded":"2024-05-08T23:18:42.793298+0000","last_fullsized":"2024-05-08T23:18:42.793298+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799491+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:18:42.799491+0000","last_peered":"2024-05-08T23:18:42.799491+0000","last_clean":"2024-05-08T23:18:42.799491+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:18:42.799491+0000","last_undegraded":"2024-05-08T23:18:42.799491+0000","last_fullsized":"2024-05-08T23:18:42.799491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796401+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:18:42.796401+0000","last_peered":"2024-05-08T23:18:42.796401+0000","last_clean":"2024-05-08T23:18:42.796401+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:18:42.796401+0000","last_undegraded":"2024-05-08T23:18:42.796401+0000","last_fullsized":"2024-05-08T23:18:42.796401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797424+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:18:42.797424+0000","last_peered":"2024-05-08T23:18:42.797424+0000","last_clean":"2024-05-08T23:18:42.797424+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:18:42.797424+0000","last_undegraded":"2024-05-08T23:18:42.797424+0000","last_fullsized":"2024-05-08T23:18:42.797424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795761+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:18:42.795761+0000","last_peered":"2024-05-08T23:18:42.795761+0000","last_clean":"2024-05-08T23:18:42.795761+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:18:42.795761+0000","last_undegraded":"2024-05-08T23:18:42.795761+0000","last_fullsized":"2024-05-08T23:18:42.795761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800861+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:18:42.800861+0000","last_peered":"2024-05-08T23:18:42.800861+0000","last_clean":"2024-05-08T23:18:42.800861+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:18:42.800861+0000","last_undegraded":"2024-05-08T23:18:42.800861+0000","last_fullsized":"2024-05-08T23:18:42.800861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034666+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:18:44.034666+0000","last_peered":"2024-05-08T23:18:44.034666+0000","last_clean":"2024-05-08T23:18:44.034666+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:18:44.034666+0000","last_undegraded":"2024-05-08T23:18:44.034666+0000","last_fullsized":"2024-05-08T23:18:44.034666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797414+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:18:42.797414+0000","last_peered":"2024-05-08T23:18:42.797414+0000","last_clean":"2024-05-08T23:18:42.797414+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:18:42.797414+0000","last_undegraded":"2024-05-08T23:18:42.797414+0000","last_fullsized":"2024-05-08T23:18:42.797414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793543+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:18:42.793543+0000","last_peered":"2024-05-08T23:18:42.793543+0000","last_clean":"2024-05-08T23:18:42.793543+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:18:42.793543+0000","last_undegraded":"2024-05-08T23:18:42.793543+0000","last_fullsized":"2024-05-08T23:18:42.793543+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800158+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:18:42.800158+0000","last_peered":"2024-05-08T23:18:42.800158+0000","last_clean":"2024-05-08T23:18:42.800158+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:18:42.800158+0000","last_undegraded":"2024-05-08T23:18:42.800158+0000","last_fullsized":"2024-05-08T23:18:42.800158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849931+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:18:42.849931+0000","last_peered":"2024-05-08T23:18:42.849931+0000","last_clean":"2024-05-08T23:18:42.849931+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:18:42.849931+0000","last_undegraded":"2024-05-08T23:18:42.849931+0000","last_fullsized":"2024-05-08T23:18:42.849931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849831+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:18:42.849831+0000","last_peered":"2024-05-08T23:18:42.849831+0000","last_clean":"2024-05-08T23:18:42.849831+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:18:42.849831+0000","last_undegraded":"2024-05-08T23:18:42.849831+0000","last_fullsized":"2024-05-08T23:18:42.849831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799862+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:18:42.799862+0000","last_peered":"2024-05-08T23:18:42.799862+0000","last_clean":"2024-05-08T23:18:42.799862+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:18:42.799862+0000","last_undegraded":"2024-05-08T23:18:42.799862+0000","last_fullsized":"2024-05-08T23:18:42.799862+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801991+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:18:42.801991+0000","last_peered":"2024-05-08T23:18:42.801991+0000","last_clean":"2024-05-08T23:18:42.801991+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:18:42.801991+0000","last_undegraded":"2024-05-08T23:18:42.801991+0000","last_fullsized":"2024-05-08T23:18:42.801991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793563+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:18:42.793563+0000","last_peered":"2024-05-08T23:18:42.793563+0000","last_clean":"2024-05-08T23:18:42.793563+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:18:42.793563+0000","last_undegraded":"2024-05-08T23:18:42.793563+0000","last_fullsized":"2024-05-08T23:18:42.793563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799013+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:18:42.799013+0000","last_peered":"2024-05-08T23:18:42.799013+0000","last_clean":"2024-05-08T23:18:42.799013+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:18:42.799013+0000","last_undegraded":"2024-05-08T23:18:42.799013+0000","last_fullsized":"2024-05-08T23:18:42.799013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800896+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:42.800896+0000","last_peered":"2024-05-08T23:18:42.800896+0000","last_clean":"2024-05-08T23:18:42.800896+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:42.800896+0000","last_undegraded":"2024-05-08T23:18:42.800896+0000","last_fullsized":"2024-05-08T23:18:42.800896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793673+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:42.793673+0000","last_peered":"2024-05-08T23:18:42.793673+0000","last_clean":"2024-05-08T23:18:42.793673+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:42.793673+0000","last_undegraded":"2024-05-08T23:18:42.793673+0000","last_fullsized":"2024-05-08T23:18:42.793673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797273+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:18:42.797273+0000","last_peered":"2024-05-08T23:18:42.797273+0000","last_clean":"2024-05-08T23:18:42.797273+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:18:42.797273+0000","last_undegraded":"2024-05-08T23:18:42.797273+0000","last_fullsized":"2024-05-08T23:18:42.797273+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792847+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:42.792847+0000","last_peered":"2024-05-08T23:18:42.792847+0000","last_clean":"2024-05-08T23:18:42.792847+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:42.792847+0000","last_undegraded":"2024-05-08T23:18:42.792847+0000","last_fullsized":"2024-05-08T23:18:42.792847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850495+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:18:42.850495+0000","last_peered":"2024-05-08T23:18:42.850495+0000","last_clean":"2024-05-08T23:18:42.850495+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:18:42.850495+0000","last_undegraded":"2024-05-08T23:18:42.850495+0000","last_fullsized":"2024-05-08T23:18:42.850495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803307+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803307+0000","last_peered":"2024-05-08T23:18:43.803307+0000","last_clean":"2024-05-08T23:18:43.803307+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803307+0000","last_undegraded":"2024-05-08T23:18:43.803307+0000","last_fullsized":"2024-05-08T23:18:43.803307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.813513+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:43.813513+0000","last_peered":"2024-05-08T23:18:43.813513+0000","last_clean":"2024-05-08T23:18:43.813513+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:43.813513+0000","last_undegraded":"2024-05-08T23:18:43.813513+0000","last_fullsized":"2024-05-08T23:18:43.813513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800094+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:18:42.800094+0000","last_peered":"2024-05-08T23:18:42.800094+0000","last_clean":"2024-05-08T23:18:42.800094+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:18:42.800094+0000","last_undegraded":"2024-05-08T23:18:42.800094+0000","last_fullsized":"2024-05-08T23:18:42.800094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792881+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:42.792881+0000","last_peered":"2024-05-08T23:18:42.792881+0000","last_clean":"2024-05-08T23:18:42.792881+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:42.792881+0000","last_undegraded":"2024-05-08T23:18:42.792881+0000","last_fullsized":"2024-05-08T23:18:42.792881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797321+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:42.797321+0000","last_peered":"2024-05-08T23:18:42.797321+0000","last_clean":"2024-05-08T23:18:42.797321+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:42.797321+0000","last_undegraded":"2024-05-08T23:18:42.797321+0000","last_fullsized":"2024-05-08T23:18:42.797321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797301+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:18:42.797301+0000","last_peered":"2024-05-08T23:18:42.797301+0000","last_clean":"2024-05-08T23:18:42.797301+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:18:42.797301+0000","last_undegraded":"2024-05-08T23:18:42.797301+0000","last_fullsized":"2024-05-08T23:18:42.797301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798260+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:42.798260+0000","last_peered":"2024-05-08T23:18:42.798260+0000","last_clean":"2024-05-08T23:18:42.798260+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:42.798260+0000","last_undegraded":"2024-05-08T23:18:42.798260+0000","last_fullsized":"2024-05-08T23:18:42.798260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":70,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:56.765160+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:18:56.765160+0000","last_peered":"2024-05-08T23:18:56.765160+0000","last_clean":"2024-05-08T23:18:56.765160+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:18:56.765160+0000","last_undegraded":"2024-05-08T23:18:56.765160+0000","last_fullsized":"2024-05-08T23:18:56.765160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796860+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:18:42.796860+0000","last_peered":"2024-05-08T23:18:42.796860+0000","last_clean":"2024-05-08T23:18:42.796860+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:18:42.796860+0000","last_undegraded":"2024-05-08T23:18:42.796860+0000","last_fullsized":"2024-05-08T23:18:42.796860+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803558+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803558+0000","last_peered":"2024-05-08T23:18:43.803558+0000","last_clean":"2024-05-08T23:18:43.803558+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803558+0000","last_undegraded":"2024-05-08T23:18:43.803558+0000","last_fullsized":"2024-05-08T23:18:43.803558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796303+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:42.796303+0000","last_peered":"2024-05-08T23:18:42.796303+0000","last_clean":"2024-05-08T23:18:42.796303+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:42.796303+0000","last_undegraded":"2024-05-08T23:18:42.796303+0000","last_fullsized":"2024-05-08T23:18:42.796303+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798231+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:18:42.798231+0000","last_peered":"2024-05-08T23:18:42.798231+0000","last_clean":"2024-05-08T23:18:42.798231+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:18:42.798231+0000","last_undegraded":"2024-05-08T23:18:42.798231+0000","last_fullsized":"2024-05-08T23:18:42.798231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801918+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:42.801918+0000","last_peered":"2024-05-08T23:18:42.801918+0000","last_clean":"2024-05-08T23:18:42.801918+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:42.801918+0000","last_undegraded":"2024-05-08T23:18:42.801918+0000","last_fullsized":"2024-05-08T23:18:42.801918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799336+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:42.799336+0000","last_peered":"2024-05-08T23:18:42.799336+0000","last_clean":"2024-05-08T23:18:42.799336+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:42.799336+0000","last_undegraded":"2024-05-08T23:18:42.799336+0000","last_fullsized":"2024-05-08T23:18:42.799336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795875+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:18:42.795875+0000","last_peered":"2024-05-08T23:18:42.795875+0000","last_clean":"2024-05-08T23:18:42.795875+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:18:42.795875+0000","last_undegraded":"2024-05-08T23:18:42.795875+0000","last_fullsized":"2024-05-08T23:18:42.795875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850309+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:42.850309+0000","last_peered":"2024-05-08T23:18:42.850309+0000","last_clean":"2024-05-08T23:18:42.850309+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:42.850309+0000","last_undegraded":"2024-05-08T23:18:42.850309+0000","last_fullsized":"2024-05-08T23:18:42.850309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797842+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:42.797842+0000","last_peered":"2024-05-08T23:18:42.797842+0000","last_clean":"2024-05-08T23:18:42.797842+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:42.797842+0000","last_undegraded":"2024-05-08T23:18:42.797842+0000","last_fullsized":"2024-05-08T23:18:42.797842+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792376+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:18:42.792376+0000","last_peered":"2024-05-08T23:18:42.792376+0000","last_clean":"2024-05-08T23:18:42.792376+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:18:42.792376+0000","last_undegraded":"2024-05-08T23:18:42.792376+0000","last_fullsized":"2024-05-08T23:18:42.792376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795976+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:42.795976+0000","last_peered":"2024-05-08T23:18:42.795976+0000","last_clean":"2024-05-08T23:18:42.795976+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:42.795976+0000","last_undegraded":"2024-05-08T23:18:42.795976+0000","last_fullsized":"2024-05-08T23:18:42.795976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:55.763298+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:18:55.763298+0000","last_peered":"2024-05-08T23:18:55.763298+0000","last_clean":"2024-05-08T23:18:55.763298+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:18:55.763298+0000","last_undegraded":"2024-05-08T23:18:55.763298+0000","last_fullsized":"2024-05-08T23:18:55.763298+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800494+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:18:42.800494+0000","last_peered":"2024-05-08T23:18:42.800494+0000","last_clean":"2024-05-08T23:18:42.800494+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:18:42.800494+0000","last_undegraded":"2024-05-08T23:18:42.800494+0000","last_fullsized":"2024-05-08T23:18:42.800494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792335+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:42.792335+0000","last_peered":"2024-05-08T23:18:42.792335+0000","last_clean":"2024-05-08T23:18:42.792335+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:42.792335+0000","last_undegraded":"2024-05-08T23:18:42.792335+0000","last_fullsized":"2024-05-08T23:18:42.792335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797044+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:42.797044+0000","last_peered":"2024-05-08T23:18:42.797044+0000","last_clean":"2024-05-08T23:18:42.797044+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:42.797044+0000","last_undegraded":"2024-05-08T23:18:42.797044+0000","last_fullsized":"2024-05-08T23:18:42.797044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850542+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:18:42.850542+0000","last_peered":"2024-05-08T23:18:42.850542+0000","last_clean":"2024-05-08T23:18:42.850542+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:18:42.850542+0000","last_undegraded":"2024-05-08T23:18:42.850542+0000","last_fullsized":"2024-05-08T23:18:42.850542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793231+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:42.793231+0000","last_peered":"2024-05-08T23:18:42.793231+0000","last_clean":"2024-05-08T23:18:42.793231+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:42.793231+0000","last_undegraded":"2024-05-08T23:18:42.793231+0000","last_fullsized":"2024-05-08T23:18:42.793231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797201+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:42.797201+0000","last_peered":"2024-05-08T23:18:42.797201+0000","last_clean":"2024-05-08T23:18:42.797201+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:42.797201+0000","last_undegraded":"2024-05-08T23:18:42.797201+0000","last_fullsized":"2024-05-08T23:18:42.797201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800205+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:18:42.800205+0000","last_peered":"2024-05-08T23:18:42.800205+0000","last_clean":"2024-05-08T23:18:42.800205+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:18:42.800205+0000","last_undegraded":"2024-05-08T23:18:42.800205+0000","last_fullsized":"2024-05-08T23:18:42.800205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792502+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:42.792502+0000","last_peered":"2024-05-08T23:18:42.792502+0000","last_clean":"2024-05-08T23:18:42.792502+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:42.792502+0000","last_undegraded":"2024-05-08T23:18:42.792502+0000","last_fullsized":"2024-05-08T23:18:42.792502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:56.764474+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:18:56.764474+0000","last_peered":"2024-05-08T23:18:56.764474+0000","last_clean":"2024-05-08T23:18:56.764474+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:18:56.764474+0000","last_undegraded":"2024-05-08T23:18:56.764474+0000","last_fullsized":"2024-05-08T23:18:56.764474+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61146,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":98,"num_read_kb":109,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797717+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:18:42.797717+0000","last_peered":"2024-05-08T23:18:42.797717+0000","last_clean":"2024-05-08T23:18:42.797717+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:18:42.797717+0000","last_undegraded":"2024-05-08T23:18:42.797717+0000","last_fullsized":"2024-05-08T23:18:42.797717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797660+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:18:42.797660+0000","last_peered":"2024-05-08T23:18:42.797660+0000","last_clean":"2024-05-08T23:18:42.797660+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:18:42.797660+0000","last_undegraded":"2024-05-08T23:18:42.797660+0000","last_fullsized":"2024-05-08T23:18:42.797660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798509+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:42.798509+0000","last_peered":"2024-05-08T23:18:42.798509+0000","last_clean":"2024-05-08T23:18:42.798509+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:42.798509+0000","last_undegraded":"2024-05-08T23:18:42.798509+0000","last_fullsized":"2024-05-08T23:18:42.798509+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799523+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:18:42.799523+0000","last_peered":"2024-05-08T23:18:42.799523+0000","last_clean":"2024-05-08T23:18:42.799523+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:18:42.799523+0000","last_undegraded":"2024-05-08T23:18:42.799523+0000","last_fullsized":"2024-05-08T23:18:42.799523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801502+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:42.801502+0000","last_peered":"2024-05-08T23:18:42.801502+0000","last_clean":"2024-05-08T23:18:42.801502+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:42.801502+0000","last_undegraded":"2024-05-08T23:18:42.801502+0000","last_fullsized":"2024-05-08T23:18:42.801502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798399+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:42.798399+0000","last_peered":"2024-05-08T23:18:42.798399+0000","last_clean":"2024-05-08T23:18:42.798399+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:42.798399+0000","last_undegraded":"2024-05-08T23:18:42.798399+0000","last_fullsized":"2024-05-08T23:18:42.798399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034932+0000","last_change":"2024-05-08T23:18:44.034932+0000","last_active":"2024-05-08T23:18:44.034932+0000","last_peered":"2024-05-08T23:18:44.034932+0000","last_clean":"2024-05-08T23:18:44.034932+0000","last_became_active":"2024-05-08T23:18:44.034439+0000","last_became_peered":"2024-05-08T23:18:44.034439+0000","last_unstale":"2024-05-08T23:18:44.034932+0000","last_undegraded":"2024-05-08T23:18:44.034932+0000","last_fullsized":"2024-05-08T23:18:44.034932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.536646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793500+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:42.793500+0000","last_peered":"2024-05-08T23:18:42.793500+0000","last_clean":"2024-05-08T23:18:42.793500+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:42.793500+0000","last_undegraded":"2024-05-08T23:18:42.793500+0000","last_fullsized":"2024-05-08T23:18:42.793500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:54.762805+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:18:54.762805+0000","last_peered":"2024-05-08T23:18:54.762805+0000","last_clean":"2024-05-08T23:18:54.762805+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:18:54.762805+0000","last_undegraded":"2024-05-08T23:18:54.762805+0000","last_fullsized":"2024-05-08T23:18:54.762805+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797290+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:18:42.797290+0000","last_peered":"2024-05-08T23:18:42.797290+0000","last_clean":"2024-05-08T23:18:42.797290+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:18:42.797290+0000","last_undegraded":"2024-05-08T23:18:42.797290+0000","last_fullsized":"2024-05-08T23:18:42.797290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800355+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:42.800355+0000","last_peered":"2024-05-08T23:18:42.800355+0000","last_clean":"2024-05-08T23:18:42.800355+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:42.800355+0000","last_undegraded":"2024-05-08T23:18:42.800355+0000","last_fullsized":"2024-05-08T23:18:42.800355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796252+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:42.796252+0000","last_peered":"2024-05-08T23:18:42.796252+0000","last_clean":"2024-05-08T23:18:42.796252+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:42.796252+0000","last_undegraded":"2024-05-08T23:18:42.796252+0000","last_fullsized":"2024-05-08T23:18:42.796252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.929400+0000","last_change":"2024-05-08T23:18:43.929400+0000","last_active":"2024-05-08T23:18:43.929400+0000","last_peered":"2024-05-08T23:18:43.929400+0000","last_clean":"2024-05-08T23:18:43.929400+0000","last_became_active":"2024-05-08T23:18:43.928980+0000","last_became_peered":"2024-05-08T23:18:43.928980+0000","last_unstale":"2024-05-08T23:18:43.929400+0000","last_undegraded":"2024-05-08T23:18:43.929400+0000","last_fullsized":"2024-05-08T23:18:43.929400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:10.433725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797719+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:18:42.797719+0000","last_peered":"2024-05-08T23:18:42.797719+0000","last_clean":"2024-05-08T23:18:42.797719+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:18:42.797719+0000","last_undegraded":"2024-05-08T23:18:42.797719+0000","last_fullsized":"2024-05-08T23:18:42.797719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.798692+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:42.798692+0000","last_peered":"2024-05-08T23:18:42.798692+0000","last_clean":"2024-05-08T23:18:42.798692+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:42.798692+0000","last_undegraded":"2024-05-08T23:18:42.798692+0000","last_fullsized":"2024-05-08T23:18:42.798692+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802290+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:18:42.802290+0000","last_peered":"2024-05-08T23:18:42.802290+0000","last_clean":"2024-05-08T23:18:42.802290+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:18:42.802290+0000","last_undegraded":"2024-05-08T23:18:42.802290+0000","last_fullsized":"2024-05-08T23:18:42.802290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796770+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:42.796770+0000","last_peered":"2024-05-08T23:18:42.796770+0000","last_clean":"2024-05-08T23:18:42.796770+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:42.796770+0000","last_undegraded":"2024-05-08T23:18:42.796770+0000","last_fullsized":"2024-05-08T23:18:42.796770+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792981+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:42.792981+0000","last_peered":"2024-05-08T23:18:42.792981+0000","last_clean":"2024-05-08T23:18:42.792981+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:42.792981+0000","last_undegraded":"2024-05-08T23:18:42.792981+0000","last_fullsized":"2024-05-08T23:18:42.792981+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796821+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:18:42.796821+0000","last_peered":"2024-05-08T23:18:42.796821+0000","last_clean":"2024-05-08T23:18:42.796821+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:18:42.796821+0000","last_undegraded":"2024-05-08T23:18:42.796821+0000","last_fullsized":"2024-05-08T23:18:42.796821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799638+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:42.799638+0000","last_peered":"2024-05-08T23:18:42.799638+0000","last_clean":"2024-05-08T23:18:42.799638+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:42.799638+0000","last_undegraded":"2024-05-08T23:18:42.799638+0000","last_fullsized":"2024-05-08T23:18:42.799638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796478+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:42.796478+0000","last_peered":"2024-05-08T23:18:42.796478+0000","last_clean":"2024-05-08T23:18:42.796478+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:42.796478+0000","last_undegraded":"2024-05-08T23:18:42.796478+0000","last_fullsized":"2024-05-08T23:18:42.796478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796378+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:18:42.796378+0000","last_peered":"2024-05-08T23:18:42.796378+0000","last_clean":"2024-05-08T23:18:42.796378+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:18:42.796378+0000","last_undegraded":"2024-05-08T23:18:42.796378+0000","last_fullsized":"2024-05-08T23:18:42.796378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799747+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:42.799747+0000","last_peered":"2024-05-08T23:18:42.799747+0000","last_clean":"2024-05-08T23:18:42.799747+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:42.799747+0000","last_undegraded":"2024-05-08T23:18:42.799747+0000","last_fullsized":"2024-05-08T23:18:42.799747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797943+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:18:42.797943+0000","last_peered":"2024-05-08T23:18:42.797943+0000","last_clean":"2024-05-08T23:18:42.797943+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:18:42.797943+0000","last_undegraded":"2024-05-08T23:18:42.797943+0000","last_fullsized":"2024-05-08T23:18:42.797943+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:55.763390+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:18:55.763390+0000","last_peered":"2024-05-08T23:18:55.763390+0000","last_clean":"2024-05-08T23:18:55.763390+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:18:55.763390+0000","last_undegraded":"2024-05-08T23:18:55.763390+0000","last_fullsized":"2024-05-08T23:18:55.763390+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796463+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:42.796463+0000","last_peered":"2024-05-08T23:18:42.796463+0000","last_clean":"2024-05-08T23:18:42.796463+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:42.796463+0000","last_undegraded":"2024-05-08T23:18:42.796463+0000","last_fullsized":"2024-05-08T23:18:42.796463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796295+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:18:42.796295+0000","last_peered":"2024-05-08T23:18:42.796295+0000","last_clean":"2024-05-08T23:18:42.796295+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:18:42.796295+0000","last_undegraded":"2024-05-08T23:18:42.796295+0000","last_fullsized":"2024-05-08T23:18:42.796295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793103+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:42.793103+0000","last_peered":"2024-05-08T23:18:42.793103+0000","last_clean":"2024-05-08T23:18:42.793103+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:42.793103+0000","last_undegraded":"2024-05-08T23:18:42.793103+0000","last_fullsized":"2024-05-08T23:18:42.793103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":335,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796517+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:42.796517+0000","last_peered":"2024-05-08T23:18:42.796517+0000","last_clean":"2024-05-08T23:18:42.796517+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:42.796517+0000","last_undegraded":"2024-05-08T23:18:42.796517+0000","last_fullsized":"2024-05-08T23:18:42.796517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796867+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:18:42.796867+0000","last_peered":"2024-05-08T23:18:42.796867+0000","last_clean":"2024-05-08T23:18:42.796867+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:18:42.796867+0000","last_undegraded":"2024-05-08T23:18:42.796867+0000","last_fullsized":"2024-05-08T23:18:42.796867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799236+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:18:42.799236+0000","last_peered":"2024-05-08T23:18:42.799236+0000","last_clean":"2024-05-08T23:18:42.799236+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:18:42.799236+0000","last_undegraded":"2024-05-08T23:18:42.799236+0000","last_fullsized":"2024-05-08T23:18:42.799236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796699+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:42.796699+0000","last_peered":"2024-05-08T23:18:42.796699+0000","last_clean":"2024-05-08T23:18:42.796699+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:42.796699+0000","last_undegraded":"2024-05-08T23:18:42.796699+0000","last_fullsized":"2024-05-08T23:18:42.796699+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.802361+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:42.802361+0000","last_peered":"2024-05-08T23:18:42.802361+0000","last_clean":"2024-05-08T23:18:42.802361+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:42.802361+0000","last_undegraded":"2024-05-08T23:18:42.802361+0000","last_fullsized":"2024-05-08T23:18:42.802361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797799+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:42.797799+0000","last_peered":"2024-05-08T23:18:42.797799+0000","last_clean":"2024-05-08T23:18:42.797799+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:42.797799+0000","last_undegraded":"2024-05-08T23:18:42.797799+0000","last_fullsized":"2024-05-08T23:18:42.797799+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799837+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:18:42.799837+0000","last_peered":"2024-05-08T23:18:42.799837+0000","last_clean":"2024-05-08T23:18:42.799837+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:18:42.799837+0000","last_undegraded":"2024-05-08T23:18:42.799837+0000","last_fullsized":"2024-05-08T23:18:42.799837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.795769+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:42.795769+0000","last_peered":"2024-05-08T23:18:42.795769+0000","last_clean":"2024-05-08T23:18:42.795769+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:42.795769+0000","last_undegraded":"2024-05-08T23:18:42.795769+0000","last_fullsized":"2024-05-08T23:18:42.795769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"81'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793582+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:18:42.793582+0000","last_peered":"2024-05-08T23:18:42.793582+0000","last_clean":"2024-05-08T23:18:42.793582+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:18:42.793582+0000","last_undegraded":"2024-05-08T23:18:42.793582+0000","last_fullsized":"2024-05-08T23:18:42.793582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797388+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:18:42.797388+0000","last_peered":"2024-05-08T23:18:42.797388+0000","last_clean":"2024-05-08T23:18:42.797388+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:18:42.797388+0000","last_undegraded":"2024-05-08T23:18:42.797388+0000","last_fullsized":"2024-05-08T23:18:42.797388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797897+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:42.797897+0000","last_peered":"2024-05-08T23:18:42.797897+0000","last_clean":"2024-05-08T23:18:42.797897+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:42.797897+0000","last_undegraded":"2024-05-08T23:18:42.797897+0000","last_fullsized":"2024-05-08T23:18:42.797897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793553+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:42.793553+0000","last_peered":"2024-05-08T23:18:42.793553+0000","last_clean":"2024-05-08T23:18:42.793553+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:42.793553+0000","last_undegraded":"2024-05-08T23:18:42.793553+0000","last_fullsized":"2024-05-08T23:18:42.793553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850431+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:18:42.850431+0000","last_peered":"2024-05-08T23:18:42.850431+0000","last_clean":"2024-05-08T23:18:42.850431+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:18:42.850431+0000","last_undegraded":"2024-05-08T23:18:42.850431+0000","last_fullsized":"2024-05-08T23:18:42.850431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793223+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:42.793223+0000","last_peered":"2024-05-08T23:18:42.793223+0000","last_clean":"2024-05-08T23:18:42.793223+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:42.793223+0000","last_undegraded":"2024-05-08T23:18:42.793223+0000","last_fullsized":"2024-05-08T23:18:42.793223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799949+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:42.799949+0000","last_peered":"2024-05-08T23:18:42.799949+0000","last_clean":"2024-05-08T23:18:42.799949+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:42.799949+0000","last_undegraded":"2024-05-08T23:18:42.799949+0000","last_fullsized":"2024-05-08T23:18:42.799949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796359+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:18:42.796359+0000","last_peered":"2024-05-08T23:18:42.796359+0000","last_clean":"2024-05-08T23:18:42.796359+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:18:42.796359+0000","last_undegraded":"2024-05-08T23:18:42.796359+0000","last_fullsized":"2024-05-08T23:18:42.796359+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793663+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:42.793663+0000","last_peered":"2024-05-08T23:18:42.793663+0000","last_clean":"2024-05-08T23:18:42.793663+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:42.793663+0000","last_undegraded":"2024-05-08T23:18:42.793663+0000","last_fullsized":"2024-05-08T23:18:42.793663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796095+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:42.796095+0000","last_peered":"2024-05-08T23:18:42.796095+0000","last_clean":"2024-05-08T23:18:42.796095+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:42.796095+0000","last_undegraded":"2024-05-08T23:18:42.796095+0000","last_fullsized":"2024-05-08T23:18:42.796095+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801296+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:18:42.801296+0000","last_peered":"2024-05-08T23:18:42.801296+0000","last_clean":"2024-05-08T23:18:42.801296+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:18:42.801296+0000","last_undegraded":"2024-05-08T23:18:42.801296+0000","last_fullsized":"2024-05-08T23:18:42.801296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.792930+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:42.792930+0000","last_peered":"2024-05-08T23:18:42.792930+0000","last_clean":"2024-05-08T23:18:42.792930+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:42.792930+0000","last_undegraded":"2024-05-08T23:18:42.792930+0000","last_fullsized":"2024-05-08T23:18:42.792930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799130+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:42.799130+0000","last_peered":"2024-05-08T23:18:42.799130+0000","last_clean":"2024-05-08T23:18:42.799130+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:42.799130+0000","last_undegraded":"2024-05-08T23:18:42.799130+0000","last_fullsized":"2024-05-08T23:18:42.799130+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796367+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:18:42.796367+0000","last_peered":"2024-05-08T23:18:42.796367+0000","last_clean":"2024-05-08T23:18:42.796367+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:18:42.796367+0000","last_undegraded":"2024-05-08T23:18:42.796367+0000","last_fullsized":"2024-05-08T23:18:42.796367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.793402+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:42.793402+0000","last_peered":"2024-05-08T23:18:42.793402+0000","last_clean":"2024-05-08T23:18:42.793402+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:42.793402+0000","last_undegraded":"2024-05-08T23:18:42.793402+0000","last_fullsized":"2024-05-08T23:18:42.793402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.808122+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:43.808122+0000","last_peered":"2024-05-08T23:18:43.808122+0000","last_clean":"2024-05-08T23:18:43.808122+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:43.808122+0000","last_undegraded":"2024-05-08T23:18:43.808122+0000","last_fullsized":"2024-05-08T23:18:43.808122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796533+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:18:42.796533+0000","last_peered":"2024-05-08T23:18:42.796533+0000","last_clean":"2024-05-08T23:18:42.796533+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:18:42.796533+0000","last_undegraded":"2024-05-08T23:18:42.796533+0000","last_fullsized":"2024-05-08T23:18:42.796533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800112+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:42.800112+0000","last_peered":"2024-05-08T23:18:42.800112+0000","last_clean":"2024-05-08T23:18:42.800112+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:42.800112+0000","last_undegraded":"2024-05-08T23:18:42.800112+0000","last_fullsized":"2024-05-08T23:18:42.800112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800360+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:42.800360+0000","last_peered":"2024-05-08T23:18:42.800360+0000","last_clean":"2024-05-08T23:18:42.800360+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:42.800360+0000","last_undegraded":"2024-05-08T23:18:42.800360+0000","last_fullsized":"2024-05-08T23:18:42.800360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797464+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:18:42.797464+0000","last_peered":"2024-05-08T23:18:42.797464+0000","last_clean":"2024-05-08T23:18:42.797464+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:18:42.797464+0000","last_undegraded":"2024-05-08T23:18:42.797464+0000","last_fullsized":"2024-05-08T23:18:42.797464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.849917+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:42.849917+0000","last_peered":"2024-05-08T23:18:42.849917+0000","last_clean":"2024-05-08T23:18:42.849917+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:42.849917+0000","last_undegraded":"2024-05-08T23:18:42.849917+0000","last_fullsized":"2024-05-08T23:18:42.849917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"76'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797406+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:18:42.797406+0000","last_peered":"2024-05-08T23:18:42.797406+0000","last_clean":"2024-05-08T23:18:42.797406+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:18:42.797406+0000","last_undegraded":"2024-05-08T23:18:42.797406+0000","last_fullsized":"2024-05-08T23:18:42.797406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797180+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:18:42.797180+0000","last_peered":"2024-05-08T23:18:42.797180+0000","last_clean":"2024-05-08T23:18:42.797180+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:18:42.797180+0000","last_undegraded":"2024-05-08T23:18:42.797180+0000","last_fullsized":"2024-05-08T23:18:42.797180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796081+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:18:42.796081+0000","last_peered":"2024-05-08T23:18:42.796081+0000","last_clean":"2024-05-08T23:18:42.796081+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:18:42.796081+0000","last_undegraded":"2024-05-08T23:18:42.796081+0000","last_fullsized":"2024-05-08T23:18:42.796081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931402+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931402+0000","last_peered":"2024-05-08T23:18:43.931402+0000","last_clean":"2024-05-08T23:18:43.931402+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931402+0000","last_undegraded":"2024-05-08T23:18:43.931402+0000","last_fullsized":"2024-05-08T23:18:43.931402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796911+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:18:42.796911+0000","last_peered":"2024-05-08T23:18:42.796911+0000","last_clean":"2024-05-08T23:18:42.796911+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:18:42.796911+0000","last_undegraded":"2024-05-08T23:18:42.796911+0000","last_fullsized":"2024-05-08T23:18:42.796911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801392+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:18:42.801392+0000","last_peered":"2024-05-08T23:18:42.801392+0000","last_clean":"2024-05-08T23:18:42.801392+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:18:42.801392+0000","last_undegraded":"2024-05-08T23:18:42.801392+0000","last_fullsized":"2024-05-08T23:18:42.801392+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034679+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:18:44.034679+0000","last_peered":"2024-05-08T23:18:44.034679+0000","last_clean":"2024-05-08T23:18:44.034679+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:18:44.034679+0000","last_undegraded":"2024-05-08T23:18:44.034679+0000","last_fullsized":"2024-05-08T23:18:44.034679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799239+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:18:42.799239+0000","last_peered":"2024-05-08T23:18:42.799239+0000","last_clean":"2024-05-08T23:18:42.799239+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:18:42.799239+0000","last_undegraded":"2024-05-08T23:18:42.799239+0000","last_fullsized":"2024-05-08T23:18:42.799239+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.850099+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:18:42.850099+0000","last_peered":"2024-05-08T23:18:42.850099+0000","last_clean":"2024-05-08T23:18:42.850099+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:18:42.850099+0000","last_undegraded":"2024-05-08T23:18:42.850099+0000","last_fullsized":"2024-05-08T23:18:42.850099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.799941+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:18:42.799941+0000","last_peered":"2024-05-08T23:18:42.799941+0000","last_clean":"2024-05-08T23:18:42.799941+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:18:42.799941+0000","last_undegraded":"2024-05-08T23:18:42.799941+0000","last_fullsized":"2024-05-08T23:18:42.799941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.803347+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:18:42.803347+0000","last_peered":"2024-05-08T23:18:42.803347+0000","last_clean":"2024-05-08T23:18:42.803347+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:18:42.803347+0000","last_undegraded":"2024-05-08T23:18:42.803347+0000","last_fullsized":"2024-05-08T23:18:42.803347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800760+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:18:42.800760+0000","last_peered":"2024-05-08T23:18:42.800760+0000","last_clean":"2024-05-08T23:18:42.800760+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:18:42.800760+0000","last_undegraded":"2024-05-08T23:18:42.800760+0000","last_fullsized":"2024-05-08T23:18:42.800760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797474+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:18:42.797474+0000","last_peered":"2024-05-08T23:18:42.797474+0000","last_clean":"2024-05-08T23:18:42.797474+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:18:42.797474+0000","last_undegraded":"2024-05-08T23:18:42.797474+0000","last_fullsized":"2024-05-08T23:18:42.797474+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.797850+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:18:42.797850+0000","last_peered":"2024-05-08T23:18:42.797850+0000","last_clean":"2024-05-08T23:18:42.797850+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:18:42.797850+0000","last_undegraded":"2024-05-08T23:18:42.797850+0000","last_fullsized":"2024-05-08T23:18:42.797850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.796074+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:18:42.796074+0000","last_peered":"2024-05-08T23:18:42.796074+0000","last_clean":"2024-05-08T23:18:42.796074+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:18:42.796074+0000","last_undegraded":"2024-05-08T23:18:42.796074+0000","last_fullsized":"2024-05-08T23:18:42.796074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800329+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:18:42.800329+0000","last_peered":"2024-05-08T23:18:42.800329+0000","last_clean":"2024-05-08T23:18:42.800329+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:18:42.800329+0000","last_undegraded":"2024-05-08T23:18:42.800329+0000","last_fullsized":"2024-05-08T23:18:42.800329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.800055+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:18:42.800055+0000","last_peered":"2024-05-08T23:18:42.800055+0000","last_clean":"2024-05-08T23:18:42.800055+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:18:42.800055+0000","last_undegraded":"2024-05-08T23:18:42.800055+0000","last_fullsized":"2024-05-08T23:18:42.800055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-08T23:18:42.801597+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:18:42.801597+0000","last_peered":"2024-05-08T23:18:42.801597+0000","last_clean":"2024-05-08T23:18:42.801597+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:18:42.801597+0000","last_undegraded":"2024-05-08T23:18:42.801597+0000","last_fullsized":"2024-05-08T23:18:42.801597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64660,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":294,"num_read_kb":215,"num_write":156,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645337,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":933888,"data_stored":463111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.622}]},{"osd":1,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.749}]},{"osd":2,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.58799999999999997}]},{"osd":5,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.495}]},{"osd":6,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.70599999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.65800000000000003}]},{"osd":8,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.85199999999999998}]},{"osd":9,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78200000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.83699999999999997}]}]},{"osd":10,"up_from":65,"seq":279172874269,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1617920,"data_stored":1111146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.49399999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.66800000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.61499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.42499999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.871,"15min":0.871},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.62}]},{"osd":3,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.62,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.746,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72999999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47899999999999998}]},{"osd":5,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.77200000000000002,"5min":1.169,"15min":1.169},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81599999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.84299999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83699999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.77700000000000002,"5min":1.016,"15min":1.016},"last":0.71599999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79800000000000004,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86199999999999999,"5min":1.3,"15min":1.3},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.76200000000000001,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83699999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68000000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.749}]}]},{"osd":9,"up_from":60,"seq":257698037792,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":467443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.623}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70699999999999996}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.79600000000000004}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.60499999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.83599999999999997}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0660000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.754},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.81200000000000006}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.88100000000000001}]}]},{"osd":8,"up_from":53,"seq":227633266724,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86499999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80800000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.626}]},{"osd":2,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48399999999999999},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.71599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.48499999999999999,"15min":0.48099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96499999999999997,"5min":1.137,"15min":1.137},"last":0.375},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78700000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.64000000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.51200000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.76400000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85999999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.5},{"interface":"front","average":{"1min":0.622,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71199999999999997,"15min":0.70999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92200000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70799999999999996},"min":{"1min":0.48399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96799999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73599999999999999},"min":{"1min":0.503,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93500000000000005,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.67000000000000004}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.83099999999999996}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.93600000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.80400000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.78200000000000003}]}]},{"osd":7,"up_from":46,"seq":197568495655,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1540096,"data_stored":1058358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.58699999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.433,"15min":0.43099999999999999},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61799999999999999,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58099999999999996}]},{"osd":3,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.48099999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.81100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61199999999999999}]},{"osd":4,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73299999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.498},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.36499999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.89100000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.627}]},{"osd":6,"last update":"Wed May 8 23:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.70299999999999996},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69899999999999995}]},{"osd":8,"last update":"Wed May 8 23:18:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.67100000000000004,"15min":0.66000000000000003},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.76800000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86399999999999999,"5min":0.997,"15min":0.997},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83399999999999996}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86799999999999999,"5min":0.999,"15min":0.999},"last":0.86299999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79700000000000004}]}]},{"osd":6,"up_from":42,"seq":180388626475,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36116,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709036,"statfs":{"total":95995035648,"available":95958052864,"internally_reserved":0,"allocated":929792,"data_stored":458523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61199999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed May 8 23:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.92800000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68700000000000006}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.505},{"interface":"front","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0049999999999999,"5min":1.177,"15min":1.177},"last":0.71599999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.75800000000000001,"15min":0.76400000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.046,"5min":1.091,"15min":1.091},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.59699999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.79700000000000004,"15min":0.79000000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.80200000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.748}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":995328,"data_stored":519579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.498,"15min":0.48699999999999999},"min":{"1min":0.41699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70899999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.39000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.40999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.34200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.248,"15min":0.248},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.443}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.59699999999999998,"15min":0.56000000000000005},"min":{"1min":0.46800000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.58299999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.621,"15min":0.59299999999999997},"min":{"1min":0.54700000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.374}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65600000000000003,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68500000000000005,"15min":0.67000000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60099999999999998}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.749}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.64000000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.68100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":954368,"data_stored":459911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.755,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.436}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.41299999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.57299999999999995,"15min":0.54500000000000004},"min":{"1min":0.504,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.628,"15min":0.61299999999999999},"min":{"1min":0.45300000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.55700000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.501},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61799999999999999}]},{"osd":8,"last update":"Wed May 8 23:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.5,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.71699999999999997}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed May 8 23:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.76500000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.78600000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444089,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":520881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.374},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 23:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.57699999999999996,"15min":0.53400000000000003},"min":{"1min":0.48799999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.56899999999999995,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.375,"15min":0.375},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.54500000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55800000000000005}]},{"osd":5,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.437,"15min":0.40899999999999997},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70299999999999996,"5min":0.872,"15min":0.872},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.35799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.753,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.46300000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.58699999999999997},"min":{"1min":0.52900000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.57299999999999995}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55900000000000005,"15min":0.54500000000000004},"min":{"1min":0.38200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65700000000000003}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.55400000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.75},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.67700000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.76300000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.77500000000000002}]}]},{"osd":3,"up_from":23,"seq":98784247861,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":942080,"data_stored":458717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.41599999999999998,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":6.3040000000000003,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.42599999999999999,"15min":0.38100000000000001},"min":{"1min":0.30299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.5049999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.39600000000000002,"5min":0.377,"15min":0.377},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.52500000000000002}]},{"osd":2,"last update":"Wed May 8 23:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.61699999999999999,"15min":0.56699999999999995},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed May 8 23:18:09 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":5.9630000000000001,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.66000000000000003,"15min":0.61199999999999999},"min":{"1min":0.51500000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.5350000000000001,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.61399999999999999}]},{"osd":5,"last update":"Wed May 8 23:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.66900000000000004,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":5.3769999999999998,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.66300000000000003}]},{"osd":6,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.624,"15min":0.58799999999999997},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91900000000000004,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.58199999999999996}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52700000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.63},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70799999999999996,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.69399999999999995}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:18:35 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.80300000000000005,"15min":0.77700000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.745},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Wed May 8 23:18:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.92200000000000004,"15min":0.94799999999999995},"min":{"1min":0.65300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.135,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.97899999999999998,"15min":1.0049999999999999},"min":{"1min":0.67800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.171,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.70899999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.77600000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018929,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":463125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.29299999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.59699999999999998,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38,"15min":0.36199999999999999},"min":{"1min":0.29999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.434,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.751},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.46500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75800000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.626}]},{"osd":3,"last update":"Wed May 8 23:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54900000000000004,"15min":0.52500000000000002},"min":{"1min":0.48699999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.56100000000000005}]},{"osd":5,"last update":"Wed May 8 23:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.56599999999999995,"15min":0.54700000000000004},"min":{"1min":0.47799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.60299999999999998,"15min":0.57999999999999996},"min":{"1min":0.51100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.318,"15min":0.318},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47799999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.56899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63900000000000001,"15min":0.63200000000000001},"min":{"1min":0.54600000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83399999999999996,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69399999999999995}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64000000000000001}]},{"osd":10,"last update":"Wed May 8 23:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.61199999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72399999999999998}]}]},{"osd":5,"up_from":35,"seq":150323855406,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1552384,"data_stored":1054325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73099999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71799999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.33800000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.34699999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.376},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.502}]},{"osd":2,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.30099999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.52500000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.26900000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.27900000000000003}]},{"osd":3,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.56599999999999995,"15min":0.58799999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.754,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.44900000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:27 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.39100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed May 8 23:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65200000000000002,"15min":0.65300000000000002},"min":{"1min":0.45300000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.64200000000000002,"15min":0.64400000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41399999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.62,"15min":0.621},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.75800000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53300000000000003}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65300000000000002}]},{"osd":10,"last update":"Wed May 8 23:18:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.56200000000000006,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.56200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:19:01.745 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T23:19:01.745 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T23:19:01.745 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T23:19:01.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T23:19:02.005 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:02.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:01 smithi016 ceph-mon[42541]: pgmap v262: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:02.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:01 smithi189 ceph-mon[46919]: pgmap v262: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:02.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:01 smithi155 ceph-mon[47105]: pgmap v262: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 -- 172.21.15.16:0/3522903522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101c00 msgr2=0x7f4568105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 --2- 172.21.15.16:0/3522903522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101c00 0x7f4568105bd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4550009990 tx=0x7f4550009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 -- 172.21.15.16:0/3522903522 shutdown_connections 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 --2- 172.21.15.16:0/3522903522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45681062b0 0x7f456810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 --2- 172.21.15.16:0/3522903522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4568101250 0x7f4568101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 --2- 172.21.15.16:0/3522903522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101c00 0x7f4568105bd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 -- 172.21.15.16:0/3522903522 >> 172.21.15.16:0/3522903522 conn(0x7f4568078100 msgr2=0x7f4568078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 -- 172.21.15.16:0/3522903522 shutdown_connections 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.019+0000 7f456e68f700 1 -- 172.21.15.16:0/3522903522 wait complete. 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 Processor -- start 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 -- start start 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 0x7f45680732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4568101c00 0x7f45680737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45681062b0 0x7f4568073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681a5760 con 0x7f4568101250 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681a58d0 con 0x7f45681062b0 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f456e68f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681a5a90 con 0x7f4568101c00 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.020+0000 7f4567fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 0x7f45680732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 0x7f45680732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52562/0 (socket says 172.21.15.16:52562) 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 -- 172.21.15.16:0/3298559290 learned_addr learned my addr 172.21.15.16:0/3298559290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f45677fe700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4568101c00 0x7f45680737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f456cc2c700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45681062b0 0x7f4568073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 -- 172.21.15.16:0/3298559290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4568101c00 msgr2=0x7f45680737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4568101c00 0x7f45680737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 -- 172.21.15.16:0/3298559290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45681062b0 msgr2=0x7f4568073d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45681062b0 0x7f4568073d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 -- 172.21.15.16:0/3298559290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500095f0 con 0x7f4568101250 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f4567fff700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 0x7f45680732a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f455800d950 tx=0x7f455800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800f890 con 0x7f4568101250 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f455800fe80 con 0x7f4568101250 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45681a5d70 con 0x7f4568101250 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800e3b0 con 0x7f4568101250 2024-05-08T23:19:03.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.021+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45681a6230 con 0x7f4568101250 2024-05-08T23:19:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.022+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45681095d0 con 0x7f4568101250 2024-05-08T23:19:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.024+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4558010460 con 0x7f4568101250 2024-05-08T23:19:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.024+0000 7f45657fa700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f45540a1230 0x7f45540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.024+0000 7f45677fe700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f45540a1230 0x7f45540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.024+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f45580ccaa0 con 0x7f4568101250 2024-05-08T23:19:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.025+0000 7f45677fe700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f45540a1230 0x7f45540a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f455000b5c0 tx=0x7f4550005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.028+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4558095dd0 con 0x7f4568101250 2024-05-08T23:19:03.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:02 smithi016 ceph-mon[42541]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:03.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:02 smithi189 ceph-mon[46919]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:03.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:02 smithi155 ceph-mon[47105]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.286+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f45681026b0 con 0x7f4568101250 2024-05-08T23:19:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.286+0000 7f45657fa700 1 -- 172.21.15.16:0/3298559290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4558016080 con 0x7f4568101250 2024-05-08T23:19:03.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:03.287 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T23:19:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f45540a1230 msgr2=0x7f45540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f45540a1230 0x7f45540a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f455000b5c0 tx=0x7f4550005fb0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 msgr2=0x7f45680732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 0x7f45680732a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f455800d950 tx=0x7f455800dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 shutdown_connections 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f45681062b0 0x7f4568073d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568101250 0x7f45680732a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f45540a1230 0x7f45540a36f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 --2- 172.21.15.16:0/3298559290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4568101c00 0x7f45680737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 >> 172.21.15.16:0/3298559290 conn(0x7f4568078100 msgr2=0x7f45680ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 shutdown_connections 2024-05-08T23:19:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.289+0000 7f456e68f700 1 -- 172.21.15.16:0/3298559290 wait complete. 2024-05-08T23:19:03.675 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T23:19:03.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 -- 172.21.15.16:0/4267130081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 msgr2=0x7f7d801081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 --2- 172.21.15.16:0/4267130081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801081a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800b0d0 tx=0x7f7d7800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 -- 172.21.15.16:0/4267130081 shutdown_connections 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 --2- 172.21.15.16:0/4267130081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801081a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 --2- 172.21.15.16:0/4267130081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800ffd60 0x7f7d80100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 --2- 172.21.15.16:0/4267130081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 0x7f7d801013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.823+0000 7f7d87f38700 1 -- 172.21.15.16:0/4267130081 >> 172.21.15.16:0/4267130081 conn(0x7f7d800fb2f0 msgr2=0x7f7d800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.824+0000 7f7d87f38700 1 -- 172.21.15.16:0/4267130081 shutdown_connections 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.824+0000 7f7d87f38700 1 -- 172.21.15.16:0/4267130081 wait complete. 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.824+0000 7f7d87f38700 1 Processor -- start 2024-05-08T23:19:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.824+0000 7f7d87f38700 1 -- start start 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800ffd60 0x7f7d8010a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 0x7f7d80108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80109990 con 0x7f7d801018f0 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80109b00 con 0x7f7d800ffd60 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d801a0e90 con 0x7f7d80100f50 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52580/0 (socket says 172.21.15.16:52580) 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 -- 172.21.15.16:0/3306113418 learned_addr learned my addr 172.21.15.16:0/3306113418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d85cd4700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800ffd60 0x7f7d8010a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d854d3700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 0x7f7d80108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 -- 172.21.15.16:0/3306113418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 msgr2=0x7f7d80108cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 0x7f7d80108cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 -- 172.21.15.16:0/3306113418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800ffd60 msgr2=0x7f7d8010a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800ffd60 0x7f7d8010a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 -- 172.21.15.16:0/3306113418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009d00 con 0x7f7d801018f0 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d854d3700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 0x7f7d80108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d864d5700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801091f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007b60 tx=0x7f7d780094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800e040 con 0x7f7d801018f0 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d78012e10 con 0x7f7d801018f0 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.825+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d801a10c0 con 0x7f7d801018f0 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.826+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7801b450 con 0x7f7d801018f0 2024-05-08T23:19:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.826+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d801a14e0 con 0x7f7d801018f0 2024-05-08T23:19:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.826+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d801071e0 con 0x7f7d801018f0 2024-05-08T23:19:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.828+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7d78012980 con 0x7f7d801018f0 2024-05-08T23:19:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.828+0000 7f7d76ffd700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7d6c0a11b0 0x7f7d6c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.828+0000 7f7d85cd4700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7d6c0a11b0 0x7f7d6c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.829+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7d780cf120 con 0x7f7d801018f0 2024-05-08T23:19:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.829+0000 7f7d85cd4700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7d6c0a11b0 0x7f7d6c0a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c005950 tx=0x7f7d7c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:03.832+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d78098480 con 0x7f7d801018f0 2024-05-08T23:19:03.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:03 smithi016 ceph-mon[42541]: pgmap v263: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:03.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:03 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3298559290' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:19:04.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.053+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d8010c590 con 0x7f7d801018f0 2024-05-08T23:19:04.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.055+0000 7f7d76ffd700 1 -- 172.21.15.16:0/3306113418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24470 (secure 0 0 0) 0x7f7d7809baa0 con 0x7f7d801018f0 2024-05-08T23:19:04.056 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:04.056 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:18:43.790069+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7d6c0a11b0 msgr2=0x7f7d6c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7d6c0a11b0 0x7f7d6c0a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c005950 tx=0x7f7d7c0058e0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 msgr2=0x7f7d801091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801091f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007b60 tx=0x7f7d780094f0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 shutdown_connections 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d801018f0 0x7f7d801091f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d800ffd60 0x7f7d8010a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7d6c0a11b0 0x7f7d6c0a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 --2- 172.21.15.16:0/3306113418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d80100f50 0x7f7d80108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.057+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 >> 172.21.15.16:0/3306113418 conn(0x7f7d800fb2f0 msgr2=0x7f7d80104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.058+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 shutdown_connections 2024-05-08T23:19:04.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.058+0000 7f7d87f38700 1 -- 172.21.15.16:0/3306113418 wait complete. 2024-05-08T23:19:04.073 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:04.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:04.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:03 smithi189 ceph-mon[46919]: pgmap v263: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:04.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:03 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3298559290' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:19:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 -- 172.21.15.16:0/540563079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 msgr2=0x7f752c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 --2- 172.21.15.16:0/540563079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c1002a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7520009990 tx=0x7f7520009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 -- 172.21.15.16:0/540563079 shutdown_connections 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 --2- 172.21.15.16:0/540563079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c101990 0x7f752c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 --2- 172.21.15.16:0/540563079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c1002a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 --2- 172.21.15.16:0/540563079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f752c100fd0 0x7f752c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 -- 172.21.15.16:0/540563079 >> 172.21.15.16:0/540563079 conn(0x7f752c0fb3f0 msgr2=0x7f752c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 -- 172.21.15.16:0/540563079 shutdown_connections 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.213+0000 7f75331ce700 1 -- 172.21.15.16:0/540563079 wait complete. 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 Processor -- start 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 -- start start 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c100fd0 0x7f752c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f752c101990 0x7f752c19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19e8d0 con 0x7f752c0ffe80 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19ea40 con 0x7f752c101990 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f75331ce700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19ec00 con 0x7f752c100fd0 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c069cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52598/0 (socket says 172.21.15.16:52598) 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 -- 172.21.15.16:0/1972422886 learned_addr learned my addr 172.21.15.16:0/1972422886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f753176b700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f752c101990 0x7f752c19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f752bfff700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c100fd0 0x7f752c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 -- 172.21.15.16:0/1972422886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c100fd0 msgr2=0x7f752c19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c100fd0 0x7f752c19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 -- 172.21.15.16:0/1972422886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f752c101990 msgr2=0x7f752c19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f752c101990 0x7f752c19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f7530f6a700 1 -- 172.21.15.16:0/1972422886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200095f0 con 0x7f752c0ffe80 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.214+0000 7f752bfff700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c100fd0 0x7f752c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.215+0000 7f7530f6a700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c069cf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75200059d0 tx=0x7f7520005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.215+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752001d070 con 0x7f752c0ffe80 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.215+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f752000be50 con 0x7f752c0ffe80 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.215+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c19ee80 con 0x7f752c0ffe80 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.215+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520022e90 con 0x7f752c0ffe80 2024-05-08T23:19:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.215+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c19f240 con 0x7f752c0ffe80 2024-05-08T23:19:04.220 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:03 smithi155 ceph-mon[47105]: pgmap v263: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:04.220 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:03 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3298559290' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T23:19:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.216+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c107360 con 0x7f752c0ffe80 2024-05-08T23:19:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.217+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7520022690 con 0x7f752c0ffe80 2024-05-08T23:19:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.218+0000 7f7529ffb700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f751c0a11b0 0x7f751c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.218+0000 7f752bfff700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f751c0a11b0 0x7f751c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.218+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f75200ce4a0 con 0x7f752c0ffe80 2024-05-08T23:19:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.219+0000 7f752bfff700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f751c0a11b0 0x7f751c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f752c06a250 tx=0x7f7518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.222+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7520097860 con 0x7f752c0ffe80 2024-05-08T23:19:04.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.441+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f752c10b660 con 0x7f752c0ffe80 2024-05-08T23:19:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.441+0000 7f7529ffb700 1 -- 172.21.15.16:0/1972422886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f752009ae80 con 0x7f752c0ffe80 2024-05-08T23:19:04.442 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f751c0a11b0 msgr2=0x7f751c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f751c0a11b0 0x7f751c0a3670 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f752c06a250 tx=0x7f7518009260 comp rx=0 tx=0).stop 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 msgr2=0x7f752c069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c069cf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75200059d0 tx=0x7f7520005e80 comp rx=0 tx=0).stop 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 shutdown_connections 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f752c101990 0x7f752c19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c0ffe80 0x7f752c069cf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f751c0a11b0 0x7f751c0a3670 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 --2- 172.21.15.16:0/1972422886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f752c100fd0 0x7f752c19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 >> 172.21.15.16:0/1972422886 conn(0x7f752c0fb3f0 msgr2=0x7f752c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 shutdown_connections 2024-05-08T23:19:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.444+0000 7f75331ce700 1 -- 172.21.15.16:0/1972422886 wait complete. 2024-05-08T23:19:04.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 -- 172.21.15.16:0/442239703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4100fb0 msgr2=0x7f1af4101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 --2- 172.21.15.16:0/442239703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4100fb0 0x7f1af4101430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009990 tx=0x7f1adc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 -- 172.21.15.16:0/442239703 shutdown_connections 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 --2- 172.21.15.16:0/442239703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1af4101970 0x7f1af4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 --2- 172.21.15.16:0/442239703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 0x7f1af4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 --2- 172.21.15.16:0/442239703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4100fb0 0x7f1af4101430 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 -- 172.21.15.16:0/442239703 >> 172.21.15.16:0/442239703 conn(0x7f1af40fb3f0 msgr2=0x7f1af40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 -- 172.21.15.16:0/442239703 shutdown_connections 2024-05-08T23:19:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.598+0000 7f1af9163700 1 -- 172.21.15.16:0/442239703 wait complete. 2024-05-08T23:19:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 Processor -- start 2024-05-08T23:19:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 -- start start 2024-05-08T23:19:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 0x7f1af4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1af4100fb0 0x7f1af4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 0x7f1af41a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41a0b00 con 0x7f1af4101970 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41a0c40 con 0x7f1af4100fb0 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.599+0000 7f1af9163700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41a0dd0 con 0x7f1af40ffe60 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 0x7f1af41a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 0x7f1af41a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52622/0 (socket says 172.21.15.16:52622) 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 -- 172.21.15.16:0/3161746414 learned_addr learned my addr 172.21.15.16:0/3161746414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af2d9d700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 0x7f1af4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1aea59c700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1af4100fb0 0x7f1af4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 -- 172.21.15.16:0/3161746414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 msgr2=0x7f1af4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 0x7f1af4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 -- 172.21.15.16:0/3161746414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1af4100fb0 msgr2=0x7f1af4199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1af4100fb0 0x7f1af4199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 -- 172.21.15.16:0/3161746414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0095f0 con 0x7f1af4101970 2024-05-08T23:19:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af2d9d700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 0x7f1af4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af359e700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 0x7f1af41a0530 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ef30 tx=0x7f1aec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec018070 con 0x7f1af4101970 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af41a1050 con 0x7f1af4101970 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1aec00f040 con 0x7f1af4101970 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00cbc0 con 0x7f1af4101970 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.600+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af41a1560 con 0x7f1af4101970 2024-05-08T23:19:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.601+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af4067cf0 con 0x7f1af4101970 2024-05-08T23:19:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.603+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1aec00cd60 con 0x7f1af4101970 2024-05-08T23:19:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.604+0000 7f1af0d99700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1ae00a11b0 0x7f1ae00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.604+0000 7f1af2d9d700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1ae00a11b0 0x7f1ae00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.604+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1aec0cce70 con 0x7f1af4101970 2024-05-08T23:19:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.604+0000 7f1af2d9d700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1ae00a11b0 0x7f1ae00a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4005950 tx=0x7f1ae400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.607+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1aec096230 con 0x7f1af4101970 2024-05-08T23:19:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.827+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1af410b510 con 0x7f1af4101970 2024-05-08T23:19:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.827+0000 7f1af0d99700 1 -- 172.21.15.16:0/3161746414 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1aec099850 con 0x7f1af4101970 2024-05-08T23:19:04.828 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.829+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1ae00a11b0 msgr2=0x7f1ae00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.829+0000 7f1af9163700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1ae00a11b0 0x7f1ae00a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4005950 tx=0x7f1ae400b410 comp rx=0 tx=0).stop 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 msgr2=0x7f1af41a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 0x7f1af41a0530 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ef30 tx=0x7f1aec00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 shutdown_connections 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af4101970 0x7f1af41a0530 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f1ae00a11b0 0x7f1ae00a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ffe60 0x7f1af4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 --2- 172.21.15.16:0/3161746414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1af4100fb0 0x7f1af4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 >> 172.21.15.16:0/3161746414 conn(0x7f1af40fb3f0 msgr2=0x7f1af4068060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 shutdown_connections 2024-05-08T23:19:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.830+0000 7f1af9163700 1 -- 172.21.15.16:0/3161746414 wait complete. 2024-05-08T23:19:04.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:04.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- 172.21.15.16:0/235467931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8101970 msgr2=0x7f2bd8108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/235467931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8101970 0x7f2bd8108290 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b0a0 tx=0x7f2bd000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- 172.21.15.16:0/235467931 shutdown_connections 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/235467931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8101970 0x7f2bd8108290 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/235467931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bd80ffe60 0x7f2bd8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/235467931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- 172.21.15.16:0/235467931 >> 172.21.15.16:0/235467931 conn(0x7f2bd80fb3f0 msgr2=0x7f2bd80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- 172.21.15.16:0/235467931 shutdown_connections 2024-05-08T23:19:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- 172.21.15.16:0/235467931 wait complete. 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 Processor -- start 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- start start 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bd80ffe60 0x7f2bd8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bd8101970 0x7f2bd81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a0c50 con 0x7f2bd8100fb0 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a0d90 con 0x7f2bd80ffe60 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bdd31e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd81a0f20 con 0x7f2bd8101970 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52632/0 (socket says 172.21.15.16:52632) 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 -- 172.21.15.16:0/197195361 learned_addr learned my addr 172.21.15.16:0/197195361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:04.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd6ffd700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bd80ffe60 0x7f2bd8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd77fe700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bd8101970 0x7f2bd81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 -- 172.21.15.16:0/197195361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bd8101970 msgr2=0x7f2bd81a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bd8101970 0x7f2bd81a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 -- 172.21.15.16:0/197195361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bd80ffe60 msgr2=0x7f2bd8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bd80ffe60 0x7f2bd8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 -- 172.21.15.16:0/197195361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0009d00 con 0x7f2bd8100fb0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.987+0000 7f2bd67fc700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8199660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00e9c0 tx=0x7f2bcc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.988+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0097b0 con 0x7f2bd8100fb0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.988+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bcc004cc0 con 0x7f2bd8100fb0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.988+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd81a1200 con 0x7f2bd8100fb0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.988+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc005530 con 0x7f2bd8100fb0 2024-05-08T23:19:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.988+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd81a16c0 con 0x7f2bd8100fb0 2024-05-08T23:19:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.989+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd8068140 con 0x7f2bd8100fb0 2024-05-08T23:19:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.990+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2bcc004750 con 0x7f2bd8100fb0 2024-05-08T23:19:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.992+0000 7f2bbffff700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2bc00a11b0 0x7f2bc00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.992+0000 7f2bd6ffd700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2bc00a11b0 0x7f2bc00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.992+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f2bcc014070 con 0x7f2bd8100fb0 2024-05-08T23:19:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.992+0000 7f2bd6ffd700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2bc00a11b0 0x7f2bc00a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007c60 tx=0x7f2bc80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:04.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:04.996+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2bcc096350 con 0x7f2bd8100fb0 2024-05-08T23:19:05.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:04 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3306113418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:04 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1972422886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:05.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:04 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3161746414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:05.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:04 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3306113418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:05.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:04 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1972422886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:05.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:04 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3161746414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:05.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:04 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3306113418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:05.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:04 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1972422886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:05.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:04 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3161746414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:05.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.214+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bd81a1940 con 0x7f2bd8100fb0 2024-05-08T23:19:05.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.215+0000 7f2bbffff700 1 -- 172.21.15.16:0/197195361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bcc099970 con 0x7f2bd8100fb0 2024-05-08T23:19:05.216 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:05.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.217+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2bc00a11b0 msgr2=0x7f2bc00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.217+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2bc00a11b0 0x7f2bc00a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007c60 tx=0x7f2bc80073d0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.217+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 msgr2=0x7f2bd8199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.217+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8199660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00e9c0 tx=0x7f2bcc00ecd0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.217+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 shutdown_connections 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.217+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f2bc00a11b0 0x7f2bc00a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.218+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bd8101970 0x7f2bd81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.218+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2bd80ffe60 0x7f2bd8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.218+0000 7f2bdd31e700 1 --2- 172.21.15.16:0/197195361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd8100fb0 0x7f2bd8199660 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.218+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 >> 172.21.15.16:0/197195361 conn(0x7f2bd80fb3f0 msgr2=0x7f2bd80fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.218+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 shutdown_connections 2024-05-08T23:19:05.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.218+0000 7f2bdd31e700 1 -- 172.21.15.16:0/197195361 wait complete. 2024-05-08T23:19:05.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 -- 172.21.15.16:0/788673261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 msgr2=0x7ff448101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 --2- 172.21.15.16:0/788673261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 0x7ff448101430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009990 tx=0x7ff43c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 -- 172.21.15.16:0/788673261 shutdown_connections 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 --2- 172.21.15.16:0/788673261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff448101970 0x7ff448108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 --2- 172.21.15.16:0/788673261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4480ffe60 0x7ff448100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 --2- 172.21.15.16:0/788673261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 0x7ff448101430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 -- 172.21.15.16:0/788673261 >> 172.21.15.16:0/788673261 conn(0x7ff4480fb3f0 msgr2=0x7ff4480fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 -- 172.21.15.16:0/788673261 shutdown_connections 2024-05-08T23:19:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.373+0000 7ff44dec4700 1 -- 172.21.15.16:0/788673261 wait complete. 2024-05-08T23:19:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.374+0000 7ff44dec4700 1 Processor -- start 2024-05-08T23:19:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 -- start start 2024-05-08T23:19:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 0x7ff448195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 0x7ff4481961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff448101970 0x7ff44819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819d7a0 con 0x7ff4480ffe60 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819d910 con 0x7ff448101970 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff44dec4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819dad0 con 0x7ff448100fb0 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 0x7ff448195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 0x7ff448195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52638/0 (socket says 172.21.15.16:52638) 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 -- 172.21.15.16:0/323051773 learned_addr learned my addr 172.21.15.16:0/323051773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff446ffd700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 0x7ff4481961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff447fff700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff448101970 0x7ff44819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 -- 172.21.15.16:0/323051773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 msgr2=0x7ff4481961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 0x7ff4481961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 -- 172.21.15.16:0/323051773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff448101970 msgr2=0x7ff44819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff448101970 0x7ff44819d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.375+0000 7ff4477fe700 1 -- 172.21.15.16:0/323051773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0095f0 con 0x7ff4480ffe60 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.376+0000 7ff4477fe700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 0x7ff448195cb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d950 tx=0x7ff43800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.376+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438009940 con 0x7ff4480ffe60 2024-05-08T23:19:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.376+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44819ddb0 con 0x7ff4480ffe60 2024-05-08T23:19:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.376+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff438010460 con 0x7ff4480ffe60 2024-05-08T23:19:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.376+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800f620 con 0x7ff4480ffe60 2024-05-08T23:19:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.376+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44819e220 con 0x7ff4480ffe60 2024-05-08T23:19:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.377+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448107320 con 0x7ff4480ffe60 2024-05-08T23:19:05.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.378+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff4380105d0 con 0x7ff4480ffe60 2024-05-08T23:19:05.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.380+0000 7ff444ff9700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff4300a11b0 0x7ff4300a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.380+0000 7ff446ffd700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff4300a11b0 0x7ff4300a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.380+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff4380ccbf0 con 0x7ff4480ffe60 2024-05-08T23:19:05.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.380+0000 7ff446ffd700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff4300a11b0 0x7ff4300a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006010 tx=0x7ff43c00b630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:05.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.384+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff438095ec0 con 0x7ff4480ffe60 2024-05-08T23:19:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.607+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff44810b5c0 con 0x7ff4480ffe60 2024-05-08T23:19:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.608+0000 7ff444ff9700 1 -- 172.21.15.16:0/323051773 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff4380160a0 con 0x7ff4480ffe60 2024-05-08T23:19:05.609 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.610+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff4300a11b0 msgr2=0x7ff4300a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.610+0000 7ff44dec4700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff4300a11b0 0x7ff4300a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006010 tx=0x7ff43c00b630 comp rx=0 tx=0).stop 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.610+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 msgr2=0x7ff448195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.610+0000 7ff44dec4700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 0x7ff448195cb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d950 tx=0x7ff43800dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.610+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 shutdown_connections 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff448101970 0x7ff44819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4480ffe60 0x7ff448195cb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff4300a11b0 0x7ff4300a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 --2- 172.21.15.16:0/323051773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff448100fb0 0x7ff4481961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 >> 172.21.15.16:0/323051773 conn(0x7ff4480fb3f0 msgr2=0x7ff448104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 shutdown_connections 2024-05-08T23:19:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.611+0000 7ff44dec4700 1 -- 172.21.15.16:0/323051773 wait complete. 2024-05-08T23:19:05.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:05.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 -- 172.21.15.16:0/2586378007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468102970 msgr2=0x7f7468109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 --2- 172.21.15.16:0/2586378007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468102970 0x7f7468109290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f746000b0a0 tx=0x7f746000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 -- 172.21.15.16:0/2586378007 shutdown_connections 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 --2- 172.21.15.16:0/2586378007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468102970 0x7f7468109290 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 --2- 172.21.15.16:0/2586378007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7468100e60 0x7f7468101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 --2- 172.21.15.16:0/2586378007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7468101fb0 0x7f7468102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 -- 172.21.15.16:0/2586378007 >> 172.21.15.16:0/2586378007 conn(0x7f74680fc3d0 msgr2=0x7f74680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 -- 172.21.15.16:0/2586378007 shutdown_connections 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.765+0000 7f746d857700 1 -- 172.21.15.16:0/2586378007 wait complete. 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 Processor -- start 2024-05-08T23:19:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 -- start start 2024-05-08T23:19:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 0x7f7468196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7468101fb0 0x7f7468197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7468102970 0x7f746819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819e930 con 0x7f7468100e60 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819ea70 con 0x7f7468101fb0 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819ec00 con 0x7f7468102970 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 0x7f7468196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 0x7f7468196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52660/0 (socket says 172.21.15.16:52660) 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 -- 172.21.15.16:0/1347433449 learned_addr learned my addr 172.21.15.16:0/1347433449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f745e7fc700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7468101fb0 0x7f7468197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f74677fe700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7468102970 0x7f746819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 -- 172.21.15.16:0/1347433449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7468102970 msgr2=0x7f746819e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7468102970 0x7f746819e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 -- 172.21.15.16:0/1347433449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7468101fb0 msgr2=0x7f7468197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7468101fb0 0x7f7468197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 -- 172.21.15.16:0/1347433449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460009d00 con 0x7f7468100e60 2024-05-08T23:19:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f7466ffd700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 0x7f7468196e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f745800d950 tx=0x7f745800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800f890 con 0x7f7468100e60 2024-05-08T23:19:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f745800fe80 con 0x7f7468100e60 2024-05-08T23:19:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746819eee0 con 0x7f7468100e60 2024-05-08T23:19:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800e3b0 con 0x7f7468100e60 2024-05-08T23:19:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.766+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746819f300 con 0x7f7468100e60 2024-05-08T23:19:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.768+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74681082d0 con 0x7f7468100e60 2024-05-08T23:19:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.769+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7458010460 con 0x7f7468100e60 2024-05-08T23:19:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.770+0000 7f746c855700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f74540a11b0 0x7f74540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.770+0000 7f745e7fc700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f74540a11b0 0x7f74540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.770+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f74580ccb60 con 0x7f7468100e60 2024-05-08T23:19:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.771+0000 7f745e7fc700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f74540a11b0 0x7f74540a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7450005950 tx=0x7f745000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:05.775+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7458095e30 con 0x7f7468100e60 2024-05-08T23:19:06.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.053+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7468068060 con 0x7f7468100e60 2024-05-08T23:19:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.054+0000 7f746c855700 1 -- 172.21.15.16:0/1347433449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f7458016090 con 0x7f7468100e60 2024-05-08T23:19:06.053 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:06.054 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:18:12.692850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f74540a11b0 msgr2=0x7f74540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f74540a11b0 0x7f74540a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7450005950 tx=0x7f745000b410 comp rx=0 tx=0).stop 2024-05-08T23:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 msgr2=0x7f7468196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 0x7f7468196e00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f745800d950 tx=0x7f745800dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 shutdown_connections 2024-05-08T23:19:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f74540a11b0 0x7f74540a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7468102970 0x7f746819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7468100e60 0x7f7468196e00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 --2- 172.21.15.16:0/1347433449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7468101fb0 0x7f7468197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 >> 172.21.15.16:0/1347433449 conn(0x7f74680fc3d0 msgr2=0x7f7468105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 shutdown_connections 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.056+0000 7f746d857700 1 -- 172.21.15.16:0/1347433449 wait complete. 2024-05-08T23:19:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:19:06.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:05 smithi016 ceph-mon[42541]: pgmap v264: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:06.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:05 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/197195361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:06.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:05 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/323051773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:06.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:06.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:05 smithi189 ceph-mon[46919]: pgmap v264: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:06.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:05 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/197195361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:06.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:05 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/323051773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 -- 172.21.15.16:0/2140928285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf81018e0 msgr2=0x7febf8103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 --2- 172.21.15.16:0/2140928285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf81018e0 0x7febf8103cd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 -- 172.21.15.16:0/2140928285 shutdown_connections 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 --2- 172.21.15.16:0/2140928285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8106b40 0x7febf8108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 --2- 172.21.15.16:0/2140928285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf81018e0 0x7febf8103cd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 --2- 172.21.15.16:0/2140928285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febf8104210 0x7febf8106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 -- 172.21.15.16:0/2140928285 >> 172.21.15.16:0/2140928285 conn(0x7febf80fb2f0 msgr2=0x7febf80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 -- 172.21.15.16:0/2140928285 shutdown_connections 2024-05-08T23:19:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.211+0000 7febfe122700 1 -- 172.21.15.16:0/2140928285 wait complete. 2024-05-08T23:19:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 Processor -- start 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 -- start start 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febf81018e0 0x7febf8101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8104210 0x7febf80ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 0x7febf80ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8100570 con 0x7febf8106b40 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81006e0 con 0x7febf8104210 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febfe122700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81a0cf0 con 0x7febf81018e0 2024-05-08T23:19:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 0x7febf80ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 0x7febf80ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52684/0 (socket says 172.21.15.16:52684) 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf7fff700 1 -- 172.21.15.16:0/3088687009 learned_addr learned my addr 172.21.15.16:0/3088687009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf77fe700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febf81018e0 0x7febf8101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf6ffd700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8104210 0x7febf80ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf7fff700 1 -- 172.21.15.16:0/3088687009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febf81018e0 msgr2=0x7febf8101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.212+0000 7febf7fff700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febf81018e0 0x7febf8101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf7fff700 1 -- 172.21.15.16:0/3088687009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8104210 msgr2=0x7febf80ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf7fff700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8104210 0x7febf80ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf7fff700 1 -- 172.21.15.16:0/3088687009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80095f0 con 0x7febf8106b40 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf7fff700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 0x7febf80ffdd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7febf000ef30 tx=0x7febf000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0018070 con 0x7febf8106b40 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febf000f040 con 0x7febf8106b40 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf81a0f20 con 0x7febf8106b40 2024-05-08T23:19:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf000ccc0 con 0x7febf8106b40 2024-05-08T23:19:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.213+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf81a1350 con 0x7febf8106b40 2024-05-08T23:19:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.214+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf80fce70 con 0x7febf8106b40 2024-05-08T23:19:06.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:05 smithi155 ceph-mon[47105]: pgmap v264: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:06.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:05 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/197195361' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:06.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:05 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/323051773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.215+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7febf0007500 con 0x7febf8106b40 2024-05-08T23:19:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.216+0000 7febf4ff9700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7febe00a11b0 0x7febe00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.216+0000 7febf77fe700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7febe00a11b0 0x7febe00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.216+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7febf00cc9c0 con 0x7febf8106b40 2024-05-08T23:19:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.217+0000 7febf77fe700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7febe00a11b0 0x7febe00a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7febe800b2e0 tx=0x7febe8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.221+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7febf0095d80 con 0x7febf8106b40 2024-05-08T23:19:06.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.440+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febf810c540 con 0x7febf8106b40 2024-05-08T23:19:06.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.442+0000 7febf4ff9700 1 -- 172.21.15.16:0/3088687009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24470 (secure 0 0 0) 0x7febf00993a0 con 0x7febf8106b40 2024-05-08T23:19:06.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:06.442 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:18:43.790069+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7febe00a11b0 msgr2=0x7febe00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7febe00a11b0 0x7febe00a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7febe800b2e0 tx=0x7febe8005fd0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 msgr2=0x7febf80ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 0x7febf80ffdd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7febf000ef30 tx=0x7febf000c450 comp rx=0 tx=0).stop 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 shutdown_connections 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8106b40 0x7febf80ffdd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7febe00a11b0 0x7febe00a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febf81018e0 0x7febf8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 --2- 172.21.15.16:0/3088687009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7febf8104210 0x7febf80ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 >> 172.21.15.16:0/3088687009 conn(0x7febf80fb2f0 msgr2=0x7febf81037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 shutdown_connections 2024-05-08T23:19:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.445+0000 7febfe122700 1 -- 172.21.15.16:0/3088687009 wait complete. 2024-05-08T23:19:06.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:06.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/416010860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98101970 msgr2=0x7f4e98108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/416010860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98101970 0x7f4e98108290 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000b0a0 tx=0x7f4e9000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/416010860 shutdown_connections 2024-05-08T23:19:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/416010860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98101970 0x7f4e98108290 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/416010860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e980ffe60 0x7f4e98100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/416010860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e98100fb0 0x7f4e98101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/416010860 >> 172.21.15.16:0/416010860 conn(0x7f4e980fb3f0 msgr2=0x7f4e980fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/416010860 shutdown_connections 2024-05-08T23:19:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.602+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/416010860 wait complete. 2024-05-08T23:19:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 Processor -- start 2024-05-08T23:19:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 -- start start 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e980ffe60 0x7f4e98068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 0x7f4e9819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e98101970 0x7f4e9819e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9819e9b0 con 0x7f4e98100fb0 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9819eb20 con 0x7f4e98101970 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9e8f3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9819ece0 con 0x7f4e980ffe60 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 0x7f4e9819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 0x7f4e9819bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57898/0 (socket says 172.21.15.16:57898) 2024-05-08T23:19:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e977fe700 1 -- 172.21.15.16:0/2472457068 learned_addr learned my addr 172.21.15.16:0/2472457068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e97fff700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e980ffe60 0x7f4e98068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.603+0000 7f4e9ce90700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e98101970 0x7f4e9819e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e977fe700 1 -- 172.21.15.16:0/2472457068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e980ffe60 msgr2=0x7f4e98068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e977fe700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e980ffe60 0x7f4e98068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e977fe700 1 -- 172.21.15.16:0/2472457068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e98101970 msgr2=0x7f4e9819e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e977fe700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e98101970 0x7f4e9819e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e977fe700 1 -- 172.21.15.16:0/2472457068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90009d00 con 0x7f4e98100fb0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e977fe700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 0x7f4e9819bba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d950 tx=0x7f4e8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00f890 con 0x7f4e98100fb0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e8c00fe80 con 0x7f4e98100fb0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9819ef60 con 0x7f4e98100fb0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00e3b0 con 0x7f4e98100fb0 2024-05-08T23:19:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.604+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9819f3e0 con 0x7f4e98100fb0 2024-05-08T23:19:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.605+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e98107200 con 0x7f4e98100fb0 2024-05-08T23:19:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.606+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4e8c010460 con 0x7f4e98100fb0 2024-05-08T23:19:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.607+0000 7f4e957fa700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e800a11b0 0x7f4e800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.607+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4e8c0ccb00 con 0x7f4e98100fb0 2024-05-08T23:19:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.607+0000 7f4e97fff700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e800a11b0 0x7f4e800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.608+0000 7f4e97fff700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e800a11b0 0x7f4e800a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800ac50 tx=0x7f4e8800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:06.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.612+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e8c095e30 con 0x7f4e98100fb0 2024-05-08T23:19:06.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.829+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e9810b510 con 0x7f4e98100fb0 2024-05-08T23:19:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.829+0000 7f4e957fa700 1 -- 172.21.15.16:0/2472457068 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e8c0160a0 con 0x7f4e98100fb0 2024-05-08T23:19:06.829 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:06.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e800a11b0 msgr2=0x7f4e800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e800a11b0 0x7f4e800a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800ac50 tx=0x7f4e8800a380 comp rx=0 tx=0).stop 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 msgr2=0x7f4e9819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 0x7f4e9819bba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d950 tx=0x7f4e8c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 shutdown_connections 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e98101970 0x7f4e9819e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4e800a11b0 0x7f4e800a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e980ffe60 0x7f4e98068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 --2- 172.21.15.16:0/2472457068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e98100fb0 0x7f4e9819bba0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 >> 172.21.15.16:0/2472457068 conn(0x7f4e980fb3f0 msgr2=0x7f4e98104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 shutdown_connections 2024-05-08T23:19:06.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.832+0000 7f4e9e8f3700 1 -- 172.21.15.16:0/2472457068 wait complete. 2024-05-08T23:19:06.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 -- 172.21.15.16:0/2886440483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8100fc0 msgr2=0x7ff9b8101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/2886440483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8100fc0 0x7ff9b8101440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009990 tx=0x7ff9ac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 -- 172.21.15.16:0/2886440483 shutdown_connections 2024-05-08T23:19:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/2886440483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9b8101980 0x7ff9b81082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/2886440483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/2886440483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8100fc0 0x7ff9b8101440 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 -- 172.21.15.16:0/2886440483 >> 172.21.15.16:0/2886440483 conn(0x7ff9b80fb400 msgr2=0x7ff9b80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 -- 172.21.15.16:0/2886440483 shutdown_connections 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.987+0000 7ff9c03b0700 1 -- 172.21.15.16:0/2886440483 wait complete. 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 Processor -- start 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 -- start start 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b8100fc0 0x7ff9b819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9b8101980 0x7ff9b819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b819e900 con 0x7ff9b80ffe70 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b819ea70 con 0x7ff9b8100fc0 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9c03b0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b819ec30 con 0x7ff9b8101980 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8069f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57926/0 (socket says 172.21.15.16:57926) 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 -- 172.21.15.16:0/3431569583 learned_addr learned my addr 172.21.15.16:0/3431569583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9bd94b700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b8100fc0 0x7ff9b819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be94d700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9b8101980 0x7ff9b819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 -- 172.21.15.16:0/3431569583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9b8101980 msgr2=0x7ff9b819e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9b8101980 0x7ff9b819e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 -- 172.21.15.16:0/3431569583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b8100fc0 msgr2=0x7ff9b819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b8100fc0 0x7ff9b819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9be14c700 1 -- 172.21.15.16:0/3431569583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0095f0 con 0x7ff9b80ffe70 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.988+0000 7ff9bd94b700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b8100fc0 0x7ff9b819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.989+0000 7ff9be14c700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8069f90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9a0 tx=0x7ff9b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.989+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4009790 con 0x7ff9b80ffe70 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.989+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9b400cbd0 con 0x7ff9b80ffe70 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.989+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b819eeb0 con 0x7ff9b80ffe70 2024-05-08T23:19:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.989+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4018680 con 0x7ff9b80ffe70 2024-05-08T23:19:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.989+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b819f2d0 con 0x7ff9b80ffe70 2024-05-08T23:19:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.990+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b81072e0 con 0x7ff9b80ffe70 2024-05-08T23:19:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.991+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff9b400cd40 con 0x7ff9b80ffe70 2024-05-08T23:19:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.992+0000 7ff9ab7fe700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff9a40a11b0 0x7ff9a40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.992+0000 7ff9bd94b700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff9a40a11b0 0x7ff9a40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.992+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff9b4014070 con 0x7ff9b80ffe70 2024-05-08T23:19:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.993+0000 7ff9bd94b700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff9a40a11b0 0x7ff9a40a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9b806a530 tx=0x7ff9ac005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:06.996+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9b4096570 con 0x7ff9b80ffe70 2024-05-08T23:19:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:06 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1347433449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:06 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3088687009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:07.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:06 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2472457068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:07.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:06 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1347433449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:06 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3088687009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:07.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:06 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2472457068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:07.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.213+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9b810b720 con 0x7ff9b80ffe70 2024-05-08T23:19:07.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.213+0000 7ff9ab7fe700 1 -- 172.21.15.16:0/3431569583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9b40260a0 con 0x7ff9b80ffe70 2024-05-08T23:19:07.213 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:07.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:06 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1347433449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:07.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:06 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3088687009' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:07.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:06 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2472457068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff9a40a11b0 msgr2=0x7ff9a40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff9a40a11b0 0x7ff9a40a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff9b806a530 tx=0x7ff9ac005940 comp rx=0 tx=0).stop 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 msgr2=0x7ff9b8069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8069f90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9a0 tx=0x7ff9b400ed60 comp rx=0 tx=0).stop 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 shutdown_connections 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7ff9a40a11b0 0x7ff9a40a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9b8101980 0x7ff9b819e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80ffe70 0x7ff9b8069f90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 --2- 172.21.15.16:0/3431569583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b8100fc0 0x7ff9b819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 >> 172.21.15.16:0/3431569583 conn(0x7ff9b80fb400 msgr2=0x7ff9b8104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 shutdown_connections 2024-05-08T23:19:07.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.216+0000 7ff9c03b0700 1 -- 172.21.15.16:0/3431569583 wait complete. 2024-05-08T23:19:07.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 -- 172.21.15.16:0/737557717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec80ffe80 msgr2=0x7faec81002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 --2- 172.21.15.16:0/737557717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec80ffe80 0x7faec81002a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faec4009990 tx=0x7faec4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 -- 172.21.15.16:0/737557717 shutdown_connections 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 --2- 172.21.15.16:0/737557717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec8101990 0x7faec81082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 --2- 172.21.15.16:0/737557717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec80ffe80 0x7faec81002a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 --2- 172.21.15.16:0/737557717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec8100fd0 0x7faec8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 -- 172.21.15.16:0/737557717 >> 172.21.15.16:0/737557717 conn(0x7faec80fb3f0 msgr2=0x7faec80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 -- 172.21.15.16:0/737557717 shutdown_connections 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.369+0000 7faeceb5b700 1 -- 172.21.15.16:0/737557717 wait complete. 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 Processor -- start 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 -- start start 2024-05-08T23:19:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec80ffe80 0x7faec8196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec8100fd0 0x7faec81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 0x7faec819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec819e950 con 0x7faec8101990 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec819ea90 con 0x7faec80ffe80 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faeceb5b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec819ec20 con 0x7faec8100fd0 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.370+0000 7faecd0f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 0x7faec819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 0x7faec819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57948/0 (socket says 172.21.15.16:57948) 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 -- 172.21.15.16:0/3892837306 learned_addr learned my addr 172.21.15.16:0/3892837306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faebffff700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec8100fd0 0x7faec81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecc8f7700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec80ffe80 0x7faec8196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 -- 172.21.15.16:0/3892837306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec8100fd0 msgr2=0x7faec81973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec8100fd0 0x7faec81973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 -- 172.21.15.16:0/3892837306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec80ffe80 msgr2=0x7faec8196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec80ffe80 0x7faec8196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 -- 172.21.15.16:0/3892837306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40095f0 con 0x7faec8101990 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faecd0f8700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 0x7faec819e380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faec000ef30 tx=0x7faec000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0018070 con 0x7faec8101990 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec819eea0 con 0x7faec8101990 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faec000f040 con 0x7faec8101990 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.371+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec000cbc0 con 0x7faec8101990 2024-05-08T23:19:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.372+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec819f2d0 con 0x7faec8101990 2024-05-08T23:19:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.372+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec8067460 con 0x7faec8101990 2024-05-08T23:19:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.374+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faec000cd80 con 0x7faec8101990 2024-05-08T23:19:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.374+0000 7faebdffb700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faeb00a11b0 0x7faeb00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.374+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7faec00cc840 con 0x7faec8101990 2024-05-08T23:19:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.374+0000 7faecc8f7700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faeb00a11b0 0x7faeb00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.375+0000 7faecc8f7700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faeb00a11b0 0x7faeb00a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faec4006010 tx=0x7faec400b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:07.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.379+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faec0095c00 con 0x7faec8101990 2024-05-08T23:19:07.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.597+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faec819f570 con 0x7faec8101990 2024-05-08T23:19:07.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.597+0000 7faebdffb700 1 -- 172.21.15.16:0/3892837306 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7faec0099220 con 0x7faec8101990 2024-05-08T23:19:07.597 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faeb00a11b0 msgr2=0x7faeb00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faeb00a11b0 0x7faeb00a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faec4006010 tx=0x7faec400b5d0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 msgr2=0x7faec819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 0x7faec819e380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faec000ef30 tx=0x7faec000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 shutdown_connections 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faec8101990 0x7faec819e380 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faec80ffe80 0x7faec8196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7faeb00a11b0 0x7faeb00a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 --2- 172.21.15.16:0/3892837306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec8100fd0 0x7faec81973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 >> 172.21.15.16:0/3892837306 conn(0x7faec80fb3f0 msgr2=0x7faec8069200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 shutdown_connections 2024-05-08T23:19:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.600+0000 7faeceb5b700 1 -- 172.21.15.16:0/3892837306 wait complete. 2024-05-08T23:19:07.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 -- 172.21.15.16:0/2650562657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0074f60 msgr2=0x7fefd00733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 --2- 172.21.15.16:0/2650562657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0074f60 0x7fefd00733b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009990 tx=0x7fefc0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 -- 172.21.15.16:0/2650562657 shutdown_connections 2024-05-08T23:19:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 --2- 172.21.15.16:0/2650562657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0108e60 0x7fefd010b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 --2- 172.21.15.16:0/2650562657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0074f60 0x7fefd00733b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 --2- 172.21.15.16:0/2650562657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefd0073980 0x7fefd0073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 -- 172.21.15.16:0/2650562657 >> 172.21.15.16:0/2650562657 conn(0x7fefd00fc4e0 msgr2=0x7fefd00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 -- 172.21.15.16:0/2650562657 shutdown_connections 2024-05-08T23:19:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.755+0000 7fefd5084700 1 -- 172.21.15.16:0/2650562657 wait complete. 2024-05-08T23:19:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 Processor -- start 2024-05-08T23:19:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 -- start start 2024-05-08T23:19:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0073980 0x7fefd0199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefd0074f60 0x7fefd01995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 0x7fefd01a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01a0b90 con 0x7fefd0108e60 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01a0cd0 con 0x7fefd0074f60 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.756+0000 7fefd5084700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01a0e60 con 0x7fefd0073980 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 0x7fefd01a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 0x7fefd01a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57962/0 (socket says 172.21.15.16:57962) 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 -- 172.21.15.16:0/898049870 learned_addr learned my addr 172.21.15.16:0/898049870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefced9d700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0073980 0x7fefd0199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefc659c700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefd0074f60 0x7fefd01995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 -- 172.21.15.16:0/898049870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0073980 msgr2=0x7fefd0199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0073980 0x7fefd0199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 -- 172.21.15.16:0/898049870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefd0074f60 msgr2=0x7fefd01995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefd0074f60 0x7fefd01995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 -- 172.21.15.16:0/898049870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00095f0 con 0x7fefd0108e60 2024-05-08T23:19:07.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefced9d700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0073980 0x7fefd0199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefcf59e700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 0x7fefd01a05c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fefc8007ee0 tx=0x7fefc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017070 con 0x7fefd0108e60 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefc800f040 con 0x7fefd0108e60 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd01a1140 con 0x7fefd0108e60 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc8005710 con 0x7fefd0108e60 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.757+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd01a1600 con 0x7fefd0108e60 2024-05-08T23:19:07.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.759+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd0107c70 con 0x7fefd0108e60 2024-05-08T23:19:07.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.760+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fefc8005870 con 0x7fefd0108e60 2024-05-08T23:19:07.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.760+0000 7fefccd99700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefbc0a11b0 0x7fefbc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:07.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.760+0000 7fefced9d700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefbc0a11b0 0x7fefbc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:07.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.760+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fefc80cc620 con 0x7fefd0108e60 2024-05-08T23:19:07.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.761+0000 7fefced9d700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefbc0a11b0 0x7fefbc0a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefc000b670 tx=0x7fefc0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:07.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.765+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fefc8095950 con 0x7fefd0108e60 2024-05-08T23:19:07.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:07 smithi016 ceph-mon[42541]: pgmap v265: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:07.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:07 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3431569583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:07.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:07 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3892837306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.981+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fefd010e970 con 0x7fefd0108e60 2024-05-08T23:19:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.982+0000 7fefccd99700 1 -- 172.21.15.16:0/898049870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7fefc8082e60 con 0x7fefd0108e60 2024-05-08T23:19:07.982 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefbc0a11b0 msgr2=0x7fefbc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefbc0a11b0 0x7fefbc0a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fefc000b670 tx=0x7fefc0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 msgr2=0x7fefd01a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 0x7fefd01a05c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fefc8007ee0 tx=0x7fefc800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 shutdown_connections 2024-05-08T23:19:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefd0108e60 0x7fefd01a05c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fefbc0a11b0 0x7fefbc0a3670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd0073980 0x7fefd0199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 --2- 172.21.15.16:0/898049870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefd0074f60 0x7fefd01995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 >> 172.21.15.16:0/898049870 conn(0x7fefd00fc4e0 msgr2=0x7fefd0104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 shutdown_connections 2024-05-08T23:19:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:07.984+0000 7fefd5084700 1 -- 172.21.15.16:0/898049870 wait complete. 2024-05-08T23:19:07.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 -- 172.21.15.16:0/2971340834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 msgr2=0x7f3ff4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 --2- 172.21.15.16:0/2971340834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff4109360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00b0a0 tx=0x7f3fec00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 -- 172.21.15.16:0/2971340834 shutdown_connections 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 --2- 172.21.15.16:0/2971340834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff4109360 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 --2- 172.21.15.16:0/2971340834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff40ff140 0x7f3ff40ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 --2- 172.21.15.16:0/2971340834 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ff4102110 0x7f3ff4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 -- 172.21.15.16:0/2971340834 >> 172.21.15.16:0/2971340834 conn(0x7f3ff40faa70 msgr2=0x7f3ff40fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:08.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 -- 172.21.15.16:0/2971340834 shutdown_connections 2024-05-08T23:19:08.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.136+0000 7f3ff9809700 1 -- 172.21.15.16:0/2971340834 wait complete. 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 Processor -- start 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 -- start start 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ff40ff140 0x7f3ff4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff4102110 0x7f3ff41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff419e8c0 con 0x7f3ff4102ab0 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff419ea00 con 0x7f3ff40ff140 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff9809700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff419eb90 con 0x7f3ff4102110 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.137+0000 7f3ff37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57976/0 (socket says 172.21.15.16:57976) 2024-05-08T23:19:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 -- 172.21.15.16:0/3537666278 learned_addr learned my addr 172.21.15.16:0/3537666278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff2ffd700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ff40ff140 0x7f3ff4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff27fc700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff4102110 0x7f3ff41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 -- 172.21.15.16:0/3537666278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff4102110 msgr2=0x7f3ff41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff4102110 0x7f3ff41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 -- 172.21.15.16:0/3537666278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ff40ff140 msgr2=0x7f3ff4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ff40ff140 0x7f3ff4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 -- 172.21.15.16:0/3537666278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec009d00 con 0x7f3ff4102ab0 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff37fe700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff419e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3fec007870 tx=0x7f3fec0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec00e040 con 0x7f3ff4102ab0 2024-05-08T23:19:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3fec00be70 con 0x7f3ff4102ab0 2024-05-08T23:19:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff419ee10 con 0x7f3ff4102ab0 2024-05-08T23:19:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec005040 con 0x7f3ff4102ab0 2024-05-08T23:19:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.138+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff419f280 con 0x7f3ff4102ab0 2024-05-08T23:19:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.139+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff4108410 con 0x7f3ff4102ab0 2024-05-08T23:19:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.141+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3fec01d470 con 0x7f3ff4102ab0 2024-05-08T23:19:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.142+0000 7f3ff8807700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3fe00a1230 0x7f3fe00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.142+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3fec0ce910 con 0x7f3ff4102ab0 2024-05-08T23:19:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.142+0000 7f3ff2ffd700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3fe00a1230 0x7f3fe00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.143+0000 7f3ff2ffd700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3fe00a1230 0x7f3fe00a36f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40692d0 tx=0x7f3fdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.145+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3fec097cd0 con 0x7f3ff4102ab0 2024-05-08T23:19:08.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:07 smithi189 ceph-mon[46919]: pgmap v265: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:08.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:07 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3431569583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:07 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3892837306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:08.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:07 smithi155 ceph-mon[47105]: pgmap v265: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:07 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3431569583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:07 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3892837306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.410+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ff4068060 con 0x7f3ff4102ab0 2024-05-08T23:19:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.411+0000 7f3ff8807700 1 -- 172.21.15.16:0/3537666278 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f3fec09b2f0 con 0x7f3ff4102ab0 2024-05-08T23:19:08.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:08.411 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:18:12.692850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3fe00a1230 msgr2=0x7f3fe00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3fe00a1230 0x7f3fe00a36f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40692d0 tx=0x7f3fdc009260 comp rx=0 tx=0).stop 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 msgr2=0x7f3ff419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff419e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3fec007870 tx=0x7f3fec0078a0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 shutdown_connections 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4102ab0 0x7f3ff419e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ff40ff140 0x7f3ff4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f3fe00a1230 0x7f3fe00a36f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 --2- 172.21.15.16:0/3537666278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff4102110 0x7f3ff41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.413+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 >> 172.21.15.16:0/3537666278 conn(0x7f3ff40faa70 msgr2=0x7f3ff4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.414+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 shutdown_connections 2024-05-08T23:19:08.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.414+0000 7f3ff9809700 1 -- 172.21.15.16:0/3537666278 wait complete. 2024-05-08T23:19:08.415 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:19:08.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 -- 172.21.15.16:0/1073559300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 msgr2=0x7efdc81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 --2- 172.21.15.16:0/1073559300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc81081a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efdc000b0d0 tx=0x7efdc000b3e0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 -- 172.21.15.16:0/1073559300 shutdown_connections 2024-05-08T23:19:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 --2- 172.21.15.16:0/1073559300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc81081a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 --2- 172.21.15.16:0/1073559300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdc80ffd60 0x7efdc8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 --2- 172.21.15.16:0/1073559300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc8100f50 0x7efdc81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 -- 172.21.15.16:0/1073559300 >> 172.21.15.16:0/1073559300 conn(0x7efdc80fb2f0 msgr2=0x7efdc80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 -- 172.21.15.16:0/1073559300 shutdown_connections 2024-05-08T23:19:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.565+0000 7efdd0199700 1 -- 172.21.15.16:0/1073559300 wait complete. 2024-05-08T23:19:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.566+0000 7efdd0199700 1 Processor -- start 2024-05-08T23:19:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.566+0000 7efdd0199700 1 -- start start 2024-05-08T23:19:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc80ffd60 0x7efdc8068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdc8100f50 0x7efdc819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819e930 con 0x7efdc81018f0 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819eaa0 con 0x7efdc80ffd60 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc819ec60 con 0x7efdc8100f50 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc819e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57992/0 (socket says 172.21.15.16:57992) 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 -- 172.21.15.16:0/4281970142 learned_addr learned my addr 172.21.15.16:0/4281970142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdcd734700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdc8100f50 0x7efdc819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdcdf35700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc80ffd60 0x7efdc8068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 -- 172.21.15.16:0/4281970142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdc8100f50 msgr2=0x7efdc819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdc8100f50 0x7efdc819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 -- 172.21.15.16:0/4281970142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc80ffd60 msgr2=0x7efdc8068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc80ffd60 0x7efdc8068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 -- 172.21.15.16:0/4281970142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc0009d00 con 0x7efdc81018f0 2024-05-08T23:19:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdce736700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc819e140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efdc00079c0 tx=0x7efdc000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc000e040 con 0x7efdc81018f0 2024-05-08T23:19:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdc0004c50 con 0x7efdc81018f0 2024-05-08T23:19:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc819ee80 con 0x7efdc81018f0 2024-05-08T23:19:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc001b6b0 con 0x7efdc81018f0 2024-05-08T23:19:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.567+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc819f300 con 0x7efdc81018f0 2024-05-08T23:19:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.568+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc81071e0 con 0x7efdc81018f0 2024-05-08T23:19:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.570+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efdc001b810 con 0x7efdc81018f0 2024-05-08T23:19:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.570+0000 7efdbeffd700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efdb40a1230 0x7efdb40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.571+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efdc00ced60 con 0x7efdc81018f0 2024-05-08T23:19:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.571+0000 7efdcdf35700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efdb40a1230 0x7efdb40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.571+0000 7efdcdf35700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efdb40a1230 0x7efdb40a36f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdc8100db0 tx=0x7efdc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.574+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efdc00981a0 con 0x7efdc81018f0 2024-05-08T23:19:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.791+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdc8067c10 con 0x7efdc81018f0 2024-05-08T23:19:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.793+0000 7efdbeffd700 1 -- 172.21.15.16:0/4281970142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24470 (secure 0 0 0) 0x7efdc0021090 con 0x7efdc81018f0 2024-05-08T23:19:08.793 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:08.793 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:18:43.790069+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efdb40a1230 msgr2=0x7efdb40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efdb40a1230 0x7efdb40a36f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efdc8100db0 tx=0x7efdc4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 msgr2=0x7efdc819e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc819e140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7efdc00079c0 tx=0x7efdc000cb60 comp rx=0 tx=0).stop 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 shutdown_connections 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc81018f0 0x7efdc819e140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdc80ffd60 0x7efdc8068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efdb40a1230 0x7efdb40a36f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 --2- 172.21.15.16:0/4281970142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efdc8100f50 0x7efdc819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 >> 172.21.15.16:0/4281970142 conn(0x7efdc80fb2f0 msgr2=0x7efdc8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 shutdown_connections 2024-05-08T23:19:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.795+0000 7efdd0199700 1 -- 172.21.15.16:0/4281970142 wait complete. 2024-05-08T23:19:08.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 -- 172.21.15.16:0/2196142513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ffd60 msgr2=0x7f1394100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 --2- 172.21.15.16:0/2196142513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ffd60 0x7f1394100180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13840099c0 tx=0x7f1384009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 -- 172.21.15.16:0/2196142513 shutdown_connections 2024-05-08T23:19:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 --2- 172.21.15.16:0/2196142513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13941018f0 0x7f13941081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 --2- 172.21.15.16:0/2196142513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13940ffd60 0x7f1394100180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 --2- 172.21.15.16:0/2196142513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1394100f50 0x7f13941013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 -- 172.21.15.16:0/2196142513 >> 172.21.15.16:0/2196142513 conn(0x7f13940fb2f0 msgr2=0x7f13940fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 -- 172.21.15.16:0/2196142513 shutdown_connections 2024-05-08T23:19:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.949+0000 7f1398e74700 1 -- 172.21.15.16:0/2196142513 wait complete. 2024-05-08T23:19:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 Processor -- start 2024-05-08T23:19:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 -- start start 2024-05-08T23:19:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13940ffd60 0x7f1394068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1394100f50 0x7f139419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 0x7f13941a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941a0a10 con 0x7f13941018f0 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941a0b50 con 0x7f13940ffd60 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.950+0000 7f1398e74700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941a0d10 con 0x7f1394100f50 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 0x7f13941a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 0x7f13941a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58012/0 (socket says 172.21.15.16:58012) 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 -- 172.21.15.16:0/1739282239 learned_addr learned my addr 172.21.15.16:0/1739282239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f139259c700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13940ffd60 0x7f1394068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1391d9b700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1394100f50 0x7f139419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 -- 172.21.15.16:0/1739282239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1394100f50 msgr2=0x7f139419deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1394100f50 0x7f139419deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 -- 172.21.15.16:0/1739282239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13940ffd60 msgr2=0x7f1394068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13940ffd60 0x7f1394068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 -- 172.21.15.16:0/1739282239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13840095f0 con 0x7f13941018f0 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f139259c700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13940ffd60 0x7f1394068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1392d9d700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 0x7f13941a0400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f138c00ef30 tx=0x7f138c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c018070 con 0x7f13941018f0 2024-05-08T23:19:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f138c00f040 con 0x7f13941018f0 2024-05-08T23:19:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941a0f90 con 0x7f13941018f0 2024-05-08T23:19:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c00cc70 con 0x7f13941018f0 2024-05-08T23:19:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.951+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a13c0 con 0x7f13941018f0 2024-05-08T23:19:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.952+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394107270 con 0x7f13941018f0 2024-05-08T23:19:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.954+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f138c003a00 con 0x7f13941018f0 2024-05-08T23:19:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.954+0000 7f13837fe700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f137c0a11b0 0x7f137c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.954+0000 7f139259c700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f137c0a11b0 0x7f137c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.954+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f138c0cce70 con 0x7f13941018f0 2024-05-08T23:19:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.955+0000 7f139259c700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f137c0a11b0 0x7f137c0a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1384000c00 tx=0x7f138401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:08.958+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f138c096230 con 0x7f13941018f0 2024-05-08T23:19:09.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/898049870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:09.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3537666278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:09.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4281970142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:09.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:08 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/898049870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:09.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:08 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3537666278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:09.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:08 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4281970142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:09.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:08 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/898049870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:09.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:08 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3537666278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:09.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:08 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4281970142' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:09.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.227+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1394067c10 con 0x7f13941018f0 2024-05-08T23:19:09.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.228+0000 7f13837fe700 1 -- 172.21.15.16:0/1739282239 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f138c003cb0 con 0x7f13941018f0 2024-05-08T23:19:09.227 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:09.227 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:18:12.692850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f137c0a11b0 msgr2=0x7f137c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f137c0a11b0 0x7f137c0a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1384000c00 tx=0x7f138401a040 comp rx=0 tx=0).stop 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 msgr2=0x7f13941a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 0x7f13941a0400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f138c00ef30 tx=0x7f138c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 shutdown_connections 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13941018f0 0x7f13941a0400 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13940ffd60 0x7f1394068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f137c0a11b0 0x7f137c0a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 --2- 172.21.15.16:0/1739282239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1394100f50 0x7f139419deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 >> 172.21.15.16:0/1739282239 conn(0x7f13940fb2f0 msgr2=0x7f13941043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 shutdown_connections 2024-05-08T23:19:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.230+0000 7f1398e74700 1 -- 172.21.15.16:0/1739282239 wait complete. 2024-05-08T23:19:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:19:09.244 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T23:17:46.078099+0000', 'modified': '2024-05-08T23:18:12.692850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24563, 'mds_1': 24550, 'mds_2': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/1260766455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1260766455}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1260766455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/3239398256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 3239398256}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 3239398256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1960878524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1960878524}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1960878524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.189:6837/2098584238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2098584238}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2098584238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.189:6839/3614950817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 3614950817}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 3614950817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6839/773665585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 773665585}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 773665585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:19:09.244 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:19:09.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:19:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 -- 172.21.15.16:0/1843617948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c1020f0 msgr2=0x7f9f2c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 --2- 172.21.15.16:0/1843617948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c1020f0 0x7f9f2c102570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009990 tx=0x7f9f1c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 -- 172.21.15.16:0/1843617948 shutdown_connections 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 --2- 172.21.15.16:0/1843617948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f2c102ab0 0x7f9f2c1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 --2- 172.21.15.16:0/1843617948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f2c100f20 0x7f9f2c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 --2- 172.21.15.16:0/1843617948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c1020f0 0x7f9f2c102570 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 -- 172.21.15.16:0/1843617948 >> 172.21.15.16:0/1843617948 conn(0x7f9f2c0fc4e0 msgr2=0x7f9f2c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 -- 172.21.15.16:0/1843617948 shutdown_connections 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.384+0000 7f9f34700700 1 -- 172.21.15.16:0/1843617948 wait complete. 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 Processor -- start 2024-05-08T23:19:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 -- start start 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 0x7f9f2c196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f2c1020f0 0x7f9f2c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f2c102ab0 0x7f9f2c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19e9a0 con 0x7f9f2c100f20 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19eae0 con 0x7f9f2c1020f0 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f34700700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c19ec70 con 0x7f9f2c102ab0 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 0x7f9f2c196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 0x7f9f2c196e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58028/0 (socket says 172.21.15.16:58028) 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 -- 172.21.15.16:0/52852815 learned_addr learned my addr 172.21.15.16:0/52852815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f2c1020f0 0x7f9f2c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f2c102ab0 0x7f9f2c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 -- 172.21.15.16:0/52852815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f2c102ab0 msgr2=0x7f9f2c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f2c102ab0 0x7f9f2c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 -- 172.21.15.16:0/52852815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f2c1020f0 msgr2=0x7f9f2c1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f2c1020f0 0x7f9f2c1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 -- 172.21.15.16:0/52852815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0095f0 con 0x7f9f2c100f20 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.385+0000 7f9f3249c700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 0x7f9f2c196e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e970 tx=0x7f9f2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.386+0000 7f9f237fe700 1 -- 172.21.15.16:0/52852815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28009790 con 0x7f9f2c100f20 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.386+0000 7f9f237fe700 1 -- 172.21.15.16:0/52852815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f2800eec0 con 0x7f9f2c100f20 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.386+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c19ef50 con 0x7f9f2c100f20 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.386+0000 7f9f237fe700 1 -- 172.21.15.16:0/52852815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28003c10 con 0x7f9f2c100f20 2024-05-08T23:19:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.386+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c19f370 con 0x7f9f2c100f20 2024-05-08T23:19:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.388+0000 7f9f237fe700 1 -- 172.21.15.16:0/52852815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9f28003db0 con 0x7f9f2c100f20 2024-05-08T23:19:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.389+0000 7f9f237fe700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f180a11b0 0x7f9f180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.389+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f180a11b0 0x7f9f180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.389+0000 7f9f237fe700 1 -- 172.21.15.16:0/52852815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9f28014070 con 0x7f9f2c100f20 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f180a11b0 0x7f9f180a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006010 tx=0x7f9f1c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 Processor -- start 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 -- start start 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x564090106d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107c20 0x5640901080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564090059100 con 0x564090107260 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564090056ac0 con 0x564090107c20 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f408bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640900523b0 con 0x564090106900 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f31c9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f31c9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58044/0 (socket says 172.21.15.16:58044) 2024-05-08T23:19:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.390+0000 7f9f31c9b700 1 -- 172.21.15.16:0/477528649 learned_addr learned my addr 172.21.15.16:0/477528649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107c20 0x5640901080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f3249c700 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x564090106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f31c9b700 1 -- 172.21.15.16:0/477528649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 msgr2=0x564090106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x564090106d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f31c9b700 1 -- 172.21.15.16:0/477528649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107c20 msgr2=0x5640901080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107c20 0x5640901080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f31c9b700 1 -- 172.21.15.16:0/477528649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56408fed3360 con 0x564090107260 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107c20 0x5640901080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901076e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c015000 tx=0x7f9f1c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dc1600fce28df7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f20ff9700 1 -- 172.21.15.16:0/477528649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c003d10 con 0x564090107260 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f20ff9700 1 -- 172.21.15.16:0/477528649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f1c01e040 con 0x564090107260 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.391+0000 7f9f20ff9700 1 -- 172.21.15.16:0/477528649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004990 con 0x564090107260 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 -- 172.21.15.16:0/477528649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 msgr2=0x5640901076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901076e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c015000 tx=0x7f9f1c020000 comp rx=0 tx=0).stop 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 -- 172.21.15.16:0/477528649 shutdown_connections 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107c20 0x5640901080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x564090106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 --2- 172.21.15.16:0/477528649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901076e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 -- 172.21.15.16:0/477528649 >> 172.21.15.16:0/477528649 conn(0x5640900601c0 msgr2=0x5640900fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 -- 172.21.15.16:0/477528649 shutdown_connections 2024-05-08T23:19:09.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.392+0000 7f9f408bab80 1 -- 172.21.15.16:0/477528649 wait complete. 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 -- start start 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 Processor -- start 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090110190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffee510 con 0x564090107260 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffee1b0 con 0x564090106900 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408fed5650 con 0x564090107c20 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f408bab80 10 client.0 _ll_register_callbacks cb 0x56408fff7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58058/0 (socket says 172.21.15.16:58058) 2024-05-08T23:19:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 -- 172.21.15.16:0/1851420273 learned_addr learned my addr 172.21.15.16:0/1851420273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f3249c700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 -- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 msgr2=0x564090117710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090117710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 -- 172.21.15.16:0/1851420273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 msgr2=0x564090110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.393+0000 7f9f31c9b700 1 -- 172.21.15.16:0/1851420273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c109d00 con 0x564090107260 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901106d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa2ab8790cea7e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f15ffb700 1 -- 172.21.15.16:0/1851420273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00b8b0 con 0x564090107260 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f15ffb700 1 -- 172.21.15.16:0/1851420273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9f1c00ba10 con 0x564090107260 2024-05-08T23:19:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56408ffab220 con 0x564090107260 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f15ffb700 1 -- 172.21.15.16:0/1851420273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f1c00bbd0 con 0x564090107260 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 10 client.15234 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56408fe7fb10 con 0x564090107260 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f15ffb700 1 -- 172.21.15.16:0/1851420273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56408ffab220 con 0x564090107260 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f15ffb700 1 -- 172.21.15.16:0/1851420273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f1c024650 con 0x564090107260 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 10 client.15234 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 10 client.15234 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 20 client.15234 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x56409011aef0 0x5640901201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 4 client.15234 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.394+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640900d17c0 con 0x56409011aef0 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.395+0000 7f9f3249c700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x56409011aef0 0x5640901201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.395+0000 7f9f3249c700 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x56409011aef0 0x5640901201d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd50b690a6c41a5c server_cookie=57c11f7ab0f06b2b in_seq=0 out_seq=0 2024-05-08T23:19:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.395+0000 7f9f15ffb700 10 client.15234 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f15ffb700 1 -- 172.21.15.16:0/1851420273 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640900d17c0 con 0x56409011aef0 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f15ffb700 10 client.15234 handle_command_reply: tid=0 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 client.15234 shutdown 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 msgr2=0x5640901106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901106d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c004f90 comp rx=0 tx=0).stop 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 shutdown_connections 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090117710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x56409011aef0 0x5640901201d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1851420273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x5640901106d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f15ffb700 0 client.15234 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 >> 172.21.15.16:0/1851420273 conn(0x5640900601c0 msgr2=0x5640900fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 shutdown_connections 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 1 -- 172.21.15.16:0/1851420273 wait complete. 2024-05-08T23:19:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.396+0000 7f9f408bab80 20 client.15234 trim_cache size 0 max 16384 2024-05-08T23:19:09.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.415+0000 7f9f408bab80 1 Processor -- start 2024-05-08T23:19:09.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.415+0000 7f9f408bab80 1 -- start start 2024-05-08T23:19:09.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x56409010f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107260 0x564090117630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 0x56409010e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f408bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640900d17f0 con 0x564090107c20 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f408bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56408ffab250 con 0x564090107260 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f408bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409004b150 con 0x564090106900 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 0x56409010e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 0x56409010e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58068/0 (socket says 172.21.15.16:58068) 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 -- 172.21.15.16:0/1260407191 learned_addr learned my addr 172.21.15.16:0/1260407191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107260 0x564090117630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f3249c700 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x56409010f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 -- 172.21.15.16:0/1260407191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 msgr2=0x56409010f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x56409010f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 -- 172.21.15.16:0/1260407191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107260 msgr2=0x564090117630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107260 0x564090117630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 -- 172.21.15.16:0/1260407191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564090018510 con 0x564090107c20 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107260 0x564090117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.416+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 0x56409010e1d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009a80 tx=0x7f9f24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86716d46e7f1773 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f217fa700 1 -- 172.21.15.16:0/1260407191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240079f0 con 0x564090107c20 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f217fa700 1 -- 172.21.15.16:0/1260407191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f2400f030 con 0x564090107c20 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f217fa700 1 -- 172.21.15.16:0/1260407191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24016e50 con 0x564090107c20 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 -- 172.21.15.16:0/1260407191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 msgr2=0x56409010e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 0x56409010e1d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009a80 tx=0x7f9f24009040 comp rx=0 tx=0).stop 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 -- 172.21.15.16:0/1260407191 shutdown_connections 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107c20 0x56409010e1d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090106900 0x56409010f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 --2- 172.21.15.16:0/1260407191 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090107260 0x564090117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 -- 172.21.15.16:0/1260407191 >> 172.21.15.16:0/1260407191 conn(0x5640900601c0 msgr2=0x564090108000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 -- 172.21.15.16:0/1260407191 shutdown_connections 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.417+0000 7f9f408bab80 1 -- 172.21.15.16:0/1260407191 wait complete. 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 -- start start 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 Processor -- start 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090111850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x564090111d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090116a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564090116fd0 con 0x564090107260 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564090117110 con 0x564090106900 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640901172d0 con 0x564090107c20 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f408bab80 10 client.0 _ll_register_callbacks cb 0x56408fff7ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f31c9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x564090111d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f31c9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x564090111d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58072/0 (socket says 172.21.15.16:58072) 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f31c9b700 1 -- 172.21.15.16:0/255249588 learned_addr learned my addr 172.21.15.16:0/255249588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f32c9d700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090116a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.418+0000 7f9f3249c700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090111850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f31c9b700 1 -- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 msgr2=0x564090116a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090116a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f31c9b700 1 -- 172.21.15.16:0/255249588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 msgr2=0x564090111850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090111850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f31c9b700 1 -- 172.21.15.16:0/255249588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24011040 con 0x564090107260 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f31c9b700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x564090111d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0098e0 tx=0x7f9f1c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cc88c42f1f2bef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f15ffb700 1 -- 172.21.15.16:0/255249588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c024ca0 con 0x564090107260 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f15ffb700 1 -- 172.21.15.16:0/255249588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9f1c0223a0 con 0x564090107260 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f1c01e040 con 0x564090107260 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f15ffb700 1 -- 172.21.15.16:0/255249588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f1c022560 con 0x564090107260 2024-05-08T23:19:09.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f408bab80 10 client.15240 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56408fed7c80 con 0x564090107260 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.419+0000 7f9f15ffb700 1 -- 172.21.15.16:0/255249588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f1c01e040 con 0x564090107260 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f15ffb700 1 -- 172.21.15.16:0/255249588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9f1c040070 con 0x564090107260 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f408bab80 10 client.15240 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f408bab80 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f408bab80 20 client.15240 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f408bab80 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564090175df0 0x5640901727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f408bab80 4 client.15240 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56409004ae70 con 0x564090175df0 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f3249c700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564090175df0 0x5640901727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f15ffb700 10 client.15240 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.420+0000 7f9f3249c700 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564090175df0 0x5640901727b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b418de4ac8f3689b server_cookie=885712e3f7d8bb0 in_seq=0 out_seq=0 2024-05-08T23:19:09.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f15ffb700 1 -- 172.21.15.16:0/255249588 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56409004ae70 con 0x564090175df0 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f15ffb700 10 client.15240 handle_command_reply: tid=0 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 client.15240 shutdown 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 msgr2=0x564090111d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x564090111d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0098e0 tx=0x7f9f1c004790 comp rx=0 tx=0).stop 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 shutdown_connections 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564090107c20 0x564090116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564090175df0 0x5640901727b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564090106900 0x564090111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 --2- 172.21.15.16:0/255249588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564090107260 0x564090111d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f15ffb700 0 client.15240 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 >> 172.21.15.16:0/255249588 conn(0x5640900601c0 msgr2=0x56409010ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 shutdown_connections 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 1 -- 172.21.15.16:0/255249588 wait complete. 2024-05-08T23:19:09.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.421+0000 7f9f408bab80 20 client.15240 trim_cache size 0 max 16384 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24529, 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 57.736332249999997, 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 83, 2024-05-08T23:19:09.428 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:19:09.429 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 98.331515414999998 2024-05-08T23:19:09.429 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:09.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f180a11b0 msgr2=0x7f9f180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f180a11b0 0x7f9f180a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006010 tx=0x7f9f1c00b5b0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 msgr2=0x7f9f2c196e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 0x7f9f2c196e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e970 tx=0x7f9f2800ec80 comp rx=0 tx=0).stop 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 shutdown_connections 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f9f180a11b0 0x7f9f180a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f2c102ab0 0x7f9f2c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c100f20 0x7f9f2c196e70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 --2- 172.21.15.16:0/52852815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f2c1020f0 0x7f9f2c1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 >> 172.21.15.16:0/52852815 conn(0x7f9f2c0fc4e0 msgr2=0x7f9f2c105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 shutdown_connections 2024-05-08T23:19:09.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.422+0000 7f9f34700700 1 -- 172.21.15.16:0/52852815 wait complete. 2024-05-08T23:19:09.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T23:19:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 -- 172.21.15.16:0/2873755675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c101a00 msgr2=0x7fb15c103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 --2- 172.21.15.16:0/2873755675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c101a00 0x7fb15c103df0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb158009990 tx=0x7fb158009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 -- 172.21.15.16:0/2873755675 shutdown_connections 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 --2- 172.21.15.16:0/2873755675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c106c60 0x7fb15c1090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 --2- 172.21.15.16:0/2873755675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c101a00 0x7fb15c103df0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 --2- 172.21.15.16:0/2873755675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c104330 0x7fb15c106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 -- 172.21.15.16:0/2873755675 >> 172.21.15.16:0/2873755675 conn(0x7fb15c0fb3f0 msgr2=0x7fb15c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 -- 172.21.15.16:0/2873755675 shutdown_connections 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.573+0000 7fb162acb700 1 -- 172.21.15.16:0/2873755675 wait complete. 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 Processor -- start 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 -- start start 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c101a00 0x7fb15c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 0x7fb15c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c106c60 0x7fb15c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c19e930 con 0x7fb15c104330 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c19ea70 con 0x7fb15c106c60 2024-05-08T23:19:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb162acb700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c19ec00 con 0x7fb15c101a00 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb153fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 0x7fb15c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb161068700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c106c60 0x7fb15c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.574+0000 7fb153fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 0x7fb15c197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58092/0 (socket says 172.21.15.16:58092) 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb160867700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c101a00 0x7fb15c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 -- 172.21.15.16:0/969297301 learned_addr learned my addr 172.21.15.16:0/969297301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 -- 172.21.15.16:0/969297301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c101a00 msgr2=0x7fb15c196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c101a00 0x7fb15c196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 -- 172.21.15.16:0/969297301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c106c60 msgr2=0x7fb15c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c106c60 0x7fb15c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 -- 172.21.15.16:0/969297301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580095f0 con 0x7fb15c104330 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb161068700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c106c60 0x7fb15c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb160867700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c101a00 0x7fb15c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb153fff700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 0x7fb15c197340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00d950 tx=0x7fb14c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb151ffb700 1 -- 172.21.15.16:0/969297301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00f890 con 0x7fb15c104330 2024-05-08T23:19:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb151ffb700 1 -- 172.21.15.16:0/969297301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb14c00fe80 con 0x7fb15c104330 2024-05-08T23:19:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb151ffb700 1 -- 172.21.15.16:0/969297301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00e3b0 con 0x7fb15c104330 2024-05-08T23:19:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.575+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c19eee0 con 0x7fb15c104330 2024-05-08T23:19:09.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.576+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c19f360 con 0x7fb15c104330 2024-05-08T23:19:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.578+0000 7fb151ffb700 1 -- 172.21.15.16:0/969297301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb14c010460 con 0x7fb15c104330 2024-05-08T23:19:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.578+0000 7fb151ffb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb1440a1230 0x7fb1440a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.578+0000 7fb151ffb700 1 -- 172.21.15.16:0/969297301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb14c0ccaa0 con 0x7fb15c104330 2024-05-08T23:19:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.578+0000 7fb160867700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb1440a1230 0x7fb1440a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.579+0000 7fb160867700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb1440a1230 0x7fb1440a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb158003780 tx=0x7fb158005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.579+0000 7fb16ec85b80 1 Processor -- start 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.579+0000 7fb16ec85b80 1 -- start start 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1de50 0x558710f1e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e74100 con 0x558710f1d4f0 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e71ac0 con 0x558710f1e810 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e6d3b0 con 0x558710f1de50 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58098/0 (socket says 172.21.15.16:58098) 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 -- 172.21.15.16:0/1783763679 learned_addr learned my addr 172.21.15.16:0/1783763679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb153fff700 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1de50 0x558710f1e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb161068700 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 -- 172.21.15.16:0/1783763679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1de50 msgr2=0x558710f1e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1de50 0x558710f1e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 -- 172.21.15.16:0/1783763679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 msgr2=0x558710f1ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f1ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 -- 172.21.15.16:0/1783763679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558710cee360 con 0x558710f1d4f0 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb160867700 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb158015000 tx=0x7fb158020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dfd246d9509ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.580+0000 7fb142ffd700 1 -- 172.21.15.16:0/1783763679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15802b070 con 0x558710f1d4f0 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb142ffd700 1 -- 172.21.15.16:0/1783763679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb15801e040 con 0x558710f1d4f0 2024-05-08T23:19:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb142ffd700 1 -- 172.21.15.16:0/1783763679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158004910 con 0x558710f1d4f0 2024-05-08T23:19:09.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 -- 172.21.15.16:0/1783763679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 msgr2=0x558710f1d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb158015000 tx=0x7fb158020000 comp rx=0 tx=0).stop 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 -- 172.21.15.16:0/1783763679 shutdown_connections 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f1ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d910 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/1783763679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1de50 0x558710f1e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 -- 172.21.15.16:0/1783763679 >> 172.21.15.16:0/1783763679 conn(0x558710e7b1c0 msgr2=0x558710f17d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 -- 172.21.15.16:0/1783763679 shutdown_connections 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.581+0000 7fb16ec85b80 1 -- 172.21.15.16:0/1783763679 wait complete. 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 -- start start 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 Processor -- start 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1d4f0 0x558710f26d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f27260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e09510 con 0x558710f1de50 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e091b0 con 0x558710f1d4f0 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710cf0650 con 0x558710f1e810 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb16ec85b80 10 client.0 _ll_register_callbacks cb 0x558710e12ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.582+0000 7fb153fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f27260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f27260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58108/0 (socket says 172.21.15.16:58108) 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 -- 172.21.15.16:0/3366660875 learned_addr learned my addr 172.21.15.16:0/3366660875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb160867700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1d4f0 0x558710f26d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb161068700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 -- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 msgr2=0x558710f2e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f2e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 -- 172.21.15.16:0/3366660875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1d4f0 msgr2=0x558710f26d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1d4f0 0x558710f26d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 -- 172.21.15.16:0/3366660875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c109cc0 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb153fff700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f27260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb14c099790 tx=0x7fb14c0996e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cedd432106eaf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb13bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb13bfff700 1 -- 172.21.15.16:0/3366660875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c099180 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558710dc6220 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb13bfff700 1 -- 172.21.15.16:0/3366660875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb14c09d8c0 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb13bfff700 1 -- 172.21.15.16:0/3366660875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14c0a1f10 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb16ec85b80 10 client.15252 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.583+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558710c9ab10 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb13bfff700 1 -- 172.21.15.16:0/3366660875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558710dc6220 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb13bfff700 1 -- 172.21.15.16:0/3366660875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb14c0a1b10 con 0x558710f1de50 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb16ec85b80 10 client.15252 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:09.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb16ec85b80 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb16ec85b80 20 client.15252 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f31f00 0x558710f371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb16ec85b80 4 client.15252 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558710eec7c0 con 0x558710f31f00 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.584+0000 7fb160867700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f31f00 0x558710f371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.585+0000 7fb13bfff700 10 client.15252 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.585+0000 7fb160867700 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f31f00 0x558710f371e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8b671e041069da6 server_cookie=1ff830e6bec4ba38 in_seq=0 out_seq=0 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb13bfff700 1 -- 172.21.15.16:0/3366660875 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558710eec7c0 con 0x558710f31f00 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb13bfff700 10 client.15252 handle_command_reply: tid=0 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 client.15252 shutdown 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 msgr2=0x558710f27260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f27260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb14c099790 tx=0x7fb14c0996e0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 shutdown_connections 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f2e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f31f00 0x558710f371e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1d4f0 0x558710f26d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/3366660875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f27260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb13bfff700 0 client.15252 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 >> 172.21.15.16:0/3366660875 conn(0x558710e7b1c0 msgr2=0x558710f17550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 shutdown_connections 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 1 -- 172.21.15.16:0/3366660875 wait complete. 2024-05-08T23:19:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.586+0000 7fb16ec85b80 20 client.15252 trim_cache size 0 max 16384 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 Processor -- start 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 -- start start 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1de50 0x558710f2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f24d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e2b2b0 con 0x558710f1d4f0 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e66150 con 0x558710f1de50 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710e0c5a0 con 0x558710f1e810 2024-05-08T23:19:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb160867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb160867700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58114/0 (socket says 172.21.15.16:58114) 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb160867700 1 -- 172.21.15.16:0/4099485130 learned_addr learned my addr 172.21.15.16:0/4099485130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.605+0000 7fb153fff700 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1de50 0x558710f2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb161068700 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f24d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb160867700 1 -- 172.21.15.16:0/4099485130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 msgr2=0x558710f24d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb160867700 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f24d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb160867700 1 -- 172.21.15.16:0/4099485130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1de50 msgr2=0x558710f2e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb160867700 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1de50 0x558710f2e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb160867700 1 -- 172.21.15.16:0/4099485130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558710e33510 con 0x558710f1d4f0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb160867700 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb1580098e0 tx=0x7fb15801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b560123e171dc728 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb1437fe700 1 -- 172.21.15.16:0/4099485130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15802b040 con 0x558710f1d4f0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb1437fe700 1 -- 172.21.15.16:0/4099485130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb15802c040 con 0x558710f1d4f0 2024-05-08T23:19:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.606+0000 7fb1437fe700 1 -- 172.21.15.16:0/4099485130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158004630 con 0x558710f1d4f0 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 -- 172.21.15.16:0/4099485130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 msgr2=0x558710f1d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb1580098e0 tx=0x7fb15801a040 comp rx=0 tx=0).stop 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 -- 172.21.15.16:0/4099485130 shutdown_connections 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1e810 0x558710f24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1d4f0 0x558710f1d8d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/4099485130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1de50 0x558710f2e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 -- 172.21.15.16:0/4099485130 >> 172.21.15.16:0/4099485130 conn(0x558710e7b1c0 msgr2=0x558710f1ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 -- 172.21.15.16:0/4099485130 shutdown_connections 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 -- 172.21.15.16:0/4099485130 wait complete. 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 -- start start 2024-05-08T23:19:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.607+0000 7fb16ec85b80 1 Processor -- start 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1d4f0 0x558710f283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f28920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710f2ddd0 con 0x558710f1de50 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710f2df40 con 0x558710f1e810 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558710f2e100 con 0x558710f1d4f0 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb16ec85b80 10 client.0 _ll_register_callbacks cb 0x558710e12ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f28920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f28920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58116/0 (socket says 172.21.15.16:58116) 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 -- 172.21.15.16:0/377396918 learned_addr learned my addr 172.21.15.16:0/377396918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb161068700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb160867700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1d4f0 0x558710f283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 -- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1d4f0 msgr2=0x558710f283e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1d4f0 0x558710f283e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 -- 172.21.15.16:0/377396918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 msgr2=0x558710f2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb153fff700 1 -- 172.21.15.16:0/377396918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb158033040 con 0x558710f1de50 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.608+0000 7fb160867700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1d4f0 0x558710f283e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb153fff700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f28920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb14c097de0 tx=0x7fb14c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82cfb2b418aa72f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb13bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb13bfff700 1 -- 172.21.15.16:0/377396918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c095ba0 con 0x558710f1de50 2024-05-08T23:19:09.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb13bfff700 1 -- 172.21.15.16:0/377396918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb14c0b2410 con 0x558710f1de50 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb15801e040 con 0x558710f1de50 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb13bfff700 1 -- 172.21.15.16:0/377396918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14c099860 con 0x558710f1de50 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 10 client.15258 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558710cf2c80 con 0x558710f1de50 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb13bfff700 1 -- 172.21.15.16:0/377396918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb15801e040 con 0x558710f1de50 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb13bfff700 1 -- 172.21.15.16:0/377396918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb14c0a6020 con 0x558710f1de50 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 20 client.15258 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f8cfe0 0x558710f89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 4 client.15258 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.609+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558710e65e70 con 0x558710f8cfe0 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb160867700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f8cfe0 0x558710f89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb13bfff700 10 client.15258 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb160867700 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f8cfe0 0x558710f89a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef19effee352a41b server_cookie=f7e182e38d2e10a9 in_seq=0 out_seq=0 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb13bfff700 1 -- 172.21.15.16:0/377396918 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558710e65e70 con 0x558710f8cfe0 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb13bfff700 10 client.15258 handle_command_reply: tid=0 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb16ec85b80 1 client.15258 shutdown 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 msgr2=0x558710f28920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.610+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f28920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb14c097de0 tx=0x7fb14c010000 comp rx=0 tx=0).stop 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 shutdown_connections 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558710f8cfe0 0x558710f89a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558710f1e810 0x558710f2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558710f1d4f0 0x558710f283e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb13bfff700 0 client.15258 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 --2- 172.21.15.16:0/377396918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558710f1de50 0x558710f28920 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 >> 172.21.15.16:0/377396918 conn(0x558710e7b1c0 msgr2=0x558710f26cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 shutdown_connections 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 1 -- 172.21.15.16:0/377396918 wait complete. 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb16ec85b80 20 client.15258 trim_cache size 0 max 16384 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24550, 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:19:09.617 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 58.931728374000002, 2024-05-08T23:19:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-05-08T23:19:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 83, 2024-05-08T23:19:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:19:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 94.930174178000001 2024-05-08T23:19:09.618 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb1440a1230 msgr2=0x7fb1440a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb1440a1230 0x7fb1440a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb158003780 tx=0x7fb158005fb0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 msgr2=0x7fb15c197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 0x7fb15c197340 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00d950 tx=0x7fb14c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 shutdown_connections 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb15c106c60 0x7fb15c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb1440a1230 0x7fb1440a36f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb15c101a00 0x7fb15c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 --2- 172.21.15.16:0/969297301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb15c104330 0x7fb15c197340 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.611+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 >> 172.21.15.16:0/969297301 conn(0x7fb15c0fb3f0 msgr2=0x7fb15c1023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.612+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 shutdown_connections 2024-05-08T23:19:09.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.612+0000 7fb162acb700 1 -- 172.21.15.16:0/969297301 wait complete. 2024-05-08T23:19:09.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:19:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 -- 172.21.15.16:0/812536909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 msgr2=0x7fcb6c1090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 --2- 172.21.15.16:0/812536909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1090b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b0a0 tx=0x7fcb6400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 -- 172.21.15.16:0/812536909 shutdown_connections 2024-05-08T23:19:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 --2- 172.21.15.16:0/812536909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1090b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 --2- 172.21.15.16:0/812536909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c101a40 0x7fcb6c103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 --2- 172.21.15.16:0/812536909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb6c104370 0x7fcb6c106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 -- 172.21.15.16:0/812536909 >> 172.21.15.16:0/812536909 conn(0x7fcb6c0fb3f0 msgr2=0x7fcb6c0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 -- 172.21.15.16:0/812536909 shutdown_connections 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.764+0000 7fcb72939700 1 -- 172.21.15.16:0/812536909 wait complete. 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 Processor -- start 2024-05-08T23:19:09.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 -- start start 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb6c101a40 0x7fcb6c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c104370 0x7fcb6c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1a0c10 con 0x7fcb6c106ca0 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1a0d50 con 0x7fcb6c104370 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb72939700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1a0ee0 con 0x7fcb6c101a40 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58132/0 (socket says 172.21.15.16:58132) 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 -- 172.21.15.16:0/878515696 learned_addr learned my addr 172.21.15.16:0/878515696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c104370 0x7fcb6c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb6c101a40 0x7fcb6c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 -- 172.21.15.16:0/878515696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb6c101a40 msgr2=0x7fcb6c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb6c101a40 0x7fcb6c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 -- 172.21.15.16:0/878515696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c104370 msgr2=0x7fcb6c199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.765+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c104370 0x7fcb6c199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb70ed6700 1 -- 172.21.15.16:0/878515696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb64009d00 con 0x7fcb6c106ca0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1a0640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009820 tx=0x7fcb6400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb697fa700 1 -- 172.21.15.16:0/878515696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400e040 con 0x7fcb6c106ca0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb697fa700 1 -- 172.21.15.16:0/878515696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb64004c50 con 0x7fcb6c106ca0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c1a1160 con 0x7fcb6c106ca0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb697fa700 1 -- 172.21.15.16:0/878515696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640052a0 con 0x7fcb6c106ca0 2024-05-08T23:19:09.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.766+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c1a1630 con 0x7fcb6c106ca0 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.768+0000 7fcb697fa700 1 -- 172.21.15.16:0/878515696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcb6401d470 con 0x7fcb6c106ca0 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.769+0000 7fcb697fa700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcb540a1230 0x7fcb540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.769+0000 7fcb697fa700 1 -- 172.21.15.16:0/878515696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcb640ce1e0 con 0x7fcb6c106ca0 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.769+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcb540a1230 0x7fcb540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.769+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcb540a1230 0x7fcb540a36f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009c60 tx=0x7fcb5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 Processor -- start 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 -- start start 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09571f100 con 0x55d0957c8e50 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09571cac0 con 0x55d0957c84f0 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0957183b0 con 0x55d0957c9810 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58134/0 (socket says 172.21.15.16:58134) 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/3564821557 learned_addr learned my addr 172.21.15.16:0/3564821557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.770+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/3564821557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 msgr2=0x55d0957c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/3564821557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 msgr2=0x55d0957c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/3564821557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d095599360 con 0x55d0957c8e50 2024-05-08T23:19:09.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957c92d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000bd60 tx=0x7fcb600106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40681c1ee47d04fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb527fc700 1 -- 172.21.15.16:0/3564821557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60010e10 con 0x55d0957c8e50 2024-05-08T23:19:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb527fc700 1 -- 172.21.15.16:0/3564821557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb60015070 con 0x55d0957c8e50 2024-05-08T23:19:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.771+0000 7fcb527fc700 1 -- 172.21.15.16:0/3564821557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600116a0 con 0x55d0957c8e50 2024-05-08T23:19:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/3564821557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 msgr2=0x55d0957c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957c92d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000bd60 tx=0x7fcb600106b0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/3564821557 shutdown_connections 2024-05-08T23:19:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/3564821557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957c92d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/3564821557 >> 172.21.15.16:0/3564821557 conn(0x55d0957261c0 msgr2=0x55d0957c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/3564821557 shutdown_connections 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/3564821557 wait complete. 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.772+0000 7fcb7eaf3b80 1 -- start start 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 Processor -- start 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0956b4510 con 0x55d0957c8e50 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0956b41b0 con 0x55d0957c84f0 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d09559b650 con 0x55d0957c9810 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb7eaf3b80 10 client.0 _ll_register_callbacks cb 0x55d0956bdae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58144/0 (socket says 172.21.15.16:58144) 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/2827210843 learned_addr learned my addr 172.21.15.16:0/2827210843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/2827210843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 msgr2=0x55d0957d9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 msgr2=0x55d0957d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.773+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/2827210843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6000ba20 con 0x55d0957c8e50 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d22d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000bd30 tx=0x7fcb60010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e26df80db9c6157 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/2827210843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000beb0 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d095671220 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/2827210843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcb60017070 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/2827210843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb6001c690 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 10 client.15270 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d095545b10 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/2827210843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d095671220 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/2827210843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb600038e0 con 0x55d0957c8e50 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 20 client.15270 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d0957dce90 0x55d0957e2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 4 client.15270 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.774+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0957977c0 con 0x55d0957dce90 2024-05-08T23:19:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.775+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d0957dce90 0x55d0957e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.775+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d0957dce90 0x55d0957e2190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc898bb382429eae server_cookie=8c3b9ec50b7d0f55 in_seq=0 out_seq=0 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.775+0000 7fcb4b7fe700 10 client.15270 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/2827210843 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0957977c0 con 0x55d0957dce90 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb4b7fe700 10 client.15270 handle_command_reply: tid=0 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 client.15270 shutdown 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 msgr2=0x55d0957d22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d22d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000bd30 tx=0x7fcb60010ec0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 shutdown_connections 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d0957dce90 0x55d0957e2190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/2827210843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d22d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 >> 172.21.15.16:0/2827210843 conn(0x55d0957261c0 msgr2=0x55d0957c2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb4b7fe700 0 client.15270 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 shutdown_connections 2024-05-08T23:19:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.776+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/2827210843 wait complete. 2024-05-08T23:19:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.777+0000 7fcb7eaf3b80 20 client.15270 trim_cache size 0 max 16384 2024-05-08T23:19:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.795+0000 7fcb7eaf3b80 1 Processor -- start 2024-05-08T23:19:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 -- start start 2024-05-08T23:19:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c8e50 0x55d0957d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 0x55d0957c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d095715010 con 0x55d0957c9810 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0956b75a0 con 0x55d0957c84f0 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0957d06c0 con 0x55d0957c8e50 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 0x55d0957c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 0x55d0957c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58148/0 (socket says 172.21.15.16:58148) 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 -- 172.21.15.16:0/1297566902 learned_addr learned my addr 172.21.15.16:0/1297566902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c8e50 0x55d0957d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 -- 172.21.15.16:0/1297566902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c8e50 msgr2=0x55d0957d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c8e50 0x55d0957d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 -- 172.21.15.16:0/1297566902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 msgr2=0x55d0957d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 -- 172.21.15.16:0/1297566902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0956de510 con 0x55d0957c9810 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.796+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 0x55d0957c3630 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcb64097760 tx=0x7fcb640a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e78cd79a013415d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb52ffd700 1 -- 172.21.15.16:0/1297566902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640cbea0 con 0x55d0957c9810 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb52ffd700 1 -- 172.21.15.16:0/1297566902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb640ce030 con 0x55d0957c9810 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb52ffd700 1 -- 172.21.15.16:0/1297566902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64099690 con 0x55d0957c9810 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1297566902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 msgr2=0x55d0957c3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 0x55d0957c3630 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcb64097760 tx=0x7fcb640a1040 comp rx=0 tx=0).stop 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1297566902 shutdown_connections 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c9810 0x55d0957c3630 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1297566902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c8e50 0x55d0957d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1297566902 >> 172.21.15.16:0/1297566902 conn(0x55d0957261c0 msgr2=0x55d0957c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1297566902 shutdown_connections 2024-05-08T23:19:09.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.797+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1297566902 wait complete. 2024-05-08T23:19:09.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 -- start start 2024-05-08T23:19:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 Processor -- start 2024-05-08T23:19:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0957d8fd0 con 0x55d0957c8e50 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0957d9140 con 0x55d0957c84f0 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.798+0000 7fcb7eaf3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0957d9300 con 0x55d0957c9810 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb7eaf3b80 10 client.0 _ll_register_callbacks cb 0x55d0956bdae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58160/0 (socket says 172.21.15.16:58160) 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/1024020267 learned_addr learned my addr 172.21.15.16:0/1024020267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb70ed6700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/1024020267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 msgr2=0x55d0957d88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 msgr2=0x55d0957c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 -- 172.21.15.16:0/1024020267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb64095e70 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb6b7fe700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d8270 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb600106f0 tx=0x7fcb600038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5b26812d0dd40f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/1024020267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600054d0 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/1024020267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcb60017030 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb60015070 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/1024020267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb6000beb0 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.799+0000 7fcb7eaf3b80 10 client.15276 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d09559dc80 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/1024020267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb60015070 con 0x55d0957c8e50 2024-05-08T23:19:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/1024020267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb60011490 con 0x55d0957c8e50 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 10 client.15276 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 20 client.15276 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d095838420 0x55d095834d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 4 client.15276 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d095710e70 con 0x55d095838420 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.800+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d095838420 0x55d095834d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb4b7fe700 10 client.15276 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb6bfff700 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d095838420 0x55d095834d70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8c4fbcb75edeb9 server_cookie=8881bfa25a93a264 in_seq=0 out_seq=0 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb4b7fe700 1 -- 172.21.15.16:0/1024020267 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d095710e70 con 0x55d095838420 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb4b7fe700 10 client.15276 handle_command_reply: tid=0 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 client.15276 shutdown 2024-05-08T23:19:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 msgr2=0x55d0957d8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d8270 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb600106f0 tx=0x7fcb600038b0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 shutdown_connections 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0957c9810 0x55d0957d88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d095838420 0x55d095834d70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d0957c84f0 0x55d0957c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 --2- 172.21.15.16:0/1024020267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d0957c8e50 0x55d0957d8270 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb4b7fe700 0 client.15276 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 >> 172.21.15.16:0/1024020267 conn(0x55d0957261c0 msgr2=0x55d0957cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 shutdown_connections 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 1 -- 172.21.15.16:0/1024020267 wait complete. 2024-05-08T23:19:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.801+0000 7fcb7eaf3b80 20 client.15276 trim_cache size 0 max 16384 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24563, 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 83.719763630000003, 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-05-08T23:19:09.808 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 83, 2024-05-08T23:19:09.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:19:09.809 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 89.748847667999996 2024-05-08T23:19:09.809 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcb540a1230 msgr2=0x7fcb540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcb540a1230 0x7fcb540a36f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009c60 tx=0x7fcb5c009260 comp rx=0 tx=0).stop 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 msgr2=0x7fcb6c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1a0640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009820 tx=0x7fcb6400bab0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 shutdown_connections 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb6c106ca0 0x7fcb6c1a0640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fcb540a1230 0x7fcb540a36f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb6c101a40 0x7fcb6c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 --2- 172.21.15.16:0/878515696 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb6c104370 0x7fcb6c199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 >> 172.21.15.16:0/878515696 conn(0x7fcb6c0fb3f0 msgr2=0x7fcb6c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 shutdown_connections 2024-05-08T23:19:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:09.802+0000 7fcb72939700 1 -- 172.21.15.16:0/878515696 wait complete. 2024-05-08T23:19:09.815 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T23:19:09.816 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T23:19:09.825 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:19:09.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T23:19:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:10.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:09 smithi016 ceph-mon[42541]: pgmap v266: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:09 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1739282239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:10.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:09 smithi189 ceph-mon[46919]: pgmap v266: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:09 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1739282239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:09 smithi155 ceph-mon[47105]: pgmap v266: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:09 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1739282239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 -- 172.21.15.16:0/1036820041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac101a20 msgr2=0x7fedac108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1036820041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac101a20 0x7fedac108340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feda4009990 tx=0x7feda4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 -- 172.21.15.16:0/1036820041 shutdown_connections 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1036820041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac101a20 0x7fedac108340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1036820041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac0fff00 0x7fedac100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1036820041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101060 0x7fedac1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 -- 172.21.15.16:0/1036820041 >> 172.21.15.16:0/1036820041 conn(0x7fedac0fb4a0 msgr2=0x7fedac0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 -- 172.21.15.16:0/1036820041 shutdown_connections 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.106+0000 7fedb0d32700 1 -- 172.21.15.16:0/1036820041 wait complete. 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 Processor -- start 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 -- start start 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0fff00 0x7fedac10a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac101060 0x7fedac108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101a20 0x7fedac109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:11.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac109b10 con 0x7fedac0fff00 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac109c80 con 0x7fedac101a20 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedb0d32700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1a0e20 con 0x7fedac101060 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedaa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0fff00 0x7fedac10a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedaa59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0fff00 0x7fedac10a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58170/0 (socket says 172.21.15.16:58170) 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedaa59c700 1 -- 172.21.15.16:0/1249877014 learned_addr learned my addr 172.21.15.16:0/1249877014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7fedaad9d700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101a20 0x7fedac109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.107+0000 7feda9d9b700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac101060 0x7fedac108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedaad9d700 1 -- 172.21.15.16:0/1249877014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac101060 msgr2=0x7fedac108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedaad9d700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac101060 0x7fedac108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedaad9d700 1 -- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0fff00 msgr2=0x7fedac10a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedaad9d700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0fff00 0x7fedac10a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedaad9d700 1 -- 172.21.15.16:0/1249877014 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40095f0 con 0x7fedac101a20 2024-05-08T23:19:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedaad9d700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101a20 0x7fedac109370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feda40059d0 tx=0x7feda4003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda401d070 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feda400bd10 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac1a1050 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4022b30 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.108+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1a1460 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.109+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac107300 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.110+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7feda4003e50 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.111+0000 7fed9b7fe700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed940a11b0 0x7fed940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.111+0000 7fedaa59c700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed940a11b0 0x7fed940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.111+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7feda40ce6f0 con 0x7fedac101a20 2024-05-08T23:19:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.112+0000 7fedaa59c700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed940a11b0 0x7fed940a3670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009550 tx=0x7fed9c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:11.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.115+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feda4097a50 con 0x7fedac101a20 2024-05-08T23:19:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.349+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedac02ce10 con 0x7fed940a11b0 2024-05-08T23:19:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.350+0000 7fed9b7fe700 1 -- 172.21.15.16:0/1249877014 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fedac02ce10 con 0x7fed940a11b0 2024-05-08T23:19:11.350 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-05-08T23:19:11.350 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-05-08T23:19:11.350 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-05-08T23:19:11.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed940a11b0 msgr2=0x7fed940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed940a11b0 0x7fed940a3670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009550 tx=0x7fed9c009040 comp rx=0 tx=0).stop 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101a20 msgr2=0x7fedac109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101a20 0x7fedac109370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feda40059d0 tx=0x7feda4003a50 comp rx=0 tx=0).stop 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 shutdown_connections 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedac101a20 0x7fedac109370 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac0fff00 0x7fedac10a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fed940a11b0 0x7fed940a3670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 --2- 172.21.15.16:0/1249877014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac101060 0x7fedac108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 >> 172.21.15.16:0/1249877014 conn(0x7fedac0fb4a0 msgr2=0x7fedac104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.352+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 shutdown_connections 2024-05-08T23:19:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:11.353+0000 7fedb0d32700 1 -- 172.21.15.16:0/1249877014 wait complete. 2024-05-08T23:19:11.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T23:19:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:12.078 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:11 smithi016 ceph-mon[42541]: pgmap v267: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:12.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:11 smithi189 ceph-mon[46919]: pgmap v267: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:12.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:11 smithi155 ceph-mon[47105]: pgmap v267: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 -- 172.21.15.16:0/2255799717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c101a90 msgr2=0x7f189c103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 --2- 172.21.15.16:0/2255799717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c101a90 0x7f189c103e80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f188c009990 tx=0x7f188c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 -- 172.21.15.16:0/2255799717 shutdown_connections 2024-05-08T23:19:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 --2- 172.21.15.16:0/2255799717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189c106cf0 0x7f189c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 --2- 172.21.15.16:0/2255799717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c101a90 0x7f189c103e80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 --2- 172.21.15.16:0/2255799717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f189c1043c0 0x7f189c1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 -- 172.21.15.16:0/2255799717 >> 172.21.15.16:0/2255799717 conn(0x7f189c0fb4a0 msgr2=0x7f189c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 -- 172.21.15.16:0/2255799717 shutdown_connections 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.996+0000 7f18a1350700 1 -- 172.21.15.16:0/2255799717 wait complete. 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 Processor -- start 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 -- start start 2024-05-08T23:19:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f189c101a90 0x7f189c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 0x7f189c0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189c106cf0 0x7f189c0fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c100720 con 0x7f189c1043c0 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c100890 con 0x7f189c106cf0 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f18a1350700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c1a0d60 con 0x7f189c101a90 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f189a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 0x7f189c0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f189a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 0x7f189c0ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58190/0 (socket says 172.21.15.16:58190) 2024-05-08T23:19:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.997+0000 7f189a7fc700 1 -- 172.21.15.16:0/3612094379 learned_addr learned my addr 172.21.15.16:0/3612094379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189affd700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f189c101a90 0x7f189c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189b7fe700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189c106cf0 0x7f189c0fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189a7fc700 1 -- 172.21.15.16:0/3612094379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f189c101a90 msgr2=0x7f189c1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189a7fc700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f189c101a90 0x7f189c1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189a7fc700 1 -- 172.21.15.16:0/3612094379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189c106cf0 msgr2=0x7f189c0fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189a7fc700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189c106cf0 0x7f189c0fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189a7fc700 1 -- 172.21.15.16:0/3612094379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0095f0 con 0x7f189c1043c0 2024-05-08T23:19:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f189a7fc700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 0x7f189c0ffa40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f189000c9b0 tx=0x7f189000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890007b00 con 0x7f189c1043c0 2024-05-08T23:19:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f189000ce50 con 0x7f189c1043c0 2024-05-08T23:19:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c1a0ff0 con 0x7f189c1043c0 2024-05-08T23:19:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890018420 con 0x7f189c1043c0 2024-05-08T23:19:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.998+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c1a1460 con 0x7f189c1043c0 2024-05-08T23:19:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:12.999+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c0fd020 con 0x7f189c1043c0 2024-05-08T23:19:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.001+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f189000f450 con 0x7f189c1043c0 2024-05-08T23:19:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.002+0000 7f1883fff700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18840a11b0 0x7f18840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.002+0000 7f189affd700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18840a11b0 0x7f18840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.002+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f18900ccf40 con 0x7f189c1043c0 2024-05-08T23:19:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.003+0000 7f189affd700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18840a11b0 0x7f18840a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f188c000c00 tx=0x7f188c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.006+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1890096300 con 0x7f189c1043c0 2024-05-08T23:19:13.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:12 smithi016 ceph-mon[42541]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:13.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:12 smithi189 ceph-mon[46919]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:12 smithi155 ceph-mon[47105]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.228+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f189c02ce10 con 0x7f18840a11b0 2024-05-08T23:19:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.242+0000 7f1883fff700 1 -- 172.21.15.16:0/3612094379 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f189c02ce10 con 0x7f18840a11b0 2024-05-08T23:19:13.242 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T23:19:13.242 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (95s) 25s ago 94s 14.0M - 18.2.2-1322-gd21e4f96 af23193ff691 851fd1374674 2024-05-08T23:19:13.242 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi155 running (93s) 28s ago 93s 30.2M - 18.2.2-1322-gd21e4f96 af23193ff691 706614860b42 2024-05-08T23:19:13.242 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi189 running (102s) 28s ago 102s 18.0M - 18.2.2-1322-gd21e4f96 af23193ff691 79d40cc3c22b 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (107s) 25s ago 107s 17.5M - 18.2.2-1322-gd21e4f96 af23193ff691 99b1274150a5 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi155 running (105s) 28s ago 105s 17.4M - 18.2.2-1322-gd21e4f96 af23193ff691 1071281a8549 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi189 running (104s) 28s ago 103s 19.9M - 18.2.2-1322-gd21e4f96 af23193ff691 100f3e4f55e2 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (97s) 25s ago 96s 14.9M - 18.2.2-1322-gd21e4f96 af23193ff691 cd707b9ad13a 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi155 running (100s) 28s ago 100s 18.7M - 18.2.2-1322-gd21e4f96 af23193ff691 720a5d77ec36 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi189 running (98s) 28s ago 98s 17.3M - 18.2.2-1322-gd21e4f96 af23193ff691 9ebee2d14196 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (109s) 25s ago 109s 14.8M - 18.2.2-1322-gd21e4f96 af23193ff691 879126779f47 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi155 running (111s) 28s ago 111s 17.3M - 18.2.2-1322-gd21e4f96 af23193ff691 d176d04fc811 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi189 running (113s) 28s ago 112s 17.4M - 18.2.2-1322-gd21e4f96 af23193ff691 ca1e9c349a88 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (7m) 25s ago 7m 531M - 18.2.2-1322-gd21e4f96 af23193ff691 06b8e61a7de0 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi155 *:8443,8765 running (5m) 28s ago 5m 460M - 18.2.2-1322-gd21e4f96 af23193ff691 c7fcb23cddc6 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi189 *:8443,8765 running (5m) 28s ago 5m 461M - 18.2.2-1322-gd21e4f96 af23193ff691 cc1458c83870 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (7m) 25s ago 7m 89.8M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 180b5f12c5cd 2024-05-08T23:19:13.243 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi155 running (5m) 28s ago 5m 60.9M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 4560bb1106db 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi189 running (5m) 28s ago 5m 60.9M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 0483630aead4 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (5m) 25s ago 5m 146M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 db0dc7f40c53 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi155 running (5m) 28s ago 5m 149M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 1ddadda7360e 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi189 running (4m) 28s ago 4m 71.2M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 b51bec67ef49 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (4m) 25s ago 4m 141M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 d6e2db5e4bb1 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi155 running (4m) 28s ago 4m 64.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 88de832a9063 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi189 running (3m) 28s ago 3m 142M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 0f94c0348a7c 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (3m) 25s ago 3m 136M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 1c46d3579b3a 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi155 running (3m) 28s ago 3m 137M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 f09f7580168c 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi189 running (3m) 28s ago 3m 63.1M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 22bfb8114824 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (2m) 25s ago 2m 61.0M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 9f2203d0b05b 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi155 running (2m) 28s ago 2m 135M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 54dbf35fdb57 2024-05-08T23:19:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi189 running (2m) 28s ago 2m 127M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 c5b7b42052ec 2024-05-08T23:19:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18840a11b0 msgr2=0x7f18840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18840a11b0 0x7f18840a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f188c000c00 tx=0x7f188c005fb0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 msgr2=0x7f189c0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 0x7f189c0ffa40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f189000c9b0 tx=0x7f189000ccc0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 shutdown_connections 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f189c106cf0 0x7f189c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f18840a11b0 0x7f18840a3670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f189c101a90 0x7f189c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 --2- 172.21.15.16:0/3612094379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189c1043c0 0x7f189c0ffa40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 >> 172.21.15.16:0/3612094379 conn(0x7f189c0fb4a0 msgr2=0x7f189c102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 shutdown_connections 2024-05-08T23:19:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:13.245+0000 7f18a1350700 1 -- 172.21.15.16:0/3612094379 wait complete. 2024-05-08T23:19:13.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T23:19:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:13 smithi189 ceph-mon[46919]: pgmap v268: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:14.199 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:13 smithi016 ceph-mon[42541]: pgmap v268: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:14.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:13 smithi155 ceph-mon[47105]: pgmap v268: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 -- 172.21.15.16:0/2797002212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f40fff00 msgr2=0x7f05f4100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 --2- 172.21.15.16:0/2797002212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f40fff00 0x7f05f4100320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009990 tx=0x7f05e4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 -- 172.21.15.16:0/2797002212 shutdown_connections 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 --2- 172.21.15.16:0/2797002212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4101a20 0x7f05f4108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 --2- 172.21.15.16:0/2797002212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f40fff00 0x7f05f4100320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 --2- 172.21.15.16:0/2797002212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f4101060 0x7f05f41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 -- 172.21.15.16:0/2797002212 >> 172.21.15.16:0/2797002212 conn(0x7f05f40fb4a0 msgr2=0x7f05f40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 -- 172.21.15.16:0/2797002212 shutdown_connections 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 -- 172.21.15.16:0/2797002212 wait complete. 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.973+0000 7f05fa541700 1 Processor -- start 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 -- start start 2024-05-08T23:19:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f40fff00 0x7f05f4103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 0x7f05f41024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4101a20 0x7f05f4102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41031b0 con 0x7f05f4101060 2024-05-08T23:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4103320 con 0x7f05f4101a20 2024-05-08T23:19:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05fa541700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41a0eb0 con 0x7f05f40fff00 2024-05-08T23:19:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 0x7f05f41024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 0x7f05f41024d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58214/0 (socket says 172.21.15.16:58214) 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f37fe700 1 -- 172.21.15.16:0/2816778426 learned_addr learned my addr 172.21.15.16:0/2816778426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f3fff700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f40fff00 0x7f05f4103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f8ade700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4101a20 0x7f05f4102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f37fe700 1 -- 172.21.15.16:0/2816778426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f40fff00 msgr2=0x7f05f4103e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.974+0000 7f05f37fe700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f40fff00 0x7f05f4103e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f37fe700 1 -- 172.21.15.16:0/2816778426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4101a20 msgr2=0x7f05f4102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f37fe700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4101a20 0x7f05f4102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f37fe700 1 -- 172.21.15.16:0/2816778426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40095f0 con 0x7f05f4101060 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f37fe700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 0x7f05f41024d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f05e800d950 tx=0x7f05e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8009940 con 0x7f05f4101060 2024-05-08T23:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f41a1140 con 0x7f05f4101060 2024-05-08T23:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05e8010460 con 0x7f05f4101060 2024-05-08T23:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e800f620 con 0x7f05f4101060 2024-05-08T23:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.975+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f41a1640 con 0x7f05f4101060 2024-05-08T23:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.976+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f404ec60 con 0x7f05f4101060 2024-05-08T23:19:14.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.977+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f05e80105d0 con 0x7f05f4101060 2024-05-08T23:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.978+0000 7f05f17fa700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f05dc0a11b0 0x7f05dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.978+0000 7f05f3fff700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f05dc0a11b0 0x7f05dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.978+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f05e80ccbe0 con 0x7f05f4101060 2024-05-08T23:19:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.978+0000 7f05f3fff700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f05dc0a11b0 0x7f05dc0a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f05e400c010 tx=0x7f05e4003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:14.982+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f05e8095f10 con 0x7f05f4101060 2024-05-08T23:19:15.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:14 smithi016 ceph-mon[42541]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:14 smithi189 ceph-mon[46919]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:15.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.205+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f05f4069090 con 0x7f05dc0a11b0 2024-05-08T23:19:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.211+0000 7f05f17fa700 1 -- 172.21.15.16:0/2816778426 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f05f4069090 con 0x7f05dc0a11b0 2024-05-08T23:19:15.211 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T23:19:15.211 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 30s ago 116s smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi155=b;smithi155=e;smithi155=h;smithi155=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-05-08T23:19:15.212 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 30s ago 5m smithi016=x;smithi155=y;smithi189=z;count:3 2024-05-08T23:19:15.212 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 30s ago 6m smithi016:172.21.15.16=a;smithi155:172.21.15.155=b;smithi189:172.21.15.189=c;count:3 2024-05-08T23:19:15.212 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 30s ago - 2024-05-08T23:19:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f05dc0a11b0 msgr2=0x7f05dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f05dc0a11b0 0x7f05dc0a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f05e400c010 tx=0x7f05e4003720 comp rx=0 tx=0).stop 2024-05-08T23:19:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 msgr2=0x7f05f41024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 0x7f05f41024d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f05e800d950 tx=0x7f05e800dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 shutdown_connections 2024-05-08T23:19:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05f4101a20 0x7f05f4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f05dc0a11b0 0x7f05dc0a3670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05f40fff00 0x7f05f4103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 --2- 172.21.15.16:0/2816778426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05f4101060 0x7f05f41024d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 >> 172.21.15.16:0/2816778426 conn(0x7f05f40fb4a0 msgr2=0x7f05f4068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 shutdown_connections 2024-05-08T23:19:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:15.213+0000 7f05fa541700 1 -- 172.21.15.16:0/2816778426 wait complete. 2024-05-08T23:19:15.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:14 smithi155 ceph-mon[47105]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:15.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T23:19:15.866 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:16.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:15 smithi016 ceph-mon[42541]: pgmap v269: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:16.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:15 smithi189 ceph-mon[46919]: pgmap v269: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:15 smithi155 ceph-mon[47105]: pgmap v269: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 -- 172.21.15.16:0/421565085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 msgr2=0x7fe600101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 --2- 172.21.15.16:0/421565085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe600101020 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b0a0 tx=0x7fe5f800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 -- 172.21.15.16:0/421565085 shutdown_connections 2024-05-08T23:19:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 --2- 172.21.15.16:0/421565085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe600101020 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 --2- 172.21.15.16:0/421565085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6000ff880 0x7fe6000ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 --2- 172.21.15.16:0/421565085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6001001e0 0x7fe600100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 -- 172.21.15.16:0/421565085 >> 172.21.15.16:0/421565085 conn(0x7fe6000fb4c0 msgr2=0x7fe6000fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 -- 172.21.15.16:0/421565085 shutdown_connections 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.860+0000 7fe607cca700 1 -- 172.21.15.16:0/421565085 wait complete. 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 Processor -- start 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 -- start start 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6000ff880 0x7fe6001990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6001001e0 0x7fe600199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:16.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe6001a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:16.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001a0c10 con 0x7fe600100ba0 2024-05-08T23:19:16.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001a0d50 con 0x7fe6001001e0 2024-05-08T23:19:16.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe607cca700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001a0ee0 con 0x7fe6000ff880 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe606267700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe6001a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe606267700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe6001a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57562/0 (socket says 172.21.15.16:57562) 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.861+0000 7fe606267700 1 -- 172.21.15.16:0/2886152829 learned_addr learned my addr 172.21.15.16:0/2886152829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe605a66700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6000ff880 0x7fe6001990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe605265700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6001001e0 0x7fe600199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe606267700 1 -- 172.21.15.16:0/2886152829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6000ff880 msgr2=0x7fe6001990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe606267700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6000ff880 0x7fe6001990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe606267700 1 -- 172.21.15.16:0/2886152829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6001001e0 msgr2=0x7fe600199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe606267700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6001001e0 0x7fe600199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe606267700 1 -- 172.21.15.16:0/2886152829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8009d00 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe606267700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe6001a0640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8004d30 tx=0x7fe5f8004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800e040 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5f800cec0 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6001a1160 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f801ba30 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.862+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6001a1620 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.863+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60010a660 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.865+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe5f801bb90 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.865+0000 7fe5f6ffd700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe5ec0a1230 0x7fe5ec0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.865+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fe5f80cea40 con 0x7fe600100ba0 2024-05-08T23:19:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.865+0000 7fe605a66700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe5ec0a1230 0x7fe5ec0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.866+0000 7fe605a66700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe5ec0a1230 0x7fe5ec0a36f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005950 tx=0x7fe5fc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:16.870+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe5f8097e00 con 0x7fe600100ba0 2024-05-08T23:19:17.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.089+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6000613e0 con 0x7fe5ec0a1230 2024-05-08T23:19:17.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.091+0000 7fe5f6ffd700 1 -- 172.21.15.16:0/2886152829 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fe6000613e0 con 0x7fe5ec0a1230 2024-05-08T23:19:17.091 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-05-08T23:19:17.091 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-05-08T23:19:17.091 INFO:teuthology.orchestra.run.smithi016.stdout:smithi155 172.21.15.155 2024-05-08T23:19:17.091 INFO:teuthology.orchestra.run.smithi016.stdout:smithi189 172.21.15.189 2024-05-08T23:19:17.091 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-05-08T23:19:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.093+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe5ec0a1230 msgr2=0x7fe5ec0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.093+0000 7fe607cca700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe5ec0a1230 0x7fe5ec0a36f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005950 tx=0x7fe5fc00a300 comp rx=0 tx=0).stop 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.093+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 msgr2=0x7fe6001a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.093+0000 7fe607cca700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe6001a0640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8004d30 tx=0x7fe5f8004d60 comp rx=0 tx=0).stop 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.093+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 shutdown_connections 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.093+0000 7fe607cca700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe600100ba0 0x7fe6001a0640 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.094+0000 7fe607cca700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fe5ec0a1230 0x7fe5ec0a36f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.094+0000 7fe607cca700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6000ff880 0x7fe6001990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.094+0000 7fe607cca700 1 --2- 172.21.15.16:0/2886152829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6001001e0 0x7fe600199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.094+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 >> 172.21.15.16:0/2886152829 conn(0x7fe6000fb4c0 msgr2=0x7fe6000fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:17.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.094+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 shutdown_connections 2024-05-08T23:19:17.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:17.094+0000 7fe607cca700 1 -- 172.21.15.16:0/2886152829 wait complete. 2024-05-08T23:19:17.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:16 smithi189 ceph-mon[46919]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:17.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:16 smithi155 ceph-mon[47105]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:17.261 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:16 smithi016 ceph-mon[42541]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:17.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T23:19:17.742 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:18.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:17 smithi189 ceph-mon[46919]: pgmap v270: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:18.186 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:17 smithi016 ceph-mon[42541]: pgmap v270: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:17 smithi155 ceph-mon[47105]: pgmap v270: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 -- 172.21.15.16:0/302784444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 msgr2=0x7fb240103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 --2- 172.21.15.16:0/302784444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240103ec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb230009990 tx=0x7fb230009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 -- 172.21.15.16:0/302784444 shutdown_connections 2024-05-08T23:19:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 --2- 172.21.15.16:0/302784444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb240106d30 0x7fb240109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 --2- 172.21.15.16:0/302784444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240103ec0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 --2- 172.21.15.16:0/302784444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb240104400 0x7fb2401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 -- 172.21.15.16:0/302784444 >> 172.21.15.16:0/302784444 conn(0x7fb2400fb4a0 msgr2=0x7fb2400fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 -- 172.21.15.16:0/302784444 shutdown_connections 2024-05-08T23:19:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.714+0000 7fb244efb700 1 -- 172.21.15.16:0/302784444 wait complete. 2024-05-08T23:19:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.715+0000 7fb244efb700 1 Processor -- start 2024-05-08T23:19:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.715+0000 7fb244efb700 1 -- start start 2024-05-08T23:19:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb244efb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb244efb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb240104400 0x7fb240197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb244efb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb240106d30 0x7fb24019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb244efb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24019e9f0 con 0x7fb240101ad0 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb244efb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24019eb30 con 0x7fb240104400 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb244efb700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb24019ecc0 con 0x7fb240106d30 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57584/0 (socket says 172.21.15.16:57584) 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 -- 172.21.15.16:0/3834953726 learned_addr learned my addr 172.21.15.16:0/3834953726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb235d9b700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb240104400 0x7fb240197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23ed9d700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb240106d30 0x7fb24019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 -- 172.21.15.16:0/3834953726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb240106d30 msgr2=0x7fb24019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb240106d30 0x7fb24019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 -- 172.21.15.16:0/3834953726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb240104400 msgr2=0x7fb240197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb240104400 0x7fb240197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.716+0000 7fb23e59c700 1 -- 172.21.15.16:0/3834953726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300095f0 con 0x7fb240101ad0 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.717+0000 7fb23e59c700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240196f00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb230003a50 tx=0x7fb230003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.717+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23001d070 con 0x7fb240101ad0 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.717+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb24019ef40 con 0x7fb240101ad0 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.717+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb23000bd70 con 0x7fb240101ad0 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.717+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230022bf0 con 0x7fb240101ad0 2024-05-08T23:19:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.717+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb24019f2a0 con 0x7fb240101ad0 2024-05-08T23:19:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.718+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2400fd020 con 0x7fb240101ad0 2024-05-08T23:19:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.720+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb230022d50 con 0x7fb240101ad0 2024-05-08T23:19:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.721+0000 7fb237fff700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb22c0a11b0 0x7fb22c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.721+0000 7fb235d9b700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb22c0a11b0 0x7fb22c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.721+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb2300ce860 con 0x7fb240101ad0 2024-05-08T23:19:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.722+0000 7fb235d9b700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb22c0a11b0 0x7fb22c0a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb228007c60 tx=0x7fb2280073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:18.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.725+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb230097bc0 con 0x7fb240101ad0 2024-05-08T23:19:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.947+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 --> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb24002ce10 con 0x7fb22c0a11b0 2024-05-08T23:19:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.948+0000 7fb237fff700 1 -- 172.21.15.16:0/3834953726 <== mgr.14152 v2:172.21.15.16:6800/781396085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fb24002ce10 con 0x7fb22c0a11b0 2024-05-08T23:19:18.948 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T23:19:18.948 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:19:18.948 INFO:teuthology.orchestra.run.smithi016.stdout:smithi155 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:19:18.948 INFO:teuthology.orchestra.run.smithi016.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T23:19:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb22c0a11b0 msgr2=0x7fb22c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb22c0a11b0 0x7fb22c0a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb228007c60 tx=0x7fb2280073d0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 msgr2=0x7fb240196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240196f00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb230003a50 tx=0x7fb230003a80 comp rx=0 tx=0).stop 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 shutdown_connections 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fb22c0a11b0 0x7fb22c0a3670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb240106d30 0x7fb24019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb240101ad0 0x7fb240196f00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 --2- 172.21.15.16:0/3834953726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb240104400 0x7fb240197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 >> 172.21.15.16:0/3834953726 conn(0x7fb2400fb4a0 msgr2=0x7fb240102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 shutdown_connections 2024-05-08T23:19:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:18.951+0000 7fb244efb700 1 -- 172.21.15.16:0/3834953726 wait complete. 2024-05-08T23:19:19.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:18 smithi189 ceph-mon[46919]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:19.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:18 smithi155 ceph-mon[47105]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:19.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:18 smithi016 ceph-mon[42541]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:19.332 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T23:19:19.344 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:19:19.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T23:19:19.601 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:20.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:19 smithi016 ceph-mon[42541]: pgmap v271: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:20.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:19 smithi016 ceph-mon[42541]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:20.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:19 smithi189 ceph-mon[46919]: pgmap v271: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:20.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:19 smithi189 ceph-mon[46919]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:20.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:19 smithi155 ceph-mon[47105]: pgmap v271: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:19 smithi155 ceph-mon[47105]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 -- 172.21.15.16:0/3683260463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 msgr2=0x7fee64100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 --2- 172.21.15.16:0/3683260463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64100320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fee54009990 tx=0x7fee54009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 -- 172.21.15.16:0/3683260463 shutdown_connections 2024-05-08T23:19:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 --2- 172.21.15.16:0/3683260463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64101a20 0x7fee64108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 --2- 172.21.15.16:0/3683260463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64100320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 --2- 172.21.15.16:0/3683260463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101060 0x7fee641014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 -- 172.21.15.16:0/3683260463 >> 172.21.15.16:0/3683260463 conn(0x7fee640fb4a0 msgr2=0x7fee640fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 -- 172.21.15.16:0/3683260463 shutdown_connections 2024-05-08T23:19:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.634+0000 7fee69163700 1 -- 172.21.15.16:0/3683260463 wait complete. 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.635+0000 7fee69163700 1 Processor -- start 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.635+0000 7fee69163700 1 -- start start 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee69163700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee69163700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64101060 0x7fee64197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee69163700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101a20 0x7fee6419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee69163700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419e980 con 0x7fee640fff00 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee69163700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419eac0 con 0x7fee64101a20 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee69163700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419ec50 con 0x7fee64101060 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57606/0 (socket says 172.21.15.16:57606) 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 -- 172.21.15.16:0/633223681 learned_addr learned my addr 172.21.15.16:0/633223681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee6259c700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64101060 0x7fee64197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee6359e700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101a20 0x7fee6419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 -- 172.21.15.16:0/633223681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64101060 msgr2=0x7fee64197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64101060 0x7fee64197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 -- 172.21.15.16:0/633223681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101a20 msgr2=0x7fee6419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101a20 0x7fee6419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 -- 172.21.15.16:0/633223681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540095f0 con 0x7fee640fff00 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee6359e700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101a20 0x7fee6419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee62d9d700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64196e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee540059d0 tx=0x7fee54005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.636+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5401d070 con 0x7fee640fff00 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.637+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fee54003f80 con 0x7fee640fff00 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.637+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6419eed0 con 0x7fee640fff00 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.637+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54022ba0 con 0x7fee640fff00 2024-05-08T23:19:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.637+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6419f230 con 0x7fee640fff00 2024-05-08T23:19:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.638+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6404ec60 con 0x7fee640fff00 2024-05-08T23:19:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.639+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fee54003af0 con 0x7fee640fff00 2024-05-08T23:19:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.639+0000 7fee4bfff700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fee4c0a11b0 0x7fee4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.640+0000 7fee6259c700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fee4c0a11b0 0x7fee4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.640+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fee540cebb0 con 0x7fee640fff00 2024-05-08T23:19:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.640+0000 7fee6259c700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fee4c0a11b0 0x7fee4c0a3670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fee58005950 tx=0x7fee5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.643+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fee5409b9a0 con 0x7fee640fff00 2024-05-08T23:19:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.908+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee6410b5c0 con 0x7fee640fff00 2024-05-08T23:19:20.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.909+0000 7fee4bfff700 1 -- 172.21.15.16:0/633223681 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2905 (secure 0 0 0) 0x7fee54027b70 con 0x7fee640fff00 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:e24 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 24 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-05-08T23:17:46.078099+0000 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-05-08T23:18:12.692850+0000 2024-05-08T23:19:20.909 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 3 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24563,1=24550,2=24529} 2024-05-08T23:19:20.910 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3,4] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24563} state up:active seq 3 addr [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{0:24521} state up:standby-replay seq 1 addr [v2:172.21.15.155:6836/1960878524,v1:172.21.15.155:6837/1960878524] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{1:24550} state up:active seq 11 addr [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{1:24517} state up:standby-replay seq 1 addr [v2:172.21.15.189:6834/3239398256,v1:172.21.15.189:6835/3239398256] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{2:24529} state up:active seq 12 addr [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{2:24497} state up:standby-replay seq 1 addr [v2:172.21.15.155:6834/1260766455,v1:172.21.15.155:6835/1260766455] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.911 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14568} state up:standby seq 1 addr [v2:172.21.15.16:6834/4018978287,v1:172.21.15.16:6835/4018978287] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{-1:14622} state up:standby seq 1 addr [v2:172.21.15.16:6838/495357433,v1:172.21.15.16:6839/495357433] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{-1:14634} state up:standby seq 1 addr [v2:172.21.15.16:6840/2654183877,v1:172.21.15.16:6841/2654183877] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{-1:24464} state up:standby seq 1 addr [v2:172.21.15.155:6832/2420649836,v1:172.21.15.155:6833/2420649836] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{-1:24469} state up:standby seq 1 addr [v2:172.21.15.189:6832/3253311334,v1:172.21.15.189:6833/3253311334] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.912 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:24485} state up:standby seq 1 addr [v2:172.21.15.16:6836/3189374355,v1:172.21.15.16:6837/3189374355] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fee4c0a11b0 msgr2=0x7fee4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fee4c0a11b0 0x7fee4c0a3670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fee58005950 tx=0x7fee5800b410 comp rx=0 tx=0).stop 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 msgr2=0x7fee64196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64196e40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee540059d0 tx=0x7fee54005740 comp rx=0 tx=0).stop 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 shutdown_connections 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fee64101a20 0x7fee6419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee640fff00 0x7fee64196e40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fee4c0a11b0 0x7fee4c0a3670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 --2- 172.21.15.16:0/633223681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64101060 0x7fee64197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:20.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 >> 172.21.15.16:0/633223681 conn(0x7fee640fb4a0 msgr2=0x7fee64068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:20.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 shutdown_connections 2024-05-08T23:19:20.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:20.911+0000 7fee69163700 1 -- 172.21.15.16:0/633223681 wait complete. 2024-05-08T23:19:20.914 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:19:21.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T23:19:21.548 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:22.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:21 smithi016 ceph-mon[42541]: pgmap v272: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:22.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:21 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/633223681' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:19:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:21 smithi189 ceph-mon[46919]: pgmap v272: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:22.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:21 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/633223681' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:19:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:21 smithi155 ceph-mon[47105]: pgmap v272: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-08T23:19:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:21 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/633223681' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T23:19:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.580+0000 7f31c10e0700 1 -- 172.21.15.16:0/1206896744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 msgr2=0x7f31bc109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 --2- 172.21.15.16:0/1206896744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc109450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f31b400b0d0 tx=0x7f31b400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 -- 172.21.15.16:0/1206896744 shutdown_connections 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 --2- 172.21.15.16:0/1206896744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc109450 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 --2- 172.21.15.16:0/1206896744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc101000 0x7f31bc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 --2- 172.21.15.16:0/1206896744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31bc102200 0x7f31bc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 -- 172.21.15.16:0/1206896744 >> 172.21.15.16:0/1206896744 conn(0x7f31bc0fc5a0 msgr2=0x7f31bc0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 -- 172.21.15.16:0/1206896744 shutdown_connections 2024-05-08T23:19:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.581+0000 7f31c10e0700 1 -- 172.21.15.16:0/1206896744 wait complete. 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 Processor -- start 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 -- start start 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc101000 0x7f31bc074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31bc102200 0x7f31bc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc074070 con 0x7f31bc102ba0 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0741e0 con 0x7f31bc102200 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31c10e0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1a10b0 con 0x7f31bc101000 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31bb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31bb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57632/0 (socket says 172.21.15.16:57632) 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31bb59e700 1 -- 172.21.15.16:0/2900501395 learned_addr learned my addr 172.21.15.16:0/2900501395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.582+0000 7f31b259c700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31bc102200 0x7f31bc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bad9d700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc101000 0x7f31bc074cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bb59e700 1 -- 172.21.15.16:0/2900501395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc101000 msgr2=0x7f31bc074cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bb59e700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc101000 0x7f31bc074cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bb59e700 1 -- 172.21.15.16:0/2900501395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31bc102200 msgr2=0x7f31bc073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bb59e700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31bc102200 0x7f31bc073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bb59e700 1 -- 172.21.15.16:0/2900501395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4009d00 con 0x7f31bc102ba0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31bb59e700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc0738d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f31b4007b60 tx=0x7f31b40094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b400e040 con 0x7f31bc102ba0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31b4004e60 con 0x7f31bc102ba0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc1a12e0 con 0x7f31bc102ba0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4012a10 con 0x7f31bc102ba0 2024-05-08T23:19:22.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.583+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1a1770 con 0x7f31bc102ba0 2024-05-08T23:19:22.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.584+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc108490 con 0x7f31bc102ba0 2024-05-08T23:19:22.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.586+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f31b4005750 con 0x7f31bc102ba0 2024-05-08T23:19:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.586+0000 7f31b8d99700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f31a80a11b0 0x7f31a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.586+0000 7f31bad9d700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f31a80a11b0 0x7f31a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.586+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f31b40ce320 con 0x7f31bc102ba0 2024-05-08T23:19:22.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.587+0000 7f31bad9d700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f31a80a11b0 0x7f31a80a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f31bc102060 tx=0x7f31ac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:22.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.591+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f31b40976e0 con 0x7f31bc102ba0 2024-05-08T23:19:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.805+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f31bc04ec60 con 0x7f31bc102ba0 2024-05-08T23:19:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.806+0000 7f31b8d99700 1 -- 172.21.15.16:0/2900501395 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) v1 ==== 56+0+5583 (secure 0 0 0) 0x7f31b409ad00 con 0x7f31bc102ba0 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 83 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-05-08T23:11:54.944695+0000 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-05-08T23:18:43.790069+0000 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 27 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-05-08T23:19:22.806 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release reef 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 76 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 77 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/3035598518,v1:172.21.15.16:6803/3035598518] [v2:172.21.15.16:6804/3035598518,v1:172.21.15.16:6805/3035598518] exists,up c7c44106-3485-4f2e-b9ca-fdd6db13f996 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6800/2944272573,v1:172.21.15.155:6801/2944272573] [v2:172.21.15.155:6802/2944272573,v1:172.21.15.155:6803/2944272573] exists,up e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 17 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/3853442888,v1:172.21.15.189:6801/3853442888] [v2:172.21.15.189:6802/3853442888,v1:172.21.15.189:6803/3853442888] exists,up 1a5d9e16-e1c7-4880-900c-e886b3e9c00d 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 23 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/2397977008,v1:172.21.15.16:6811/2397977008] [v2:172.21.15.16:6812/2397977008,v1:172.21.15.16:6813/2397977008] exists,up 810b101a-3d2e-4761-be92-dc7bc786a11c 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 30 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6808/2036488025,v1:172.21.15.155:6809/2036488025] [v2:172.21.15.155:6810/2036488025,v1:172.21.15.155:6811/2036488025] exists,up d1578371-9303-4583-95e0-6c7812f396ad 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 35 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/925680266,v1:172.21.15.189:6809/925680266] [v2:172.21.15.189:6810/925680266,v1:172.21.15.189:6811/925680266] exists,up b6855d48-5018-4f4a-a7e6-d97ce06a38e8 2024-05-08T23:19:22.807 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 42 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/3858453894,v1:172.21.15.16:6819/3858453894] [v2:172.21.15.16:6820/3858453894,v1:172.21.15.16:6821/3858453894] exists,up 6f82436c-9ec7-4fb4-9c5d-5d815b86a408 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 46 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6816/283518572,v1:172.21.15.155:6817/283518572] [v2:172.21.15.155:6818/283518572,v1:172.21.15.155:6819/283518572] exists,up 96cf8304-3a9c-49d8-8789-543a1ddcc7d5 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 53 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/2123508255,v1:172.21.15.189:6817/2123508255] [v2:172.21.15.189:6818/2123508255,v1:172.21.15.189:6819/2123508255] exists,up 2a625c37-f744-4a80-ac8f-44990dbfd83d 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 60 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/3426310247,v1:172.21.15.16:6827/3426310247] [v2:172.21.15.16:6828/3426310247,v1:172.21.15.16:6829/3426310247] exists,up 1821a996-5473-46aa-8f43-49fadd38fbe0 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 65 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.155:6824/1164143288,v1:172.21.15.155:6825/1164143288] [v2:172.21.15.155:6826/1164143288,v1:172.21.15.155:6827/1164143288] exists,up 913decb0-6802-4e12-ace9-bdd7f341456e 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 72 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/2978765234,v1:172.21.15.189:6825/2978765234] [v2:172.21.15.189:6826/2978765234,v1:172.21.15.189:6827/2978765234] exists,up 72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.b [8,11] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.e [2,1] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1d [2,4,8,11] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.11 [4,8] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.13 [4,9] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.21 [5,9] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.36 [4,9] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.1b [9,8] 2024-05-08T23:19:22.808 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.1c [9,8] 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/4190536936 expires 2024-05-09T23:12:42.499707+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/30115990 expires 2024-05-09T23:12:42.499707+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/972303159 expires 2024-05-09T23:12:42.499707+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1569877251 expires 2024-05-09T23:12:42.499707+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1837768303 expires 2024-05-09T23:12:19.352750+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/3684081056 expires 2024-05-09T23:12:19.352750+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/4028430980 expires 2024-05-09T23:12:19.352750+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/4149284949 expires 2024-05-09T23:12:19.352750+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/3684081056 expires 2024-05-09T23:12:19.352750+0000 2024-05-08T23:19:22.809 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1569877251 expires 2024-05-09T23:12:42.499707+0000 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f31a80a11b0 msgr2=0x7f31a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f31a80a11b0 0x7f31a80a3670 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f31bc102060 tx=0x7f31ac009190 comp rx=0 tx=0).stop 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 msgr2=0x7f31bc0738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc0738d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f31b4007b60 tx=0x7f31b40094c0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 shutdown_connections 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc102ba0 0x7f31bc0738d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f31a80a11b0 0x7f31a80a3670 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31bc101000 0x7f31bc074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 --2- 172.21.15.16:0/2900501395 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31bc102200 0x7f31bc073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 >> 172.21.15.16:0/2900501395 conn(0x7f31bc0fc5a0 msgr2=0x7f31bc1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.808+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 shutdown_connections 2024-05-08T23:19:22.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:22.809+0000 7f31c10e0700 1 -- 172.21.15.16:0/2900501395 wait complete. 2024-05-08T23:19:23.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:22 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2900501395' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:19:23.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:22 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2900501395' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:19:23.198 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T23:19:23.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:23.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:22 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2900501395' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T23:19:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 -- 172.21.15.16:0/3314300778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06540ff6e0 msgr2=0x7f06540ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 --2- 172.21.15.16:0/3314300778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06540ff6e0 0x7f06540ffb00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0644009990 tx=0x7f0644009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 -- 172.21.15.16:0/3314300778 shutdown_connections 2024-05-08T23:19:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 --2- 172.21.15.16:0/3314300778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0654100a00 0x7f0654100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 --2- 172.21.15.16:0/3314300778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06540ff6e0 0x7f06540ffb00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 --2- 172.21.15.16:0/3314300778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0654100040 0x7f06541004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 -- 172.21.15.16:0/3314300778 >> 172.21.15.16:0/3314300778 conn(0x7f06540fb300 msgr2=0x7f06540fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 -- 172.21.15.16:0/3314300778 shutdown_connections 2024-05-08T23:19:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.360+0000 7f06594e0700 1 -- 172.21.15.16:0/3314300778 wait complete. 2024-05-08T23:19:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.361+0000 7f06594e0700 1 Processor -- start 2024-05-08T23:19:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 -- start start 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06540ff6e0 0x7f0654198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0654100040 0x7f06541993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 0x7f06541a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06541a09a0 con 0x7f0654100a00 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06541a0ae0 con 0x7f06540ff6e0 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06594e0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06541a0c70 con 0x7f0654100040 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 0x7f06541a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 0x7f06541a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57658/0 (socket says 172.21.15.16:57658) 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 -- 172.21.15.16:0/4032672485 learned_addr learned my addr 172.21.15.16:0/4032672485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f0652ffd700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06540ff6e0 0x7f0654198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06527fc700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0654100040 0x7f06541993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 -- 172.21.15.16:0/4032672485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0654100040 msgr2=0x7f06541993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0654100040 0x7f06541993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 -- 172.21.15.16:0/4032672485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06540ff6e0 msgr2=0x7f0654198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06540ff6e0 0x7f0654198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 -- 172.21.15.16:0/4032672485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440095f0 con 0x7f0654100a00 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.362+0000 7f06537fe700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 0x7f06541a03d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f064c00ada0 tx=0x7f064c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.363+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c00f070 con 0x7f0654100a00 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.363+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f064c007b30 con 0x7f0654100a00 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.363+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0053e0 con 0x7f0654100a00 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.363+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06541a0f50 con 0x7f0654100a00 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.363+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06541a1320 con 0x7f0654100a00 2024-05-08T23:19:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.364+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06540680d0 con 0x7f0654100a00 2024-05-08T23:19:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.365+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f064c00c980 con 0x7f0654100a00 2024-05-08T23:19:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.366+0000 7f063bfff700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f063c0a11b0 0x7f063c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.366+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f064c0cc8e0 con 0x7f0654100a00 2024-05-08T23:19:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.366+0000 7f0652ffd700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f063c0a11b0 0x7f063c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.366+0000 7f0652ffd700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f063c0a11b0 0x7f063c0a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0644003780 tx=0x7f0644005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.369+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f064c095ca0 con 0x7f0654100a00 2024-05-08T23:19:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.587+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f065410e700 con 0x7f0654100a00 2024-05-08T23:19:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.588+0000 7f063bfff700 1 -- 172.21.15.16:0/4032672485 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24470 (secure 0 0 0) 0x7f064c00cc30 con 0x7f0654100a00 2024-05-08T23:19:23.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:23.589 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:18:43.790069+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f063c0a11b0 msgr2=0x7f063c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f063c0a11b0 0x7f063c0a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0644003780 tx=0x7f0644005fd0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 msgr2=0x7f06541a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 0x7f06541a03d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f064c00ada0 tx=0x7f064c00c330 comp rx=0 tx=0).stop 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 shutdown_connections 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0654100a00 0x7f06541a03d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06540ff6e0 0x7f0654198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f063c0a11b0 0x7f063c0a3670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 --2- 172.21.15.16:0/4032672485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0654100040 0x7f06541993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 >> 172.21.15.16:0/4032672485 conn(0x7f06540fb300 msgr2=0x7f06540fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 shutdown_connections 2024-05-08T23:19:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.591+0000 7f06594e0700 1 -- 172.21.15.16:0/4032672485 wait complete. 2024-05-08T23:19:23.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:23.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:23.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 -- 172.21.15.16:0/1979533546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b80ffe60 msgr2=0x7f99b8100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 --2- 172.21.15.16:0/1979533546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b80ffe60 0x7f99b8100280 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009990 tx=0x7f99b0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 -- 172.21.15.16:0/1979533546 shutdown_connections 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 --2- 172.21.15.16:0/1979533546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b8101970 0x7f99b8108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 --2- 172.21.15.16:0/1979533546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b80ffe60 0x7f99b8100280 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 --2- 172.21.15.16:0/1979533546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b8100fb0 0x7f99b8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 -- 172.21.15.16:0/1979533546 >> 172.21.15.16:0/1979533546 conn(0x7f99b80fb3f0 msgr2=0x7f99b80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 -- 172.21.15.16:0/1979533546 shutdown_connections 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.748+0000 7f99bdf98700 1 -- 172.21.15.16:0/1979533546 wait complete. 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 Processor -- start 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 -- start start 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b80ffe60 0x7f99b8068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b8100fb0 0x7f99b819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 0x7f99b819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b819e900 con 0x7f99b8101970 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b819ea70 con 0x7f99b80ffe60 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.749+0000 7f99bdf98700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b819ec30 con 0x7f99b8100fb0 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 0x7f99b819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 0x7f99b819e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57668/0 (socket says 172.21.15.16:57668) 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 -- 172.21.15.16:0/2876474532 learned_addr learned my addr 172.21.15.16:0/2876474532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b6ffd700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b8100fb0 0x7f99b819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b77fe700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b80ffe60 0x7f99b8068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 -- 172.21.15.16:0/2876474532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b8100fb0 msgr2=0x7f99b819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b8100fb0 0x7f99b819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 -- 172.21.15.16:0/2876474532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b80ffe60 msgr2=0x7f99b8068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b80ffe60 0x7f99b8068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 -- 172.21.15.16:0/2876474532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b00095f0 con 0x7f99b8101970 2024-05-08T23:19:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b7fff700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 0x7f99b819e160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f99a8007ee0 tx=0x7f99a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8017070 con 0x7f99b8101970 2024-05-08T23:19:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99a800f040 con 0x7f99b8101970 2024-05-08T23:19:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b819ef10 con 0x7f99b8101970 2024-05-08T23:19:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8005570 con 0x7f99b8101970 2024-05-08T23:19:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.750+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b819f390 con 0x7f99b8101970 2024-05-08T23:19:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.751+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b8107320 con 0x7f99b8101970 2024-05-08T23:19:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.753+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f99a800db20 con 0x7f99b8101970 2024-05-08T23:19:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.754+0000 7f99b4ff9700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99a40a1230 0x7f99a40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.754+0000 7f99b77fe700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99a40a1230 0x7f99a40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.754+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f99a80cc4a0 con 0x7f99b8101970 2024-05-08T23:19:23.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.754+0000 7f99b77fe700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99a40a1230 0x7f99a40a36f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99b000c010 tx=0x7f99b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.757+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f99a80957d0 con 0x7f99b8101970 2024-05-08T23:19:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.975+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99b810b560 con 0x7f99b8101970 2024-05-08T23:19:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.975+0000 7f99b4ff9700 1 -- 172.21.15.16:0/2876474532 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99a8021020 con 0x7f99b8101970 2024-05-08T23:19:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.977+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99a40a1230 msgr2=0x7f99a40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.977+0000 7f99bdf98700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99a40a1230 0x7f99a40a36f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99b000c010 tx=0x7f99b0005fd0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.977+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 msgr2=0x7f99b819e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.977+0000 7f99bdf98700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 0x7f99b819e160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f99a8007ee0 tx=0x7f99a800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 shutdown_connections 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99b8101970 0x7f99b819e160 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b80ffe60 0x7f99b8068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99a40a1230 0x7f99a40a36f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 --2- 172.21.15.16:0/2876474532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b8100fb0 0x7f99b819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 >> 172.21.15.16:0/2876474532 conn(0x7f99b80fb3f0 msgr2=0x7f99b8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 shutdown_connections 2024-05-08T23:19:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:23.978+0000 7f99bdf98700 1 -- 172.21.15.16:0/2876474532 wait complete. 2024-05-08T23:19:23.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:24.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 -- 172.21.15.16:0/3632400614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5070100fb0 msgr2=0x7f5070101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 --2- 172.21.15.16:0/3632400614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5070100fb0 0x7f5070101430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5058009990 tx=0x7f5058009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 -- 172.21.15.16:0/3632400614 shutdown_connections 2024-05-08T23:19:24.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 --2- 172.21.15.16:0/3632400614 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070101970 0x7f5070108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 --2- 172.21.15.16:0/3632400614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50700ffe60 0x7f5070100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 --2- 172.21.15.16:0/3632400614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5070100fb0 0x7f5070101430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 -- 172.21.15.16:0/3632400614 >> 172.21.15.16:0/3632400614 conn(0x7f50700fb3f0 msgr2=0x7f50700fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 -- 172.21.15.16:0/3632400614 shutdown_connections 2024-05-08T23:19:24.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.133+0000 7f5075c07700 1 -- 172.21.15.16:0/3632400614 wait complete. 2024-05-08T23:19:24.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.134+0000 7f5075c07700 1 Processor -- start 2024-05-08T23:19:24.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.134+0000 7f5075c07700 1 -- start start 2024-05-08T23:19:24.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f5075c07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 0x7f5070195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f5075c07700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070100fb0 0x7f5070196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f5075c07700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5070101970 0x7f507019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f5075c07700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507019d830 con 0x7f50700ffe60 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f5075c07700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507019d9a0 con 0x7f5070100fb0 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f5075c07700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507019db60 con 0x7f5070101970 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 0x7f5070195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 0x7f5070195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57676/0 (socket says 172.21.15.16:57676) 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 -- 172.21.15.16:0/2085739152 learned_addr learned my addr 172.21.15.16:0/2085739152 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506effd700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070100fb0 0x7f5070196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506ffff700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5070101970 0x7f507019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 -- 172.21.15.16:0/2085739152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5070101970 msgr2=0x7f507019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5070101970 0x7f507019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 -- 172.21.15.16:0/2085739152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070100fb0 msgr2=0x7f5070196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070100fb0 0x7f5070196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 -- 172.21.15.16:0/2085739152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580095f0 con 0x7f50700ffe60 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506effd700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070100fb0 0x7f5070196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.135+0000 7f506f7fe700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 0x7f5070195d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5060007910 tx=0x7f5060007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.136+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060012070 con 0x7f50700ffe60 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.136+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f506000d810 con 0x7f50700ffe60 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.136+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5070068050 con 0x7f50700ffe60 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.136+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060005280 con 0x7f50700ffe60 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.136+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5070068560 con 0x7f50700ffe60 2024-05-08T23:19:24.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.137+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5070067cf0 con 0x7f50700ffe60 2024-05-08T23:19:24.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.138+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5060016070 con 0x7f50700ffe60 2024-05-08T23:19:24.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.139+0000 7f506cff9700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f505c0a11b0 0x7f505c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.139+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f50600ce8e0 con 0x7f50700ffe60 2024-05-08T23:19:24.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.139+0000 7f506effd700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f505c0a11b0 0x7f505c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.139+0000 7f506effd700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f505c0a11b0 0x7f505c0a3670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5058006010 tx=0x7f505800b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.142+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5060097bb0 con 0x7f50700ffe60 2024-05-08T23:19:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:23 smithi189 ceph-mon[46919]: pgmap v273: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:23 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4032672485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:23 smithi155 ceph-mon[47105]: pgmap v273: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:23 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4032672485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.363+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f507010b6f0 con 0x7f50700ffe60 2024-05-08T23:19:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.363+0000 7f506cff9700 1 -- 172.21.15.16:0/2085739152 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f506001c020 con 0x7f50700ffe60 2024-05-08T23:19:24.375 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:24.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:23 smithi016 ceph-mon[42541]: pgmap v273: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-08T23:19:24.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:23 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4032672485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f505c0a11b0 msgr2=0x7f505c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f505c0a11b0 0x7f505c0a3670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5058006010 tx=0x7f505800b5d0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 msgr2=0x7f5070195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 0x7f5070195d40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5060007910 tx=0x7f5060007cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 shutdown_connections 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f505c0a11b0 0x7f505c0a3670 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5070101970 0x7f507019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50700ffe60 0x7f5070195d40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 --2- 172.21.15.16:0/2085739152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5070100fb0 0x7f5070196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 >> 172.21.15.16:0/2085739152 conn(0x7f50700fb3f0 msgr2=0x7f5070104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 shutdown_connections 2024-05-08T23:19:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.366+0000 7f5075c07700 1 -- 172.21.15.16:0/2085739152 wait complete. 2024-05-08T23:19:24.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 -- 172.21.15.16:0/2002262584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28101970 msgr2=0x7f0d28108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/2002262584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28101970 0x7f0d28108290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0d2000b0a0 tx=0x7f0d2000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 -- 172.21.15.16:0/2002262584 shutdown_connections 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/2002262584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28101970 0x7f0d28108290 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/2002262584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d280ffe60 0x7f0d28100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/2002262584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d28100fb0 0x7f0d28101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.542+0000 7f0d2fb09700 1 -- 172.21.15.16:0/2002262584 >> 172.21.15.16:0/2002262584 conn(0x7f0d280fb3f0 msgr2=0x7f0d280fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.543+0000 7f0d2fb09700 1 -- 172.21.15.16:0/2002262584 shutdown_connections 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.543+0000 7f0d2fb09700 1 -- 172.21.15.16:0/2002262584 wait complete. 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.543+0000 7f0d2fb09700 1 Processor -- start 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.543+0000 7f0d2fb09700 1 -- start start 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2fb09700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d280ffe60 0x7f0d28068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2fb09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 0x7f0d2819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2fb09700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d28101970 0x7f0d2819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2fb09700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2819ea80 con 0x7f0d28100fb0 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2fb09700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2819ebf0 con 0x7f0d280ffe60 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2fb09700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2819edb0 con 0x7f0d28101970 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 0x7f0d2819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 0x7f0d2819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57688/0 (socket says 172.21.15.16:57688) 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 -- 172.21.15.16:0/1268564211 learned_addr learned my addr 172.21.15.16:0/1268564211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d8a5700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d280ffe60 0x7f0d28068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2e0a6700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d28101970 0x7f0d2819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 -- 172.21.15.16:0/1268564211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d28101970 msgr2=0x7f0d2819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d28101970 0x7f0d2819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 -- 172.21.15.16:0/1268564211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d280ffe60 msgr2=0x7f0d28068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d280ffe60 0x7f0d28068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 -- 172.21.15.16:0/1268564211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20009d00 con 0x7f0d28100fb0 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.544+0000 7f0d2d0a4700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 0x7f0d2819bc70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800d950 tx=0x7f0d1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.545+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800f890 con 0x7f0d28100fb0 2024-05-08T23:19:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.545+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d1800fe80 con 0x7f0d28100fb0 2024-05-08T23:19:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.545+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2819efd0 con 0x7f0d28100fb0 2024-05-08T23:19:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.545+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800e3b0 con 0x7f0d28100fb0 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.545+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2819f500 con 0x7f0d28100fb0 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.546+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d281072a0 con 0x7f0d28100fb0 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.547+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0d18010460 con 0x7f0d28100fb0 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.547+0000 7f0d1effd700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0d140a1230 0x7f0d140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.548+0000 7f0d2d8a5700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0d140a1230 0x7f0d140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.548+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0d180ccaa0 con 0x7f0d28100fb0 2024-05-08T23:19:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.548+0000 7f0d2d8a5700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0d140a1230 0x7f0d140a36f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009c60 tx=0x7f0d24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.552+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0d18095dd0 con 0x7f0d28100fb0 2024-05-08T23:19:24.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.772+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d2810b560 con 0x7f0d28100fb0 2024-05-08T23:19:24.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.773+0000 7f0d1effd700 1 -- 172.21.15.16:0/1268564211 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d180160a0 con 0x7f0d28100fb0 2024-05-08T23:19:24.773 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0d140a1230 msgr2=0x7f0d140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0d140a1230 0x7f0d140a36f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009c60 tx=0x7f0d24009260 comp rx=0 tx=0).stop 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 msgr2=0x7f0d2819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 0x7f0d2819bc70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800d950 tx=0x7f0d1800dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 shutdown_connections 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f0d140a1230 0x7f0d140a36f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d28101970 0x7f0d2819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d280ffe60 0x7f0d28068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 --2- 172.21.15.16:0/1268564211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d28100fb0 0x7f0d2819bc70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 >> 172.21.15.16:0/1268564211 conn(0x7f0d280fb3f0 msgr2=0x7f0d281043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 shutdown_connections 2024-05-08T23:19:24.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.775+0000 7f0d2fb09700 1 -- 172.21.15.16:0/1268564211 wait complete. 2024-05-08T23:19:24.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 -- 172.21.15.16:0/1871107469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 msgr2=0x7f8d54103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1871107469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54103de0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009990 tx=0x7f8d3c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 -- 172.21.15.16:0/1871107469 shutdown_connections 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1871107469 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 0x7f8d541090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1871107469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54103de0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1871107469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d54104320 0x7f8d54106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 -- 172.21.15.16:0/1871107469 >> 172.21.15.16:0/1871107469 conn(0x7f8d540fb400 msgr2=0x7f8d540fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 -- 172.21.15.16:0/1871107469 shutdown_connections 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 -- 172.21.15.16:0/1871107469 wait complete. 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.934+0000 7f8d59f78700 1 Processor -- start 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 -- start start 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d54104320 0x7f8d541973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 0x7f8d5419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419e9b0 con 0x7f8d541019f0 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419eaf0 con 0x7f8d54106c50 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d59f78700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5419ec80 con 0x7f8d54104320 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57696/0 (socket says 172.21.15.16:57696) 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 -- 172.21.15.16:0/1425112245 learned_addr learned my addr 172.21.15.16:0/1425112245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d53fff700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 0x7f8d5419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d52ffd700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d54104320 0x7f8d541973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 -- 172.21.15.16:0/1425112245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d54104320 msgr2=0x7f8d541973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d54104320 0x7f8d541973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 -- 172.21.15.16:0/1425112245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 msgr2=0x7f8d5419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 0x7f8d5419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d537fe700 1 -- 172.21.15.16:0/1425112245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0095f0 con 0x7f8d541019f0 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.935+0000 7f8d53fff700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 0x7f8d5419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.936+0000 7f8d537fe700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54196e80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0098e0 tx=0x7f8d3c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.936+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c01d070 con 0x7f8d541019f0 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.936+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d3c00bd90 con 0x7f8d541019f0 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.936+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5419ef00 con 0x7f8d541019f0 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.936+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c022dc0 con 0x7f8d541019f0 2024-05-08T23:19:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.936+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5419f2c0 con 0x7f8d541019f0 2024-05-08T23:19:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.937+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540fcf80 con 0x7f8d541019f0 2024-05-08T23:19:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.938+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f8d3c003ff0 con 0x7f8d541019f0 2024-05-08T23:19:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.939+0000 7f8d50ff9700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8d400a1230 0x7f8d400a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.939+0000 7f8d52ffd700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8d400a1230 0x7f8d400a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.939+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8d3c0ce270 con 0x7f8d541019f0 2024-05-08T23:19:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.939+0000 7f8d52ffd700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8d400a1230 0x7f8d400a36f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009af0 tx=0x7f8d440094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:24.943+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8d3c0976f0 con 0x7f8d541019f0 2024-05-08T23:19:25.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:24 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2876474532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:24 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2085739152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:24 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1268564211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.163+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8d54068140 con 0x7f8d541019f0 2024-05-08T23:19:25.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.164+0000 7f8d50ff9700 1 -- 172.21.15.16:0/1425112245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8d3c09ad10 con 0x7f8d541019f0 2024-05-08T23:19:25.164 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8d400a1230 msgr2=0x7f8d400a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8d400a1230 0x7f8d400a36f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009af0 tx=0x7f8d440094f0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 msgr2=0x7f8d54196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54196e80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0098e0 tx=0x7f8d3c003b00 comp rx=0 tx=0).stop 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 shutdown_connections 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d54106c50 0x7f8d5419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d541019f0 0x7f8d54196e80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f8d400a1230 0x7f8d400a36f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 --2- 172.21.15.16:0/1425112245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d54104320 0x7f8d541973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 >> 172.21.15.16:0/1425112245 conn(0x7f8d540fb400 msgr2=0x7f8d541039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:25.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 shutdown_connections 2024-05-08T23:19:25.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.166+0000 7f8d59f78700 1 -- 172.21.15.16:0/1425112245 wait complete. 2024-05-08T23:19:25.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:24 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2876474532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:24 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2085739152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:24 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1268564211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:24 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2876474532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:24 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2085739152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:24 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1268564211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 -- 172.21.15.16:0/3865362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9358100f50 msgr2=0x7f93581013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 --2- 172.21.15.16:0/3865362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9358100f50 0x7f93581013b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f93400099c0 tx=0x7f9340009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 -- 172.21.15.16:0/3865362915 shutdown_connections 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 --2- 172.21.15.16:0/3865362915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93581018f0 0x7f93581081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 --2- 172.21.15.16:0/3865362915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93580ffd60 0x7f9358100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 --2- 172.21.15.16:0/3865362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9358100f50 0x7f93581013b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 -- 172.21.15.16:0/3865362915 >> 172.21.15.16:0/3865362915 conn(0x7f93580fb2f0 msgr2=0x7f93580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 -- 172.21.15.16:0/3865362915 shutdown_connections 2024-05-08T23:19:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.318+0000 7f935cb69700 1 -- 172.21.15.16:0/3865362915 wait complete. 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 Processor -- start 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 -- start start 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 0x7f9358198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358100f50 0x7f9358199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93581018f0 0x7f93581a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581a0a40 con 0x7f93581018f0 2024-05-08T23:19:25.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581a0b80 con 0x7f9358100f50 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935cb69700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93581a0d10 con 0x7f93580ffd60 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93581018f0 0x7f93581a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935659c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 0x7f9358198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f9356d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93581018f0 0x7f93581a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57712/0 (socket says 172.21.15.16:57712) 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f935659c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 0x7f9358198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:59220/0 (socket says 172.21.15.16:59220) 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f9356d9d700 1 -- 172.21.15.16:0/3173227692 learned_addr learned my addr 172.21.15.16:0/3173227692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.319+0000 7f934ffff700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358100f50 0x7f9358199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935659c700 1 -- 172.21.15.16:0/3173227692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358100f50 msgr2=0x7f9358199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935659c700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358100f50 0x7f9358199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935659c700 1 -- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93581018f0 msgr2=0x7f93581a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935659c700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93581018f0 0x7f93581a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935659c700 1 -- 172.21.15.16:0/3173227692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400095f0 con 0x7f93580ffd60 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935659c700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 0x7f9358198f10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9c0 tx=0x7f934800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348004220 con 0x7f93580ffd60 2024-05-08T23:19:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93581a0ff0 con 0x7f93580ffd60 2024-05-08T23:19:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9348004380 con 0x7f93580ffd60 2024-05-08T23:19:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348003b40 con 0x7f93580ffd60 2024-05-08T23:19:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.320+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93581a1420 con 0x7f93580ffd60 2024-05-08T23:19:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.321+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358067c10 con 0x7f93580ffd60 2024-05-08T23:19:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.322+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9348003ca0 con 0x7f93580ffd60 2024-05-08T23:19:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.323+0000 7f934f7fe700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f93440a11b0 0x7f93440a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.323+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9348014070 con 0x7f93580ffd60 2024-05-08T23:19:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.323+0000 7f934ffff700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f93440a11b0 0x7f93440a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.324+0000 7f934ffff700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f93440a11b0 0x7f93440a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f9340003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:25.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.327+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9348096a80 con 0x7f93580ffd60 2024-05-08T23:19:25.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.594+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93581a1670 con 0x7f93580ffd60 2024-05-08T23:19:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.595+0000 7f934f7fe700 1 -- 172.21.15.16:0/3173227692 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f934809a0a0 con 0x7f93580ffd60 2024-05-08T23:19:25.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:25.595 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:18:12.692850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f93440a11b0 msgr2=0x7f93440a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f93440a11b0 0x7f93440a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f9340003680 comp rx=0 tx=0).stop 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 msgr2=0x7f9358198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 0x7f9358198f10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9c0 tx=0x7f934800ecd0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 shutdown_connections 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93581018f0 0x7f93581a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f93440a11b0 0x7f93440a3670 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93580ffd60 0x7f9358198f10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 --2- 172.21.15.16:0/3173227692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9358100f50 0x7f9358199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 >> 172.21.15.16:0/3173227692 conn(0x7f93580fb2f0 msgr2=0x7f9358068020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 shutdown_connections 2024-05-08T23:19:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.598+0000 7f935cb69700 1 -- 172.21.15.16:0/3173227692 wait complete. 2024-05-08T23:19:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:19:25.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 -- 172.21.15.16:0/1737982046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 msgr2=0x7f4e0c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1737982046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c102440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009990 tx=0x7f4e00009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 -- 172.21.15.16:0/1737982046 shutdown_connections 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1737982046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 0x7f4e0c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1737982046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 0x7f4e0c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1737982046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c102440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 -- 172.21.15.16:0/1737982046 >> 172.21.15.16:0/1737982046 conn(0x7f4e0c0fc400 msgr2=0x7f4e0c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 -- 172.21.15.16:0/1737982046 shutdown_connections 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.750+0000 7f4e127d5700 1 -- 172.21.15.16:0/1737982046 wait complete. 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 Processor -- start 2024-05-08T23:19:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 -- start start 2024-05-08T23:19:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 0x7f4e0c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 0x7f4e0c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c074070 con 0x7f4e0c101fc0 2024-05-08T23:19:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0741e0 con 0x7f4e0c102980 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e127d5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c1a0d60 con 0x7f4e0c100e70 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57738/0 (socket says 172.21.15.16:57738) 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e0b7fe700 1 -- 172.21.15.16:0/1080040889 learned_addr learned my addr 172.21.15.16:0/1080040889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e0bfff700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 0x7f4e0c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.751+0000 7f4e10d72700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 0x7f4e0c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0b7fe700 1 -- 172.21.15.16:0/1080040889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 msgr2=0x7f4e0c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0b7fe700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 0x7f4e0c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0b7fe700 1 -- 172.21.15.16:0/1080040889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 msgr2=0x7f4e0c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0b7fe700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 0x7f4e0c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0b7fe700 1 -- 172.21.15.16:0/1080040889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e000095f0 con 0x7f4e0c101fc0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0bfff700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 0x7f4e0c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e10d72700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 0x7f4e0c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e0b7fe700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c073390 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b5c0 tx=0x7f4e00003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0001d070 con 0x7f4e0c101fc0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e0000bd10 con 0x7f4e0c101fc0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c1a0f90 con 0x7f4e0c101fc0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00022dc0 con 0x7f4e0c101fc0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.752+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c1a1300 con 0x7f4e0c101fc0 2024-05-08T23:19:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.753+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c108320 con 0x7f4e0c101fc0 2024-05-08T23:19:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.754+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4e00003ff0 con 0x7f4e0c101fc0 2024-05-08T23:19:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.755+0000 7f4e097fa700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4df40a11b0 0x7f4df40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.755+0000 7f4e0bfff700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4df40a11b0 0x7f4df40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.755+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f4e000cde90 con 0x7f4e0c101fc0 2024-05-08T23:19:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.756+0000 7f4e0bfff700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4df40a11b0 0x7f4df40a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc005950 tx=0x7f4dfc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.759+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e000971f0 con 0x7f4e0c101fc0 2024-05-08T23:19:25.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.980+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e0c10c520 con 0x7f4e0c101fc0 2024-05-08T23:19:25.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.982+0000 7f4e097fa700 1 -- 172.21.15.16:0/1080040889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24470 (secure 0 0 0) 0x7f4e0009a810 con 0x7f4e0c101fc0 2024-05-08T23:19:25.982 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:25.982 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:18:43.790069+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4df40a11b0 msgr2=0x7f4df40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4df40a11b0 0x7f4df40a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc005950 tx=0x7f4dfc0058e0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 msgr2=0x7f4e0c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c073390 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b5c0 tx=0x7f4e00003b00 comp rx=0 tx=0).stop 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 shutdown_connections 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c102980 0x7f4e0c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f4df40a11b0 0x7f4df40a3670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e0c100e70 0x7f4e0c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 --2- 172.21.15.16:0/1080040889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0c101fc0 0x7f4e0c073390 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 >> 172.21.15.16:0/1080040889 conn(0x7f4e0c0fc400 msgr2=0x7f4e0c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 shutdown_connections 2024-05-08T23:19:25.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:25.984+0000 7f4e127d5700 1 -- 172.21.15.16:0/1080040889 wait complete. 2024-05-08T23:19:25.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:25.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 -- 172.21.15.16:0/7396154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 msgr2=0x7efc18100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 --2- 172.21.15.16:0/7396154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18100280 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efc08009990 tx=0x7efc08009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 -- 172.21.15.16:0/7396154 shutdown_connections 2024-05-08T23:19:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 --2- 172.21.15.16:0/7396154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 0x7efc18108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 --2- 172.21.15.16:0/7396154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18100280 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 --2- 172.21.15.16:0/7396154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc18100fb0 0x7efc18101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 -- 172.21.15.16:0/7396154 >> 172.21.15.16:0/7396154 conn(0x7efc180fb3f0 msgr2=0x7efc180fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 -- 172.21.15.16:0/7396154 shutdown_connections 2024-05-08T23:19:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.140+0000 7efc1c88e700 1 -- 172.21.15.16:0/7396154 wait complete. 2024-05-08T23:19:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 Processor -- start 2024-05-08T23:19:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 -- start start 2024-05-08T23:19:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18103d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc18100fb0 0x7efc18102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 0x7efc18102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc18103100 con 0x7efc180ffe60 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc18103270 con 0x7efc18101970 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.141+0000 7efc1c88e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc181a0ed0 con 0x7efc18100fb0 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18103d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18103d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57764/0 (socket says 172.21.15.16:57764) 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 -- 172.21.15.16:0/1128988386 learned_addr learned my addr 172.21.15.16:0/1128988386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc0dd9b700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc18100fb0 0x7efc18102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc16d9d700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 0x7efc18102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 -- 172.21.15.16:0/1128988386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc18100fb0 msgr2=0x7efc18102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc18100fb0 0x7efc18102420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 -- 172.21.15.16:0/1128988386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 msgr2=0x7efc18102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 0x7efc18102960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 -- 172.21.15.16:0/1128988386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc080095f0 con 0x7efc180ffe60 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc16d9d700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 0x7efc18102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1659c700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18103d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc080059d0 tx=0x7efc08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0801d070 con 0x7efc180ffe60 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc181a1100 con 0x7efc180ffe60 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc0800bdd0 con 0x7efc180ffe60 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08022da0 con 0x7efc180ffe60 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.142+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc181a14c0 con 0x7efc180ffe60 2024-05-08T23:19:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.143+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc18067cf0 con 0x7efc180ffe60 2024-05-08T23:19:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.145+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7efc08022570 con 0x7efc180ffe60 2024-05-08T23:19:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.145+0000 7efc0ffff700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efc040a11b0 0x7efc040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.145+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efc080ce440 con 0x7efc180ffe60 2024-05-08T23:19:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.145+0000 7efc0dd9b700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efc040a11b0 0x7efc040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.146+0000 7efc0dd9b700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efc040a11b0 0x7efc040a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efc00009c60 tx=0x7efc00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.150+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc08097800 con 0x7efc180ffe60 2024-05-08T23:19:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:25 smithi189 ceph-mon[46919]: pgmap v274: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1425112245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3173227692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:25 smithi155 ceph-mon[47105]: pgmap v274: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1425112245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3173227692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:26.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:25 smithi016 ceph-mon[42541]: pgmap v274: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-08T23:19:26.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1425112245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:26.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3173227692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.366+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc181a17f0 con 0x7efc180ffe60 2024-05-08T23:19:26.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.366+0000 7efc0ffff700 1 -- 172.21.15.16:0/1128988386 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc08027020 con 0x7efc180ffe60 2024-05-08T23:19:26.366 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efc040a11b0 msgr2=0x7efc040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efc040a11b0 0x7efc040a3670 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efc00009c60 tx=0x7efc00009260 comp rx=0 tx=0).stop 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 msgr2=0x7efc18103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18103d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efc080059d0 tx=0x7efc08004730 comp rx=0 tx=0).stop 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 shutdown_connections 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc18101970 0x7efc18102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc180ffe60 0x7efc18103d90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efc040a11b0 0x7efc040a3670 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 --2- 172.21.15.16:0/1128988386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc18100fb0 0x7efc18102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 >> 172.21.15.16:0/1128988386 conn(0x7efc180fb3f0 msgr2=0x7efc180680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 shutdown_connections 2024-05-08T23:19:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.369+0000 7efc1c88e700 1 -- 172.21.15.16:0/1128988386 wait complete. 2024-05-08T23:19:26.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 -- 172.21.15.16:0/2907122677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 msgr2=0x7f5f941090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 --2- 172.21.15.16:0/2907122677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f941090b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00b0a0 tx=0x7f5f8c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 -- 172.21.15.16:0/2907122677 shutdown_connections 2024-05-08T23:19:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 --2- 172.21.15.16:0/2907122677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f941090b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 --2- 172.21.15.16:0/2907122677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f941019e0 0x7f5f94103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 --2- 172.21.15.16:0/2907122677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f94104310 0x7f5f94106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 -- 172.21.15.16:0/2907122677 >> 172.21.15.16:0/2907122677 conn(0x7f5f940fb3f0 msgr2=0x7f5f940fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 -- 172.21.15.16:0/2907122677 shutdown_connections 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 -- 172.21.15.16:0/2907122677 wait complete. 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.524+0000 7f5f993bf700 1 Processor -- start 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 -- start start 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f941019e0 0x7f5f94196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f94104310 0x7f5f941973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f9419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9419e9c0 con 0x7f5f94106c40 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9419eb00 con 0x7f5f94104310 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f993bf700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9419ec90 con 0x7f5f941019e0 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f9419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f9419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41472/0 (socket says 172.21.15.16:41472) 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 -- 172.21.15.16:0/4134737877 learned_addr learned my addr 172.21.15.16:0/4134737877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f92ffd700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f941019e0 0x7f5f94196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f927fc700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f94104310 0x7f5f941973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 -- 172.21.15.16:0/4134737877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f941019e0 msgr2=0x7f5f94196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f941019e0 0x7f5f94196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 -- 172.21.15.16:0/4134737877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f94104310 msgr2=0x7f5f941973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.525+0000 7f5f937fe700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f94104310 0x7f5f941973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f937fe700 1 -- 172.21.15.16:0/4134737877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f88009520 con 0x7f5f94106c40 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f92ffd700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f941019e0 0x7f5f94196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f927fc700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f94104310 0x7f5f941973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f937fe700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f9419e3f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009820 tx=0x7f5f8c00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00e040 con 0x7f5f94106c40 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f8c004e60 con 0x7f5f94106c40 2024-05-08T23:19:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c009d00 con 0x7f5f94106c40 2024-05-08T23:19:26.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0052a0 con 0x7f5f94106c40 2024-05-08T23:19:26.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.526+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9419f1b0 con 0x7f5f94106c40 2024-05-08T23:19:26.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.527+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f940fcf70 con 0x7f5f94106c40 2024-05-08T23:19:26.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.528+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5f8c01d990 con 0x7f5f94106c40 2024-05-08T23:19:26.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.529+0000 7f5f7bfff700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5f7c0a11b0 0x7f5f7c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.529+0000 7f5f92ffd700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5f7c0a11b0 0x7f5f7c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.529+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f5f8c0ce460 con 0x7f5f94106c40 2024-05-08T23:19:26.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.530+0000 7f5f92ffd700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5f7c0a11b0 0x7f5f7c0a3670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5f84005fd0 tx=0x7f5f84005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:26.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.534+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5f8c0977c0 con 0x7f5f94106c40 2024-05-08T23:19:26.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.751+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f940fc950 con 0x7f5f94106c40 2024-05-08T23:19:26.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.752+0000 7f5f7bfff700 1 -- 172.21.15.16:0/4134737877 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f8c09ade0 con 0x7f5f94106c40 2024-05-08T23:19:26.752 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5f7c0a11b0 msgr2=0x7f5f7c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5f7c0a11b0 0x7f5f7c0a3670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5f84005fd0 tx=0x7f5f84005f60 comp rx=0 tx=0).stop 2024-05-08T23:19:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 msgr2=0x7f5f9419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f9419e3f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009820 tx=0x7f5f8c00bb60 comp rx=0 tx=0).stop 2024-05-08T23:19:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 shutdown_connections 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f94106c40 0x7f5f9419e3f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f5f7c0a11b0 0x7f5f7c0a3670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f941019e0 0x7f5f94196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 --2- 172.21.15.16:0/4134737877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f94104310 0x7f5f941973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 >> 172.21.15.16:0/4134737877 conn(0x7f5f940fb3f0 msgr2=0x7f5f941022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 shutdown_connections 2024-05-08T23:19:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.754+0000 7f5f993bf700 1 -- 172.21.15.16:0/4134737877 wait complete. 2024-05-08T23:19:26.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 -- 172.21.15.16:0/3084350574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0102b90 msgr2=0x7f7af0109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 --2- 172.21.15.16:0/3084350574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0102b90 0x7f7af0109440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800b0a0 tx=0x7f7ae800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 -- 172.21.15.16:0/3084350574 shutdown_connections 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 --2- 172.21.15.16:0/3084350574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af0102b90 0x7f7af0109440 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 --2- 172.21.15.16:0/3084350574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af00ff220 0x7f7af00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 --2- 172.21.15.16:0/3084350574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7af01021f0 0x7f7af0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.910+0000 7f7af771e700 1 -- 172.21.15.16:0/3084350574 >> 172.21.15.16:0/3084350574 conn(0x7f7af00fab50 msgr2=0x7f7af00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.911+0000 7f7af771e700 1 -- 172.21.15.16:0/3084350574 shutdown_connections 2024-05-08T23:19:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.911+0000 7f7af771e700 1 -- 172.21.15.16:0/3084350574 wait complete. 2024-05-08T23:19:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.911+0000 7f7af771e700 1 Processor -- start 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 -- start start 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7af00ff220 0x7f7af0101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 0x7f7af01001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af0102b90 0x7f7af01006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0100e80 con 0x7f7af01021f0 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0100ff0 con 0x7f7af00ff220 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af771e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a0f80 con 0x7f7af0102b90 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 0x7f7af01001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 0x7f7af01001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41492/0 (socket says 172.21.15.16:41492) 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 -- 172.21.15.16:0/1446657635 learned_addr learned my addr 172.21.15.16:0/1446657635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af54ba700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7af00ff220 0x7f7af0101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af5cbb700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af0102b90 0x7f7af01006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 -- 172.21.15.16:0/1446657635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af0102b90 msgr2=0x7f7af01006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af0102b90 0x7f7af01006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 -- 172.21.15.16:0/1446657635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7af00ff220 msgr2=0x7f7af0101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7af00ff220 0x7f7af0101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.912+0000 7f7af4cb9700 1 -- 172.21.15.16:0/1446657635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8009d00 con 0x7f7af01021f0 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.913+0000 7f7af4cb9700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 0x7f7af01001a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e9a0 tx=0x7f7ae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.914+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0009790 con 0x7f7af01021f0 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.914+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae0010430 con 0x7f7af01021f0 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.914+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af01a1210 con 0x7f7af01021f0 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.914+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0018680 con 0x7f7af01021f0 2024-05-08T23:19:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.914+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01a1770 con 0x7f7af01021f0 2024-05-08T23:19:26.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.915+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af0108520 con 0x7f7af01021f0 2024-05-08T23:19:26.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.916+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7ae00105a0 con 0x7f7af01021f0 2024-05-08T23:19:26.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.916+0000 7f7ae67fc700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7adc0a1230 0x7f7adc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:26.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.917+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7ae0014070 con 0x7f7af01021f0 2024-05-08T23:19:26.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.917+0000 7f7af54ba700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7adc0a1230 0x7f7adc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:26.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.917+0000 7f7af54ba700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7adc0a1230 0x7f7adc0a36f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7af0067df0 tx=0x7f7aec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:26.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:26.920+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ae00964f0 con 0x7f7af01021f0 2024-05-08T23:19:27.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:26 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1080040889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:27.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:26 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1128988386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:27.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:26 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4134737877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:27.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.136+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7af010c860 con 0x7f7af01021f0 2024-05-08T23:19:27.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.136+0000 7f7ae67fc700 1 -- 172.21.15.16:0/1446657635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ae00260a0 con 0x7f7af01021f0 2024-05-08T23:19:27.136 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7adc0a1230 msgr2=0x7f7adc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7adc0a1230 0x7f7adc0a36f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7af0067df0 tx=0x7f7aec009260 comp rx=0 tx=0).stop 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 msgr2=0x7f7af01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 0x7f7af01001a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e9a0 tx=0x7f7ae000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 shutdown_connections 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f7adc0a1230 0x7f7adc0a36f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af0102b90 0x7f7af01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7af00ff220 0x7f7af0101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 --2- 172.21.15.16:0/1446657635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7af01021f0 0x7f7af01001a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 >> 172.21.15.16:0/1446657635 conn(0x7f7af00fab50 msgr2=0x7f7af0105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 shutdown_connections 2024-05-08T23:19:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.139+0000 7f7af771e700 1 -- 172.21.15.16:0/1446657635 wait complete. 2024-05-08T23:19:27.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:27.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1080040889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:27.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1128988386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:27.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4134737877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:27.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1080040889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:27.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1128988386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:27.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4134737877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3184858878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc1001b0 msgr2=0x7fa3dc100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3184858878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc1001b0 0x7fa3dc100630 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009990 tx=0x7fa3c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3184858878 shutdown_connections 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3184858878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3dc100b70 0x7fa3dc100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3184858878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc0ff850 0x7fa3dc0ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3184858878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc1001b0 0x7fa3dc100630 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3184858878 >> 172.21.15.16:0/3184858878 conn(0x7fa3dc0fb3f0 msgr2=0x7fa3dc0fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3184858878 shutdown_connections 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3184858878 wait complete. 2024-05-08T23:19:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.294+0000 7fa3e1d8e700 1 Processor -- start 2024-05-08T23:19:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 -- start start 2024-05-08T23:19:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3dc0ff850 0x7fa3dc198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc1001b0 0x7fa3dc199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 0x7fa3dc1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1a0a60 con 0x7fa3dc100b70 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1a0ba0 con 0x7fa3dc1001b0 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3e1d8e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc1a0d30 con 0x7fa3dc0ff850 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 0x7fa3dc1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 0x7fa3dc1a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41514/0 (socket says 172.21.15.16:41514) 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 -- 172.21.15.16:0/3547661275 learned_addr learned my addr 172.21.15.16:0/3547661275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3daffd700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc1001b0 0x7fa3dc199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3db7fe700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3dc0ff850 0x7fa3dc198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 -- 172.21.15.16:0/3547661275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3dc0ff850 msgr2=0x7fa3dc198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3dc0ff850 0x7fa3dc198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 -- 172.21.15.16:0/3547661275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc1001b0 msgr2=0x7fa3dc199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc1001b0 0x7fa3dc199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.295+0000 7fa3dbfff700 1 -- 172.21.15.16:0/3547661275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40095f0 con 0x7fa3dc100b70 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.296+0000 7fa3dbfff700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 0x7fa3dc1a0490 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400ef30 tx=0x7fa3d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.296+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4018070 con 0x7fa3dc100b70 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.296+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3d400f040 con 0x7fa3dc100b70 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.296+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc1a0fb0 con 0x7fa3dc100b70 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.296+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400cc70 con 0x7fa3dc100b70 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.296+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc1a1430 con 0x7fa3dc100b70 2024-05-08T23:19:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.297+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc10a880 con 0x7fa3dc100b70 2024-05-08T23:19:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.298+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa3d4003a00 con 0x7fa3dc100b70 2024-05-08T23:19:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.299+0000 7fa3d8ff9700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa3c80a11b0 0x7fa3c80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.299+0000 7fa3db7fe700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa3c80a11b0 0x7fa3c80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.299+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa3d40cce80 con 0x7fa3dc100b70 2024-05-08T23:19:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.300+0000 7fa3db7fe700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa3c80a11b0 0x7fa3c80a3670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005ea0 tx=0x7fa3cc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.303+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3d4096240 con 0x7fa3dc100b70 2024-05-08T23:19:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.521+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa3dc10e820 con 0x7fa3dc100b70 2024-05-08T23:19:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.522+0000 7fa3d8ff9700 1 -- 172.21.15.16:0/3547661275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa3d4003cb0 con 0x7fa3dc100b70 2024-05-08T23:19:27.522 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa3c80a11b0 msgr2=0x7fa3c80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa3c80a11b0 0x7fa3c80a3670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc005ea0 tx=0x7fa3cc005e30 comp rx=0 tx=0).stop 2024-05-08T23:19:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 msgr2=0x7fa3dc1a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 0x7fa3dc1a0490 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400ef30 tx=0x7fa3d400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 shutdown_connections 2024-05-08T23:19:27.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc100b70 0x7fa3dc1a0490 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa3c80a11b0 0x7fa3c80a3670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3dc0ff850 0x7fa3dc198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 --2- 172.21.15.16:0/3547661275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc1001b0 0x7fa3dc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 >> 172.21.15.16:0/3547661275 conn(0x7fa3dc0fb3f0 msgr2=0x7fa3dc0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 shutdown_connections 2024-05-08T23:19:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.524+0000 7fa3e1d8e700 1 -- 172.21.15.16:0/3547661275 wait complete. 2024-05-08T23:19:27.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 -- 172.21.15.16:0/4098737674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4094100f50 msgr2=0x7f40941013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 --2- 172.21.15.16:0/4098737674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4094100f50 0x7f40941013b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40840099c0 tx=0x7f4084009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 -- 172.21.15.16:0/4098737674 shutdown_connections 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 --2- 172.21.15.16:0/4098737674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40941018f0 0x7f40941081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 --2- 172.21.15.16:0/4098737674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40940ffd60 0x7f4094100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 --2- 172.21.15.16:0/4098737674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4094100f50 0x7f40941013b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 -- 172.21.15.16:0/4098737674 >> 172.21.15.16:0/4098737674 conn(0x7f40940fb2f0 msgr2=0x7f40940fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 -- 172.21.15.16:0/4098737674 shutdown_connections 2024-05-08T23:19:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 -- 172.21.15.16:0/4098737674 wait complete. 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.676+0000 7f409bbc2700 1 Processor -- start 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 -- start start 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40940ffd60 0x7f4094198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4094100f50 0x7f4094199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 0x7f40941a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a0a20 con 0x7f40941018f0 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a0b60 con 0x7f40940ffd60 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409bbc2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a0cf0 con 0x7f4094100f50 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 0x7f40941a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 0x7f40941a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41526/0 (socket says 172.21.15.16:41526) 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409915d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4094100f50 0x7f4094199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 -- 172.21.15.16:0/130896443 learned_addr learned my addr 172.21.15.16:0/130896443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409995e700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40940ffd60 0x7f4094198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 -- 172.21.15.16:0/130896443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4094100f50 msgr2=0x7f4094199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4094100f50 0x7f4094199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 -- 172.21.15.16:0/130896443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40940ffd60 msgr2=0x7f4094198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40940ffd60 0x7f4094198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.677+0000 7f409a15f700 1 -- 172.21.15.16:0/130896443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840095f0 con 0x7f40941018f0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.678+0000 7f409a15f700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 0x7f40941a0450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ef30 tx=0x7f408c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.678+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c018070 con 0x7f40941018f0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.678+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f408c00f040 con 0x7f40941018f0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.678+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40941a0f70 con 0x7f40941018f0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.678+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00cbc0 con 0x7f40941018f0 2024-05-08T23:19:27.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.678+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40941a13a0 con 0x7f40941018f0 2024-05-08T23:19:27.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.679+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4094067c10 con 0x7f40941018f0 2024-05-08T23:19:27.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.680+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f408c007500 con 0x7f40941018f0 2024-05-08T23:19:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.681+0000 7f408affd700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f40800a11b0 0x7f40800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.681+0000 7f409995e700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f40800a11b0 0x7f40800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.681+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f408c0cc8b0 con 0x7f40941018f0 2024-05-08T23:19:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.681+0000 7f409995e700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f40800a11b0 0x7f40800a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4094100db0 tx=0x7f409000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.685+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f408c095c10 con 0x7f40941018f0 2024-05-08T23:19:27.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.955+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f409410b420 con 0x7f40941018f0 2024-05-08T23:19:27.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.956+0000 7f408affd700 1 -- 172.21.15.16:0/130896443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f408c099230 con 0x7f40941018f0 2024-05-08T23:19:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:27.956 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:18:12.692850+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f40800a11b0 msgr2=0x7f40800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f40800a11b0 0x7f40800a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4094100db0 tx=0x7f409000b410 comp rx=0 tx=0).stop 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 msgr2=0x7f40941a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 0x7f40941a0450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f408c00ef30 tx=0x7f408c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 shutdown_connections 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40941018f0 0x7f40941a0450 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40940ffd60 0x7f4094198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f40800a11b0 0x7f40800a3670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 --2- 172.21.15.16:0/130896443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4094100f50 0x7f4094199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 >> 172.21.15.16:0/130896443 conn(0x7f40940fb2f0 msgr2=0x7f4094068020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 shutdown_connections 2024-05-08T23:19:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:27.959+0000 7f409bbc2700 1 -- 172.21.15.16:0/130896443 wait complete. 2024-05-08T23:19:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-05-08T23:19:27.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 -- 172.21.15.16:0/1124780242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48241042e0 msgr2=0x7f48241066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 --2- 172.21.15.16:0/1124780242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48241042e0 0x7f48241066d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4814009990 tx=0x7f4814009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 -- 172.21.15.16:0/1124780242 shutdown_connections 2024-05-08T23:19:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 --2- 172.21.15.16:0/1124780242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4824106c10 0x7f4824109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 --2- 172.21.15.16:0/1124780242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48241019b0 0x7f4824103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 --2- 172.21.15.16:0/1124780242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48241042e0 0x7f48241066d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 -- 172.21.15.16:0/1124780242 >> 172.21.15.16:0/1124780242 conn(0x7f48240fb320 msgr2=0x7f48240fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 -- 172.21.15.16:0/1124780242 shutdown_connections 2024-05-08T23:19:28.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.110+0000 7f482b7ed700 1 -- 172.21.15.16:0/1124780242 wait complete. 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 Processor -- start 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 -- start start 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48241019b0 0x7f4824196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48241042e0 0x7f4824197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 0x7f482419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482419e800 con 0x7f4824106c10 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482419e970 con 0x7f48241019b0 2024-05-08T23:19:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.111+0000 7f482b7ed700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482419eb30 con 0x7f48241042e0 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 0x7f482419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4828d88700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48241042e0 0x7f4824197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 0x7f482419e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41550/0 (socket says 172.21.15.16:41550) 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 -- 172.21.15.16:0/713282950 learned_addr learned my addr 172.21.15.16:0/713282950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829589700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48241019b0 0x7f4824196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 -- 172.21.15.16:0/713282950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48241042e0 msgr2=0x7f4824197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48241042e0 0x7f4824197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 -- 172.21.15.16:0/713282950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48241019b0 msgr2=0x7f4824196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48241019b0 0x7f4824196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 -- 172.21.15.16:0/713282950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140095f0 con 0x7f4824106c10 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f4829d8a700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 0x7f482419e230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f481c007ee0 tx=0x7f481c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c00ddd0 con 0x7f4824106c10 2024-05-08T23:19:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f481c00f040 con 0x7f4824106c10 2024-05-08T23:19:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482419ee10 con 0x7f4824106c10 2024-05-08T23:19:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c0047d0 con 0x7f4824106c10 2024-05-08T23:19:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.112+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482419f290 con 0x7f4824106c10 2024-05-08T23:19:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.113+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48240fcea0 con 0x7f4824106c10 2024-05-08T23:19:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.115+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f481c003f20 con 0x7f4824106c10 2024-05-08T23:19:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.115+0000 7f481a7fc700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48100a1230 0x7f48100a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.115+0000 7f4829589700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48100a1230 0x7f48100a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.115+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f481c0ccde0 con 0x7f4824106c10 2024-05-08T23:19:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.116+0000 7f4829589700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48100a1230 0x7f48100a36f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4820005950 tx=0x7f48200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.119+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f481c096110 con 0x7f4824106c10 2024-05-08T23:19:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:27 smithi189 ceph-mon[46919]: pgmap v275: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:28.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:27 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1446657635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:28.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:27 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3547661275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:28.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:27 smithi155 ceph-mon[47105]: pgmap v275: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:27 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1446657635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:27 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3547661275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.337+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4824067380 con 0x7f4824106c10 2024-05-08T23:19:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.339+0000 7f481a7fc700 1 -- 172.21.15.16:0/713282950 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+24470 (secure 0 0 0) 0x7f481c023020 con 0x7f4824106c10 2024-05-08T23:19:28.339 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:28.339 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":83,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:18:43.790069+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48100a1230 msgr2=0x7f48100a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48100a1230 0x7f48100a36f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4820005950 tx=0x7f48200058e0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 msgr2=0x7f482419e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 0x7f482419e230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f481c007ee0 tx=0x7f481c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 shutdown_connections 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4824106c10 0x7f482419e230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48241019b0 0x7f4824196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f48100a1230 0x7f48100a36f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 --2- 172.21.15.16:0/713282950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48241042e0 0x7f4824197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 >> 172.21.15.16:0/713282950 conn(0x7f48240fb320 msgr2=0x7f4824103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 shutdown_connections 2024-05-08T23:19:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.341+0000 7f482b7ed700 1 -- 172.21.15.16:0/713282950 wait complete. 2024-05-08T23:19:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:27 smithi016 ceph-mon[42541]: pgmap v275: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:19:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:27 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1446657635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:28.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:27 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3547661275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:28.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T23:17:46.078099+0000', 'modified': '2024-05-08T23:18:12.692850+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24563, 'mds_1': 24550, 'mds_2': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6835/1260766455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 1260766455}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 1260766455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24517': {'gid': 24517, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/3239398256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 3239398256}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 3239398256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.155:6837/1960878524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6836', 'nonce': 1960878524}, {'type': 'v1', 'addr': '172.21.15.155:6837', 'nonce': 1960878524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.189:6837/2098584238', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2098584238}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2098584238}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.189:6839/3614950817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 3614950817}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 3614950817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6839/773665585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6838', 'nonce': 773665585}, {'type': 'v1', 'addr': '172.21.15.155:6839', 'nonce': 773665585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T23:19:28.356 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-08T23:19:28.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.494+0000 7fd4de685700 1 -- 172.21.15.16:0/4265821593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8106ca0 msgr2=0x7fd4d81090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.494+0000 7fd4de685700 1 --2- 172.21.15.16:0/4265821593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8106ca0 0x7fd4d81090b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000b0a0 tx=0x7fd4d000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.494+0000 7fd4de685700 1 -- 172.21.15.16:0/4265821593 shutdown_connections 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.494+0000 7fd4de685700 1 --2- 172.21.15.16:0/4265821593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8106ca0 0x7fd4d81090b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.494+0000 7fd4de685700 1 --2- 172.21.15.16:0/4265821593 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8101a40 0x7fd4d8103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.495+0000 7fd4de685700 1 --2- 172.21.15.16:0/4265821593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d8104370 0x7fd4d8106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.495+0000 7fd4de685700 1 -- 172.21.15.16:0/4265821593 >> 172.21.15.16:0/4265821593 conn(0x7fd4d80fb3f0 msgr2=0x7fd4d80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.495+0000 7fd4de685700 1 -- 172.21.15.16:0/4265821593 shutdown_connections 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.495+0000 7fd4de685700 1 -- 172.21.15.16:0/4265821593 wait complete. 2024-05-08T23:19:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.495+0000 7fd4de685700 1 Processor -- start 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.495+0000 7fd4de685700 1 -- start start 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4de685700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d8101a40 0x7fd4d8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4de685700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 0x7fd4d81973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4de685700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8106ca0 0x7fd4d819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4de685700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d819e960 con 0x7fd4d8104370 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4de685700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d819eaa0 con 0x7fd4d8106ca0 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4de685700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d819ec30 con 0x7fd4d8101a40 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 0x7fd4d81973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 0x7fd4d81973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41568/0 (socket says 172.21.15.16:41568) 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 -- 172.21.15.16:0/4162851813 learned_addr learned my addr 172.21.15.16:0/4162851813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d8101a40 0x7fd4d8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8106ca0 0x7fd4d819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 -- 172.21.15.16:0/4162851813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d8101a40 msgr2=0x7fd4d8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d8101a40 0x7fd4d8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 -- 172.21.15.16:0/4162851813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8106ca0 msgr2=0x7fd4d819e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8106ca0 0x7fd4d819e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 -- 172.21.15.16:0/4162851813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0009d00 con 0x7fd4d8104370 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8106ca0 0x7fd4d819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 0x7fd4d81973c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c9b0 tx=0x7fd4cc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.496+0000 7fd4d57fa700 1 -- 172.21.15.16:0/4162851813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc007b00 con 0x7fd4d8104370 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.497+0000 7fd4d57fa700 1 -- 172.21.15.16:0/4162851813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4cc007c60 con 0x7fd4d8104370 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.497+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d819ef10 con 0x7fd4d8104370 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.497+0000 7fd4d57fa700 1 -- 172.21.15.16:0/4162851813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc018660 con 0x7fd4d8104370 2024-05-08T23:19:28.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.497+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d819f390 con 0x7fd4d8104370 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.499+0000 7fd4d57fa700 1 -- 172.21.15.16:0/4162851813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd4cc00f450 con 0x7fd4d8104370 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.499+0000 7fd4d57fa700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4c00a11b0 0x7fd4c00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.500+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4c00a11b0 0x7fd4c00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.500+0000 7fd4d57fa700 1 -- 172.21.15.16:0/4162851813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd4cc0ccde0 con 0x7fd4d8104370 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.500+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4c00a11b0 0x7fd4c00a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005950 tx=0x7fd4c80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.500+0000 7fd4ea83fb80 1 Processor -- start 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 -- start start 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a417910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a4182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a418c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a36e100 con 0x559c8a418810 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a36bac0 con 0x559c8a4174f0 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a3673b0 con 0x559c8a417e50 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a418c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a418c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41582/0 (socket says 172.21.15.16:41582) 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 -- 172.21.15.16:0/4232672454 learned_addr learned my addr 172.21.15.16:0/4232672454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a4182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a417910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 -- 172.21.15.16:0/4232672454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 msgr2=0x559c8a4182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a4182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 -- 172.21.15.16:0/4232672454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 msgr2=0x559c8a417910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a417910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 -- 172.21.15.16:0/4232672454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c8a1e8360 con 0x559c8a418810 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a417910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.501+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a418c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00062a0 tx=0x7fd4d0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b1f1e25e378e60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4be7fc700 1 -- 172.21.15.16:0/4232672454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000e050 con 0x559c8a418810 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4be7fc700 1 -- 172.21.15.16:0/4232672454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4d00090d0 con 0x559c8a418810 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4be7fc700 1 -- 172.21.15.16:0/4232672454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0005040 con 0x559c8a418810 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/4232672454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 msgr2=0x559c8a418c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a418c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00062a0 tx=0x7fd4d0009c20 comp rx=0 tx=0).stop 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/4232672454 shutdown_connections 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a418c90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a417910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/4232672454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a4182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/4232672454 >> 172.21.15.16:0/4232672454 conn(0x559c8a3751c0 msgr2=0x559c8a411d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/4232672454 shutdown_connections 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.502+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/4232672454 wait complete. 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 -- start start 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 Processor -- start 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a420d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a421240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a418810 0x559c8a428280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a303510 con 0x559c8a4174f0 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a3031b0 con 0x559c8a418810 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a1ea650 con 0x559c8a417e50 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4ea83fb80 10 client.0 _ll_register_callbacks cb 0x559c8a30cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.503+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a420d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a420d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41584/0 (socket says 172.21.15.16:41584) 2024-05-08T23:19:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 -- 172.21.15.16:0/656849496 learned_addr learned my addr 172.21.15.16:0/656849496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a421240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a418810 0x559c8a428280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 -- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 msgr2=0x559c8a421240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a421240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 -- 172.21.15.16:0/656849496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a418810 msgr2=0x559c8a428280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a418810 0x559c8a428280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 -- 172.21.15.16:0/656849496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d8109ca0 con 0x559c8a4174f0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a420d00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005d30 tx=0x7fd4c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4468edecf8165a47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4b77fe700 1 -- 172.21.15.16:0/656849496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c801f630 con 0x559c8a4174f0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4b77fe700 1 -- 172.21.15.16:0/656849496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd4c801f790 con 0x559c8a4174f0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c8a2c0220 con 0x559c8a4174f0 2024-05-08T23:19:28.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4b77fe700 1 -- 172.21.15.16:0/656849496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4c8011e70 con 0x559c8a4174f0 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4ea83fb80 10 client.15408 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.504+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559c8a194b10 con 0x559c8a4174f0 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4b77fe700 1 -- 172.21.15.16:0/656849496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c8a2c0220 con 0x559c8a4174f0 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4b77fe700 1 -- 172.21.15.16:0/656849496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4c8025020 con 0x559c8a4174f0 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4ea83fb80 10 client.15408 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4ea83fb80 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4ea83fb80 20 client.15408 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a42b9f0 0x559c8a4312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4ea83fb80 4 client.15408 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c8a3e67c0 con 0x559c8a42b9f0 2024-05-08T23:19:28.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.505+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a42b9f0 0x559c8a4312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.506+0000 7fd4b77fe700 10 client.15408 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.506+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a42b9f0 0x559c8a4312e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28b058be0064115 server_cookie=3fd5fd95749f927e in_seq=0 out_seq=0 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.506+0000 7fd4b77fe700 1 -- 172.21.15.16:0/656849496 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c8a3e67c0 con 0x559c8a42b9f0 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4b77fe700 10 client.15408 handle_command_reply: tid=0 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 client.15408 shutdown 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 msgr2=0x559c8a420d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a420d00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005d30 tx=0x7fd4c8004750 comp rx=0 tx=0).stop 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 shutdown_connections 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a42b9f0 0x559c8a4312e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a418810 0x559c8a428280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a420d00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4b77fe700 0 client.15408 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/656849496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a421240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 >> 172.21.15.16:0/656849496 conn(0x559c8a3751c0 msgr2=0x559c8a411550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 shutdown_connections 2024-05-08T23:19:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/656849496 wait complete. 2024-05-08T23:19:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.507+0000 7fd4ea83fb80 20 client.15408 trim_cache size 0 max 16384 2024-05-08T23:19:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 Processor -- start 2024-05-08T23:19:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 -- start start 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a4178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a417e50 0x559c8a41b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a418810 0x559c8a41bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a360150 con 0x559c8a4174f0 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a42d860 con 0x559c8a417e50 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.526+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a412b60 con 0x559c8a418810 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a4178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a4178d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41596/0 (socket says 172.21.15.16:41596) 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 -- 172.21.15.16:0/1652462760 learned_addr learned my addr 172.21.15.16:0/1652462760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a417e50 0x559c8a41b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a418810 0x559c8a41bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 -- 172.21.15.16:0/1652462760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a418810 msgr2=0x559c8a41bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a418810 0x559c8a41bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 -- 172.21.15.16:0/1652462760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a417e50 msgr2=0x559c8a41b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a417e50 0x559c8a41b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 -- 172.21.15.16:0/1652462760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c8a32d510 con 0x559c8a4174f0 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a417e50 0x559c8a41b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a4178d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8027a30 tx=0x7fd4c8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32c0696921c541a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4beffd700 1 -- 172.21.15.16:0/1652462760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8027600 con 0x559c8a4174f0 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4beffd700 1 -- 172.21.15.16:0/1652462760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4c8011e70 con 0x559c8a4174f0 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.527+0000 7fd4beffd700 1 -- 172.21.15.16:0/1652462760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c801fc40 con 0x559c8a4174f0 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/1652462760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 msgr2=0x559c8a4178d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a4178d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8027a30 tx=0x7fd4c8005b00 comp rx=0 tx=0).stop 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/1652462760 shutdown_connections 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a418810 0x559c8a41bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a4174f0 0x559c8a4178d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/1652462760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a417e50 0x559c8a41b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/1652462760 >> 172.21.15.16:0/1652462760 conn(0x559c8a3751c0 msgr2=0x559c8a418bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/1652462760 shutdown_connections 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/1652462760 wait complete. 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.528+0000 7fd4ea83fb80 1 -- start start 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 Processor -- start 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a41dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a41c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a41cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a41e540 con 0x559c8a418810 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a41d130 con 0x559c8a4174f0 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8a41d2f0 con 0x559c8a417e50 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4ea83fb80 10 client.0 _ll_register_callbacks cb 0x559c8a30cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a41cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a41cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41602/0 (socket says 172.21.15.16:41602) 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 -- 172.21.15.16:0/3546487507 learned_addr learned my addr 172.21.15.16:0/3546487507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a41dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4d77fe700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a41c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 -- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 msgr2=0x559c8a41c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a41c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 -- 172.21.15.16:0/3546487507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 msgr2=0x559c8a41dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a41dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.529+0000 7fd4dcc22700 1 -- 172.21.15.16:0/3546487507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c8a194b10 con 0x559c8a418810 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4dcc22700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a41cbc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009a20 tx=0x7fd4d000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4f8f76e56ac2af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4b77fe700 1 -- 172.21.15.16:0/3546487507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00076a0 con 0x559c8a418810 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4b77fe700 1 -- 172.21.15.16:0/3546487507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd4d0007800 con 0x559c8a418810 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4d00090d0 con 0x559c8a418810 2024-05-08T23:19:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4b77fe700 1 -- 172.21.15.16:0/3546487507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4d000e020 con 0x559c8a418810 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 10 client.15414 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559c8a1ecc80 con 0x559c8a418810 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4b77fe700 1 -- 172.21.15.16:0/3546487507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4d00090d0 con 0x559c8a418810 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4b77fe700 1 -- 172.21.15.16:0/3546487507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4d0035070 con 0x559c8a418810 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 10 client.15414 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 20 client.15414 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a48b590 0x559c8a487f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.530+0000 7fd4ea83fb80 4 client.15414 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.531+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559c8a35fe70 con 0x559c8a48b590 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.531+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a48b590 0x559c8a487f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.531+0000 7fd4b77fe700 10 client.15414 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.531+0000 7fd4d7fff700 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a48b590 0x559c8a487f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd7a9eda84be8a35 server_cookie=996aae75f0228e8 in_seq=0 out_seq=0 2024-05-08T23:19:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4b77fe700 1 -- 172.21.15.16:0/3546487507 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559c8a35fe70 con 0x559c8a48b590 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4b77fe700 10 client.15414 handle_command_reply: tid=0 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 client.15414 shutdown 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 msgr2=0x559c8a41cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a41cbc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009a20 tx=0x7fd4d000b910 comp rx=0 tx=0).stop 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 shutdown_connections 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c8a418810 0x559c8a41cbc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559c8a48b590 0x559c8a487f40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c8a4174f0 0x559c8a41dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 --2- 172.21.15.16:0/3546487507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c8a417e50 0x559c8a41c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 >> 172.21.15.16:0/3546487507 conn(0x559c8a3751c0 msgr2=0x559c8a41fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4b77fe700 0 client.15414 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 shutdown_connections 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 1 -- 172.21.15.16:0/3546487507 wait complete. 2024-05-08T23:19:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.532+0000 7fd4ea83fb80 20 client.15414 trim_cache size 0 max 16384 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24529, 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 76.847178377000006, 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-05-08T23:19:28.539 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 83, 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 117.442350207 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4c00a11b0 msgr2=0x7fd4c00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4c00a11b0 0x7fd4c00a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005950 tx=0x7fd4c80058e0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 msgr2=0x7fd4d81973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 0x7fd4d81973c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c9b0 tx=0x7fd4cc00cd70 comp rx=0 tx=0).stop 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 shutdown_connections 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4d8106ca0 0x7fd4d819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd4c00a11b0 0x7fd4c00a3670 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d8101a40 0x7fd4d8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 --2- 172.21.15.16:0/4162851813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4d8104370 0x7fd4d81973c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 >> 172.21.15.16:0/4162851813 conn(0x7fd4d80fb3f0 msgr2=0x7fd4d81023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 shutdown_connections 2024-05-08T23:19:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.533+0000 7fd4de685700 1 -- 172.21.15.16:0/4162851813 wait complete. 2024-05-08T23:19:28.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 -- 172.21.15.16:0/3000510170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc1021f0 msgr2=0x7f51cc102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 --2- 172.21.15.16:0/3000510170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc1021f0 0x7f51cc102650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009990 tx=0x7f51c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 -- 172.21.15.16:0/3000510170 shutdown_connections 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 --2- 172.21.15.16:0/3000510170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc102b90 0x7f51cc109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 --2- 172.21.15.16:0/3000510170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51cc0ff220 0x7f51cc0ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 --2- 172.21.15.16:0/3000510170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc1021f0 0x7f51cc102650 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 -- 172.21.15.16:0/3000510170 >> 172.21.15.16:0/3000510170 conn(0x7f51cc0fab50 msgr2=0x7f51cc0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 -- 172.21.15.16:0/3000510170 shutdown_connections 2024-05-08T23:19:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.685+0000 7f51d10db700 1 -- 172.21.15.16:0/3000510170 wait complete. 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 Processor -- start 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 -- start start 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ff220 0x7f51cc101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51cc1021f0 0x7f51cc1001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 0x7f51cc1006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc100e80 con 0x7f51cc102b90 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc100ff0 con 0x7f51cc0ff220 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51d10db700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc1a0e20 con 0x7f51cc1021f0 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51cb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 0x7f51cc1006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51cb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 0x7f51cc1006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41624/0 (socket says 172.21.15.16:41624) 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51cb59e700 1 -- 172.21.15.16:0/3666788279 learned_addr learned my addr 172.21.15.16:0/3666788279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51cad9d700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ff220 0x7f51cc101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.686+0000 7f51ca59c700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51cc1021f0 0x7f51cc1001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51cb59e700 1 -- 172.21.15.16:0/3666788279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51cc1021f0 msgr2=0x7f51cc1001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51cb59e700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51cc1021f0 0x7f51cc1001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51cb59e700 1 -- 172.21.15.16:0/3666788279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ff220 msgr2=0x7f51cc101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51cb59e700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ff220 0x7f51cc101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51cb59e700 1 -- 172.21.15.16:0/3666788279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00095f0 con 0x7f51cc102b90 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51cb59e700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 0x7f51cc1006e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ef30 tx=0x7f51c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51b3fff700 1 -- 172.21.15.16:0/3666788279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4018070 con 0x7f51cc102b90 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc1a1050 con 0x7f51cc102b90 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51b3fff700 1 -- 172.21.15.16:0/3666788279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c400f040 con 0x7f51cc102b90 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51b3fff700 1 -- 172.21.15.16:0/3666788279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c400cb60 con 0x7f51cc102b90 2024-05-08T23:19:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.687+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc1a1560 con 0x7f51cc102b90 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.689+0000 7f51b3fff700 1 -- 172.21.15.16:0/3666788279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f51c4007500 con 0x7f51cc102b90 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.690+0000 7f51b3fff700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f51b40a11b0 0x7f51b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.690+0000 7f51cad9d700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f51b40a11b0 0x7f51b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.690+0000 7f51b3fff700 1 -- 172.21.15.16:0/3666788279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f51c40cc8b0 con 0x7f51cc102b90 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.691+0000 7f51cad9d700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f51b40a11b0 0x7f51b40a3670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f51cc067df0 tx=0x7f51bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.691+0000 7f51dd295b80 1 Processor -- start 2024-05-08T23:19:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.691+0000 7f51dd295b80 1 -- start start 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e8922910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e8923c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51dd295b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e8879100 con 0x5626e8922e50 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51dd295b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e8876ac0 con 0x5626e89224f0 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51dd295b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e88723b0 con 0x5626e8923810 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89232d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41626/0 (socket says 172.21.15.16:41626) 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 -- 172.21.15.16:0/1719877774 learned_addr learned my addr 172.21.15.16:0/1719877774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51cad9d700 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e8922910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51cb59e700 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e8923c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 -- 172.21.15.16:0/1719877774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 msgr2=0x5626e8923c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e8923c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 -- 172.21.15.16:0/1719877774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 msgr2=0x5626e8922910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e8922910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 -- 172.21.15.16:0/1719877774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626e86f3360 con 0x5626e8922e50 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51cad9d700 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e8922910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51ca59c700 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89232d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009990 tx=0x7f51c0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816ebcb81497057f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51b17fa700 1 -- 172.21.15.16:0/1719877774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c001d070 con 0x5626e8922e50 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51b17fa700 1 -- 172.21.15.16:0/1719877774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c001e040 con 0x5626e8922e50 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.692+0000 7f51b17fa700 1 -- 172.21.15.16:0/1719877774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000b710 con 0x5626e8922e50 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 -- 172.21.15.16:0/1719877774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 msgr2=0x5626e89232d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89232d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009990 tx=0x7f51c0005bc0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 -- 172.21.15.16:0/1719877774 shutdown_connections 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e8923c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e8922910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 --2- 172.21.15.16:0/1719877774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89232d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 -- 172.21.15.16:0/1719877774 >> 172.21.15.16:0/1719877774 conn(0x5626e88801c0 msgr2=0x5626e891cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 -- 172.21.15.16:0/1719877774 shutdown_connections 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.693+0000 7f51dd295b80 1 -- 172.21.15.16:0/1719877774 wait complete. 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 -- start start 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 Processor -- start 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e89224f0 0x5626e892bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e8922e50 0x5626e892c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e8933240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e880e510 con 0x5626e8923810 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e880e1b0 con 0x5626e8922e50 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e86f5650 con 0x5626e89224f0 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51dd295b80 10 client.0 _ll_register_callbacks cb 0x5626e8817ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e8933240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e8933240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41638/0 (socket says 172.21.15.16:41638) 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 -- 172.21.15.16:0/628204635 learned_addr learned my addr 172.21.15.16:0/628204635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cad9d700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e89224f0 0x5626e892bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51ca59c700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e8922e50 0x5626e892c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 -- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e89224f0 msgr2=0x5626e892bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e89224f0 0x5626e892bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 -- 172.21.15.16:0/628204635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e8922e50 msgr2=0x5626e892c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e8922e50 0x5626e892c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.694+0000 7f51cb59e700 1 -- 172.21.15.16:0/628204635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc109d40 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51cb59e700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e8933240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f51c40c9f30 tx=0x7f51c4012400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d6a1c49cd60a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51aeff5700 1 -- 172.21.15.16:0/628204635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c40998d0 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51aeff5700 1 -- 172.21.15.16:0/628204635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f51c409d700 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626e87cb220 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51aeff5700 1 -- 172.21.15.16:0/628204635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c40b2200 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 10 client.15426 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5626e869fb10 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51aeff5700 1 -- 172.21.15.16:0/628204635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5626e87cb220 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51aeff5700 1 -- 172.21.15.16:0/628204635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51c4098dc0 con 0x5626e8923810 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 20 client.15426 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8936d90 0x5626e893c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 4 client.15426 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.695+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626e88f17c0 con 0x5626e8936d90 2024-05-08T23:19:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.696+0000 7f51cad9d700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8936d90 0x5626e893c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.696+0000 7f51aeff5700 10 client.15426 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.696+0000 7f51cad9d700 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8936d90 0x5626e893c090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f002a8d13d2b87bd server_cookie=51f858d2c73f00ac in_seq=0 out_seq=0 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51aeff5700 1 -- 172.21.15.16:0/628204635 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626e88f17c0 con 0x5626e8936d90 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51aeff5700 10 client.15426 handle_command_reply: tid=0 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 client.15426 shutdown 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 msgr2=0x5626e8933240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e8933240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f51c40c9f30 tx=0x7f51c4012400 comp rx=0 tx=0).stop 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 shutdown_connections 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e8933240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e89224f0 0x5626e892bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8936d90 0x5626e893c090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 --2- 172.21.15.16:0/628204635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e8922e50 0x5626e892c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 >> 172.21.15.16:0/628204635 conn(0x5626e88801c0 msgr2=0x5626e891c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51aeff5700 0 client.15426 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 shutdown_connections 2024-05-08T23:19:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 1 -- 172.21.15.16:0/628204635 wait complete. 2024-05-08T23:19:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.697+0000 7f51dd295b80 20 client.15426 trim_cache size 0 max 16384 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 Processor -- start 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 -- start start 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e89228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e892b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e891dc00 con 0x5626e8922e50 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e892ad00 con 0x5626e89224f0 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51dd295b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e87cb250 con 0x5626e8923810 2024-05-08T23:19:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41646/0 (socket says 172.21.15.16:41646) 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.716+0000 7f51ca59c700 1 -- 172.21.15.16:0/124619556 learned_addr learned my addr 172.21.15.16:0/124619556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51cb59e700 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e892b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51cad9d700 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e89228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51ca59c700 1 -- 172.21.15.16:0/124619556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 msgr2=0x5626e892b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51ca59c700 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e892b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51ca59c700 1 -- 172.21.15.16:0/124619556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 msgr2=0x5626e89228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51ca59c700 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e89228d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51ca59c700 1 -- 172.21.15.16:0/124619556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626e8838510 con 0x5626e8922e50 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51cad9d700 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e89228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51ca59c700 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89338f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f51c0005f50 tx=0x7f51c00092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07fb704df963f46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51b1ffb700 1 -- 172.21.15.16:0/124619556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c001d040 con 0x5626e8922e50 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51b1ffb700 1 -- 172.21.15.16:0/124619556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c0031070 con 0x5626e8922e50 2024-05-08T23:19:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.717+0000 7f51b1ffb700 1 -- 172.21.15.16:0/124619556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000bb40 con 0x5626e8922e50 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 -- 172.21.15.16:0/124619556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 msgr2=0x5626e89338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89338f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f51c0005f50 tx=0x7f51c00092f0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 -- 172.21.15.16:0/124619556 shutdown_connections 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8923810 0x5626e892b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e89228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 --2- 172.21.15.16:0/124619556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8922e50 0x5626e89338f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 -- 172.21.15.16:0/124619556 >> 172.21.15.16:0/124619556 conn(0x5626e88801c0 msgr2=0x5626e8923bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 -- 172.21.15.16:0/124619556 shutdown_connections 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 -- 172.21.15.16:0/124619556 wait complete. 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 -- start start 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 Processor -- start 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e891bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.718+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8922e50 0x5626e891c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51dd295b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e89327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51dd295b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e8932d00 con 0x5626e8923810 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51dd295b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e8932e70 con 0x5626e89224f0 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51dd295b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626e8933030 con 0x5626e8922e50 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51dd295b80 10 client.0 _ll_register_callbacks cb 0x5626e8817ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e89327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e89327c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41652/0 (socket says 172.21.15.16:41652) 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 -- 172.21.15.16:0/2684308920 learned_addr learned my addr 172.21.15.16:0/2684308920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51ca59c700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8922e50 0x5626e891c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cad9d700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e891bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 -- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8922e50 msgr2=0x5626e891c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8922e50 0x5626e891c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 -- 172.21.15.16:0/2684308920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 msgr2=0x5626e891bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e891bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 -- 172.21.15.16:0/2684308920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626e86f3360 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51cb59e700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e89327c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51c4097c20 tx=0x7f51c40ca360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6f0bedf73a3de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51aeff5700 1 -- 172.21.15.16:0/2684308920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c409aae0 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51aeff5700 1 -- 172.21.15.16:0/2684308920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f51c40a1a80 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.719+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51c001e040 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51aeff5700 1 -- 172.21.15.16:0/2684308920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c40b2200 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 10 client.15432 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5626e86f7c80 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51aeff5700 1 -- 172.21.15.16:0/2684308920 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c407e020 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51aeff5700 1 -- 172.21.15.16:0/2684308920 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f51c409d170 con 0x5626e8923810 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:28.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 10 client.15432 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 20 client.15432 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8991ed0 0x5626e898e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 4 client.15432 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626e886ae70 con 0x5626e8991ed0 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.720+0000 7f51cad9d700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8991ed0 0x5626e898e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51aeff5700 10 client.15432 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51cad9d700 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8991ed0 0x5626e898e860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b462d03d35227080 server_cookie=35654aeb449ee4a5 in_seq=0 out_seq=0 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51aeff5700 1 -- 172.21.15.16:0/2684308920 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5626e886ae70 con 0x5626e8991ed0 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51aeff5700 10 client.15432 handle_command_reply: tid=0 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 client.15432 shutdown 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 msgr2=0x5626e89327c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e89327c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51c4097c20 tx=0x7f51c40ca360 comp rx=0 tx=0).stop 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 shutdown_connections 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626e8923810 0x5626e89327c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5626e8991ed0 0x5626e898e860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5626e89224f0 0x5626e891bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 --2- 172.21.15.16:0/2684308920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626e8922e50 0x5626e891c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51aeff5700 0 client.15432 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 >> 172.21.15.16:0/2684308920 conn(0x5626e88801c0 msgr2=0x5626e89266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 shutdown_connections 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 1 -- 172.21.15.16:0/2684308920 wait complete. 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.721+0000 7f51dd295b80 20 client.15432 trim_cache size 0 max 16384 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-05-08T23:19:28.728 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24550, 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 78.042287548000004, 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 83, 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 114.04073304000001 2024-05-08T23:19:28.729 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f51b40a11b0 msgr2=0x7f51b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f51b40a11b0 0x7f51b40a3670 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f51cc067df0 tx=0x7f51bc00b410 comp rx=0 tx=0).stop 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 msgr2=0x7f51cc1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 0x7f51cc1006e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f51c400ef30 tx=0x7f51c400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 shutdown_connections 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51cc102b90 0x7f51cc1006e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51cc0ff220 0x7f51cc101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f51b40a11b0 0x7f51b40a3670 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 --2- 172.21.15.16:0/3666788279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51cc1021f0 0x7f51cc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 >> 172.21.15.16:0/3666788279 conn(0x7f51cc0fab50 msgr2=0x7f51cc1055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 shutdown_connections 2024-05-08T23:19:28.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.722+0000 7f51d10db700 1 -- 172.21.15.16:0/3666788279 wait complete. 2024-05-08T23:19:28.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T23:19:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 -- 172.21.15.16:0/1421929057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c100fd0 msgr2=0x7f240c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 --2- 172.21.15.16:0/1421929057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c100fd0 0x7f240c101450 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009990 tx=0x7f23f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 -- 172.21.15.16:0/1421929057 shutdown_connections 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 --2- 172.21.15.16:0/1421929057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f240c101990 0x7f240c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 --2- 172.21.15.16:0/1421929057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f240c0ffe80 0x7f240c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 --2- 172.21.15.16:0/1421929057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c100fd0 0x7f240c101450 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 -- 172.21.15.16:0/1421929057 >> 172.21.15.16:0/1421929057 conn(0x7f240c0fb3f0 msgr2=0x7f240c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 -- 172.21.15.16:0/1421929057 shutdown_connections 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.873+0000 7f2411131700 1 -- 172.21.15.16:0/1421929057 wait complete. 2024-05-08T23:19:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 Processor -- start 2024-05-08T23:19:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 -- start start 2024-05-08T23:19:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 0x7f240c069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f240c100fd0 0x7f240c19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f240c101990 0x7f240c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19e950 con 0x7f240c0ffe80 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19eac0 con 0x7f240c100fd0 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f2411131700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19ec80 con 0x7f240c101990 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f240ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 0x7f240c069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f240ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 0x7f240c069ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41678/0 (socket says 172.21.15.16:41678) 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.874+0000 7f240ad9d700 1 -- 172.21.15.16:0/1019921925 learned_addr learned my addr 172.21.15.16:0/1019921925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240a59c700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f240c100fd0 0x7f240c19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240b59e700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f240c101990 0x7f240c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240ad9d700 1 -- 172.21.15.16:0/1019921925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f240c101990 msgr2=0x7f240c19e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240ad9d700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f240c101990 0x7f240c19e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240ad9d700 1 -- 172.21.15.16:0/1019921925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f240c100fd0 msgr2=0x7f240c19bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240ad9d700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f240c100fd0 0x7f240c19bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240ad9d700 1 -- 172.21.15.16:0/1019921925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f40095f0 con 0x7f240c0ffe80 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f240ad9d700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 0x7f240c069ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e9a0 tx=0x7f23fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f2403fff700 1 -- 172.21.15.16:0/1019921925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc009790 con 0x7f240c0ffe80 2024-05-08T23:19:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f2403fff700 1 -- 172.21.15.16:0/1019921925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23fc010430 con 0x7f240c0ffe80 2024-05-08T23:19:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c19ef00 con 0x7f240c0ffe80 2024-05-08T23:19:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f2403fff700 1 -- 172.21.15.16:0/1019921925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00cea0 con 0x7f240c0ffe80 2024-05-08T23:19:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.875+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c19f320 con 0x7f240c0ffe80 2024-05-08T23:19:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.878+0000 7f2403fff700 1 -- 172.21.15.16:0/1019921925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f23fc0105a0 con 0x7f240c0ffe80 2024-05-08T23:19:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.878+0000 7f2403fff700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23f80a11b0 0x7f23f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.878+0000 7f2403fff700 1 -- 172.21.15.16:0/1019921925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f23fc014070 con 0x7f240c0ffe80 2024-05-08T23:19:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.878+0000 7f240a59c700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23f80a11b0 0x7f23f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f240a59c700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23f80a11b0 0x7f23f80a3670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23f4006010 tx=0x7f23f400b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 Processor -- start 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 -- start start 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa57696910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa57697c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa575ed100 con 0x55aa57696e50 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa575eaac0 con 0x55aa576964f0 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.879+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa575e63b0 con 0x55aa57697810 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41692/0 (socket says 172.21.15.16:41692) 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 -- 172.21.15.16:0/2931479188 learned_addr learned my addr 172.21.15.16:0/2931479188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240b59e700 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa57697c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240ad9d700 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa57696910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 -- 172.21.15.16:0/2931479188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 msgr2=0x55aa57697c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa57697c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 -- 172.21.15.16:0/2931479188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 msgr2=0x55aa57696910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa57696910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 -- 172.21.15.16:0/2931479188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa57467360 con 0x55aa57696e50 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f240a59c700 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576972d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f23f4015000 tx=0x7f23f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10c90749bc11ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f24017fa700 1 -- 172.21.15.16:0/2931479188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4003c90 con 0x55aa57696e50 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f24017fa700 1 -- 172.21.15.16:0/2931479188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23f401e040 con 0x55aa57696e50 2024-05-08T23:19:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.880+0000 7f24017fa700 1 -- 172.21.15.16:0/2931479188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4004900 con 0x55aa57696e50 2024-05-08T23:19:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2931479188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 msgr2=0x55aa576972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576972d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f23f4015000 tx=0x7f23f4020000 comp rx=0 tx=0).stop 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2931479188 shutdown_connections 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa57697c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa57696910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2931479188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576972d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2931479188 >> 172.21.15.16:0/2931479188 conn(0x55aa575f41c0 msgr2=0x55aa57690d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2931479188 shutdown_connections 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.881+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2931479188 wait complete. 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 -- start start 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 Processor -- start 2024-05-08T23:19:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa5769fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa576a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa57582510 con 0x55aa57696e50 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa575821b0 con 0x55aa57697810 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa57469650 con 0x55aa576964f0 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f241d2ebb80 10 client.0 _ll_register_callbacks cb 0x55aa5758bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41708/0 (socket says 172.21.15.16:41708) 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 -- 172.21.15.16:0/3867019248 learned_addr learned my addr 172.21.15.16:0/3867019248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240b59e700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa576a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240ad9d700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa5769fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 -- 172.21.15.16:0/3867019248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 msgr2=0x55aa5769fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa5769fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 -- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 msgr2=0x55aa576a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa576a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240a59c700 1 -- 172.21.15.16:0/3867019248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c108be0 con 0x55aa57696e50 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240ad9d700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa5769fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.882+0000 7f240b59e700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa576a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f240a59c700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a02c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009350 tx=0x7f23f4004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4f88d95807a8b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f23f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f23f27fc700 1 -- 172.21.15.16:0/3867019248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f400b6b0 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f23f27fc700 1 -- 172.21.15.16:0/3867019248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f23f400b810 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa5753f220 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f23f27fc700 1 -- 172.21.15.16:0/3867019248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f400b9d0 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 10 client.15444 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55aa57413b10 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f23f27fc700 1 -- 172.21.15.16:0/3867019248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa5753f220 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f23f27fc700 1 -- 172.21.15.16:0/3867019248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23f402f440 con 0x55aa57696e50 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 10 client.15444 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 20 client.15444 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa576aaae0 0x55aa576afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 4 client.15444 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.883+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa576657c0 con 0x55aa576aaae0 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.884+0000 7f240ad9d700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa576aaae0 0x55aa576afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.884+0000 7f240ad9d700 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa576aaae0 0x55aa576afdc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b84701759a623d3 server_cookie=193256db8affc972 in_seq=0 out_seq=0 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.884+0000 7f23f27fc700 10 client.15444 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f23f27fc700 1 -- 172.21.15.16:0/3867019248 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa576657c0 con 0x55aa576aaae0 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f23f27fc700 10 client.15444 handle_command_reply: tid=0 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 client.15444 shutdown 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 msgr2=0x55aa576a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a02c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009350 tx=0x7f23f4004ea0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 shutdown_connections 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa576aaae0 0x55aa576afdc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa576a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f23f27fc700 0 client.15444 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa5769fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/3867019248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a02c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 >> 172.21.15.16:0/3867019248 conn(0x55aa575f41c0 msgr2=0x55aa57690550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 shutdown_connections 2024-05-08T23:19:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 1 -- 172.21.15.16:0/3867019248 wait complete. 2024-05-08T23:19:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.885+0000 7f241d2ebb80 20 client.15444 trim_cache size 0 max 16384 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 Processor -- start 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 -- start start 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa576968d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa5769ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa5769fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa575821b0 con 0x55aa57696e50 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa576657c0 con 0x55aa57697810 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.904+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa5753f220 con 0x55aa576964f0 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa5769ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa5769ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41720/0 (socket says 172.21.15.16:41720) 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 -- 172.21.15.16:0/2530470991 learned_addr learned my addr 172.21.15.16:0/2530470991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240ad9d700 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa576968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240b59e700 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa5769fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 -- 172.21.15.16:0/2530470991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 msgr2=0x55aa576968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa576968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 -- 172.21.15.16:0/2530470991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 msgr2=0x55aa5769fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa5769fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 -- 172.21.15.16:0/2530470991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa575ac510 con 0x55aa57696e50 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f240a59c700 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa5769ddc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23f4003960 tx=0x7f23f40042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e889960a1f903d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f2401ffb700 1 -- 172.21.15.16:0/2530470991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40383f0 con 0x55aa57696e50 2024-05-08T23:19:28.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f2401ffb700 1 -- 172.21.15.16:0/2530470991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23f400bb40 con 0x55aa57696e50 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.905+0000 7f2401ffb700 1 -- 172.21.15.16:0/2530470991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4038800 con 0x55aa57696e50 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2530470991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 msgr2=0x55aa5769ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa5769ddc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23f4003960 tx=0x7f23f40042c0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2530470991 shutdown_connections 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa57697810 0x55aa5769fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa576964f0 0x55aa576968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/2530470991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa5769ddc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2530470991 >> 172.21.15.16:0/2530470991 conn(0x55aa575f41c0 msgr2=0x55aa57697bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2530470991 shutdown_connections 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 -- 172.21.15.16:0/2530470991 wait complete. 2024-05-08T23:19:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 -- start start 2024-05-08T23:19:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.906+0000 7f241d2ebb80 1 Processor -- start 2024-05-08T23:19:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa576a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa576a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa576a25a0 con 0x55aa57696e50 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa576a2710 con 0x55aa576964f0 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa576a28d0 con 0x55aa57697810 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f241d2ebb80 10 client.0 _ll_register_callbacks cb 0x55aa5758bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41732/0 (socket says 172.21.15.16:41732) 2024-05-08T23:19:28.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 -- 172.21.15.16:0/4039986937 learned_addr learned my addr 172.21.15.16:0/4039986937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240ad9d700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa576a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240b59e700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa576a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 -- 172.21.15.16:0/4039986937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 msgr2=0x55aa576a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa576a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 -- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 msgr2=0x55aa576a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa576a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 -- 172.21.15.16:0/4039986937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa57413b10 con 0x55aa57696e50 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f240a59c700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a1980 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f23f402f7a0 tx=0x7f23f402fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8dfbd6bcea949d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.907+0000 7f23f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f23f27fc700 1 -- 172.21.15.16:0/4039986937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f400b6b0 con 0x55aa57696e50 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f23f27fc700 1 -- 172.21.15.16:0/4039986937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f23f400b810 con 0x55aa57696e50 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa575dee70 con 0x55aa57696e50 2024-05-08T23:19:28.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f23f27fc700 1 -- 172.21.15.16:0/4039986937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f400b9d0 con 0x55aa57696e50 2024-05-08T23:19:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 10 client.15450 fetch_fsmap learned FSMap version 24 2024-05-08T23:19:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55aa5746bc80 con 0x55aa57696e50 2024-05-08T23:19:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f23f27fc700 1 -- 172.21.15.16:0/4039986937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa575dee70 con 0x55aa57696e50 2024-05-08T23:19:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f23f27fc700 1 -- 172.21.15.16:0/4039986937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23f4046070 con 0x55aa57696e50 2024-05-08T23:19:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 10 client.15450 fetch_fsmap finished waiting for FSMap version 24 2024-05-08T23:19:28.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 10 client.15450 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 20 client.15450 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa57709aa0 0x55aa57706410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 4 client.15450 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa57434ca0 con 0x55aa57709aa0 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.908+0000 7f240ad9d700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa57709aa0 0x55aa57706410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f23f27fc700 10 client.15450 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f240ad9d700 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa57709aa0 0x55aa57706410 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f55d3adfa495ba server_cookie=252fe740c32f8396 in_seq=0 out_seq=0 2024-05-08T23:19:28.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f23f27fc700 1 -- 172.21.15.16:0/4039986937 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55aa57434ca0 con 0x55aa57709aa0 2024-05-08T23:19:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f23f27fc700 10 client.15450 handle_command_reply: tid=0 2024-05-08T23:19:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 client.15450 shutdown 2024-05-08T23:19:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 msgr2=0x55aa576a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a1980 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f23f402f7a0 tx=0x7f23f402fb40 comp rx=0 tx=0).stop 2024-05-08T23:19:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 shutdown_connections 2024-05-08T23:19:28.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa57697810 0x55aa576a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55aa57709aa0 0x55aa57706410 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55aa576964f0 0x55aa576a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 --2- 172.21.15.16:0/4039986937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55aa57696e50 0x55aa576a1980 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f23f27fc700 0 client.15450 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 >> 172.21.15.16:0/4039986937 conn(0x55aa575f41c0 msgr2=0x55aa57690d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 shutdown_connections 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.909+0000 7f241d2ebb80 1 -- 172.21.15.16:0/4039986937 wait complete. 2024-05-08T23:19:28.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f241d2ebb80 20 client.15450 trim_cache size 0 max 16384 2024-05-08T23:19:28.920 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:28.920 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "378de17a-0d90-11ef-bc97-c7b262605968", 2024-05-08T23:19:28.920 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-05-08T23:19:28.920 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24563, 2024-05-08T23:19:28.920 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 102.828039308, 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 24, 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 83, 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 81, 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 108.857123507 2024-05-08T23:19:28.921 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23f80a11b0 msgr2=0x7f23f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23f80a11b0 0x7f23f80a3670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23f4006010 tx=0x7f23f400b5d0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 msgr2=0x7f240c069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 0x7f240c069ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00e9a0 tx=0x7f23fc00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 shutdown_connections 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f23f80a11b0 0x7f23f80a3670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f240c101990 0x7f240c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f240c0ffe80 0x7f240c069ff0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 --2- 172.21.15.16:0/1019921925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f240c100fd0 0x7f240c19bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 >> 172.21.15.16:0/1019921925 conn(0x7f240c0fb3f0 msgr2=0x7f240c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 shutdown_connections 2024-05-08T23:19:28.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:28.910+0000 7f2411131700 1 -- 172.21.15.16:0/1019921925 wait complete. 2024-05-08T23:19:28.925 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T23:19:28.934 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T23:19:28.934 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T23:19:28.935 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T23:19:28.935 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-08T23:19:28.936 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:19:28.936 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:19:28.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-08T23:19:29.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/471575197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100ff0 msgr2=0x7fa9f4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/471575197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100ff0 0x7fa9f4101450 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80099c0 tx=0x7fa9e8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/471575197 shutdown_connections 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/471575197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 0x7fa9f4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/471575197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/471575197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100ff0 0x7fa9f4101450 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/471575197 >> 172.21.15.16:0/471575197 conn(0x7fa9f40fb390 msgr2=0x7fa9f40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/471575197 shutdown_connections 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.073+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/471575197 wait complete. 2024-05-08T23:19:29.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.074+0000 7fa9f9ef5700 1 Processor -- start 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.074+0000 7fa9f9ef5700 1 -- start start 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f9ef5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f9ef5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f4100ff0 0x7fa9f41961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f9ef5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 0x7fa9f419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f9ef5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419d780 con 0x7fa9f40ffe00 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f9ef5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419d8f0 con 0x7fa9f4101990 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f9ef5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419dab0 con 0x7fa9f4100ff0 2024-05-08T23:19:29.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41756/0 (socket says 172.21.15.16:41756) 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 -- 172.21.15.16:0/2340535476 learned_addr learned my addr 172.21.15.16:0/2340535476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f3fff700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 0x7fa9f419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f2ffd700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f4100ff0 0x7fa9f41961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 -- 172.21.15.16:0/2340535476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f4100ff0 msgr2=0x7fa9f41961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f4100ff0 0x7fa9f41961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 -- 172.21.15.16:0/2340535476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 msgr2=0x7fa9f419d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 0x7fa9f419d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 -- 172.21.15.16:0/2340535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80095f0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f2ffd700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f4100ff0 0x7fa9f41961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f3fff700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 0x7fa9f419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.075+0000 7fa9f37fe700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4195c90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e970 tx=0x7fa9e400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:29.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.076+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4009790 con 0x7fa9f40ffe00 2024-05-08T23:19:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.076+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9e40044b0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.076+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f419dd90 con 0x7fa9f40ffe00 2024-05-08T23:19:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.076+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4003dd0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.076+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f419e1e0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.077+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4107320 con 0x7fa9f40ffe00 2024-05-08T23:19:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.078+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa9e4003f30 con 0x7fa9f40ffe00 2024-05-08T23:19:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.079+0000 7fa9f0ff9700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa9dc0a11b0 0x7fa9dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.079+0000 7fa9f2ffd700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa9dc0a11b0 0x7fa9dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.079+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa9e4014070 con 0x7fa9f40ffe00 2024-05-08T23:19:29.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.079+0000 7fa9f2ffd700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa9dc0a11b0 0x7fa9dc0a3670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:29.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.082+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa9e40964b0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:28 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/130896443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:29.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:28 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/713282950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:29.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:28 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/130896443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:29.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:28 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/713282950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:29.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:28 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/130896443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:29.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:28 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/713282950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:29.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.369+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fa9f402cda0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.936+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7fa9e40041e0 con 0x7fa9f40ffe00 2024-05-08T23:19:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.937+0000 7fa9f0ff9700 1 -- 172.21.15.16:0/2340535476 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa9e4099f40 con 0x7fa9f40ffe00 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa9dc0a11b0 msgr2=0x7fa9dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa9dc0a11b0 0x7fa9dc0a3670 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e800b540 comp rx=0 tx=0).stop 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 msgr2=0x7fa9f4195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4195c90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e970 tx=0x7fa9e400ed30 comp rx=0 tx=0).stop 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 shutdown_connections 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9f4101990 0x7fa9f419d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f40ffe00 0x7fa9f4195c90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fa9dc0a11b0 0x7fa9dc0a3670 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 --2- 172.21.15.16:0/2340535476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f4100ff0 0x7fa9f41961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 >> 172.21.15.16:0/2340535476 conn(0x7fa9f40fb390 msgr2=0x7fa9f4104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 shutdown_connections 2024-05-08T23:19:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:29.940+0000 7fa9f9ef5700 1 -- 172.21.15.16:0/2340535476 wait complete. 2024-05-08T23:19:29.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-08T23:19:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 -- 172.21.15.16:0/4042916509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840739a0 msgr2=0x7efd84073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 --2- 172.21.15.16:0/4042916509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840739a0 0x7efd84073e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd78009990 tx=0x7efd78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 -- 172.21.15.16:0/4042916509 shutdown_connections 2024-05-08T23:19:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 --2- 172.21.15.16:0/4042916509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd84108df0 0x7efd8410b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 --2- 172.21.15.16:0/4042916509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd84074f80 0x7efd840733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 --2- 172.21.15.16:0/4042916509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840739a0 0x7efd84073e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 -- 172.21.15.16:0/4042916509 >> 172.21.15.16:0/4042916509 conn(0x7efd840fc490 msgr2=0x7efd840fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 -- 172.21.15.16:0/4042916509 shutdown_connections 2024-05-08T23:19:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.103+0000 7efd8a879700 1 -- 172.21.15.16:0/4042916509 wait complete. 2024-05-08T23:19:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.104+0000 7efd8a879700 1 Processor -- start 2024-05-08T23:19:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.104+0000 7efd8a879700 1 -- start start 2024-05-08T23:19:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd8a879700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 0x7efd84198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd8a879700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd84074f80 0x7efd841994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd8a879700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd84108df0 0x7efd841a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd8a879700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd841a0ae0 con 0x7efd84108df0 2024-05-08T23:19:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd8a879700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd841a0c20 con 0x7efd840739a0 2024-05-08T23:19:30.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd8a879700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd841a0db0 con 0x7efd84074f80 2024-05-08T23:19:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 0x7efd84198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 0x7efd84198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39304/0 (socket says 172.21.15.16:39304) 2024-05-08T23:19:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 -- 172.21.15.16:0/646132298 learned_addr learned my addr 172.21.15.16:0/646132298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd837fe700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd84074f80 0x7efd841994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd88e16700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd84108df0 0x7efd841a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 -- 172.21.15.16:0/646132298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd84074f80 msgr2=0x7efd841994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd84074f80 0x7efd841994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 -- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd84108df0 msgr2=0x7efd841a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd84108df0 0x7efd841a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 -- 172.21.15.16:0/646132298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780095f0 con 0x7efd840739a0 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.105+0000 7efd83fff700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 0x7efd84198fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd7400e9a0 tx=0x7efd7400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.106+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400ca10 con 0x7efd840739a0 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.106+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd74010430 con 0x7efd840739a0 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.106+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74018680 con 0x7efd840739a0 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.106+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd841a1090 con 0x7efd840739a0 2024-05-08T23:19:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.106+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd841a1550 con 0x7efd840739a0 2024-05-08T23:19:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.107+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd84107c00 con 0x7efd840739a0 2024-05-08T23:19:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.110+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7efd740105a0 con 0x7efd840739a0 2024-05-08T23:19:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.111+0000 7efd817fa700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 0x7efd6c0a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.111+0000 7efd837fe700 1 -- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 msgr2=0x7efd6c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/781396085 2024-05-08T23:19:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.111+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efd74014070 con 0x7efd840739a0 2024-05-08T23:19:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.111+0000 7efd837fe700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 0x7efd6c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:19:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.113+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efd74096e10 con 0x7efd840739a0 2024-05-08T23:19:30.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:29 smithi155 ceph-mon[47105]: pgmap v276: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:29 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2340535476' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:19:30.215 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:29 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: ignoring --setuser ceph since I am not root 2024-05-08T23:19:30.215 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:29 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: ignoring --setgroup ceph since I am not root 2024-05-08T23:19:30.215 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:29 smithi155 ceph-mgr[48614]: -- 172.21.15.155:0/1507427973 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562efb7341a0 con 0x562efb70b000 2024-05-08T23:19:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.312+0000 7efd837fe700 1 -- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 msgr2=0x7efd6c0a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/781396085 2024-05-08T23:19:30.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.312+0000 7efd837fe700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 0x7efd6c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:19:30.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.332+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7efd8410ea90 con 0x7efd840739a0 2024-05-08T23:19:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.337+0000 7efd817fa700 1 -- 172.21.15.16:0/646132298 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7efd7409a430 con 0x7efd840739a0 2024-05-08T23:19:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 msgr2=0x7efd6c0a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:19:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 0x7efd6c0a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 msgr2=0x7efd84198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 0x7efd84198fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd7400e9a0 tx=0x7efd7400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 shutdown_connections 2024-05-08T23:19:30.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd84108df0 0x7efd841a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd840739a0 0x7efd84198fb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7efd6c0a1200 0x7efd6c0a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 --2- 172.21.15.16:0/646132298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd84074f80 0x7efd841994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 >> 172.21.15.16:0/646132298 conn(0x7efd840fc490 msgr2=0x7efd84104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 shutdown_connections 2024-05-08T23:19:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.340+0000 7efd8a879700 1 -- 172.21.15.16:0/646132298 wait complete. 2024-05-08T23:19:30.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:29 smithi016 ceph-mon[42541]: pgmap v276: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:30.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:29 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2340535476' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:19:30.353 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:29 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: ignoring --setuser ceph since I am not root 2024-05-08T23:19:30.353 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:29 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: ignoring --setgroup ceph since I am not root 2024-05-08T23:19:30.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-08T23:19:30.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:29 smithi189 ceph-mon[46919]: pgmap v276: 129 pgs: 129 active+clean; 640 KiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:19:30.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:29 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2340535476' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T23:19:30.375 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: ignoring --setuser ceph since I am not root 2024-05-08T23:19:30.375 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: ignoring --setgroup ceph since I am not root 2024-05-08T23:19:30.375 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:29 smithi189 ceph-mgr[48803]: -- 172.21.15.189:0/969906524 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559f153a1a0 con 0x5559f1511000 2024-05-08T23:19:30.376 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:30.374+0000 7f8d1a598200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:19:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 -- 172.21.15.16:0/748002868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c108dd0 msgr2=0x7fd25c10b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 --2- 172.21.15.16:0/748002868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c108dd0 0x7fd25c10b260 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd25400b0a0 tx=0x7fd25400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 -- 172.21.15.16:0/748002868 shutdown_connections 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 --2- 172.21.15.16:0/748002868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c108dd0 0x7fd25c10b260 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 --2- 172.21.15.16:0/748002868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd25c074fe0 0x7fd25c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 --2- 172.21.15.16:0/748002868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c073a00 0x7fd25c073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 -- 172.21.15.16:0/748002868 >> 172.21.15.16:0/748002868 conn(0x7fd25c0fc4a0 msgr2=0x7fd25c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 -- 172.21.15.16:0/748002868 shutdown_connections 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.494+0000 7fd262288700 1 -- 172.21.15.16:0/748002868 wait complete. 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.495+0000 7fd262288700 1 Processor -- start 2024-05-08T23:19:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.495+0000 7fd262288700 1 -- start start 2024-05-08T23:19:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.495+0000 7fd262288700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 0x7fd25c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.495+0000 7fd262288700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c074fe0 0x7fd25c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.495+0000 7fd262288700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd25c108dd0 0x7fd25c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.495+0000 7fd262288700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1a0b90 con 0x7fd25c073a00 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd262288700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1a0cd0 con 0x7fd25c074fe0 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd262288700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c1a0e60 con 0x7fd25c108dd0 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 0x7fd25c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 0x7fd25c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41790/0 (socket says 172.21.15.16:41790) 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 -- 172.21.15.16:0/1120762461 learned_addr learned my addr 172.21.15.16:0/1120762461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25b7fe700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c074fe0 0x7fd25c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd260825700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd25c108dd0 0x7fd25c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 -- 172.21.15.16:0/1120762461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd25c108dd0 msgr2=0x7fd25c1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd25c108dd0 0x7fd25c1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 -- 172.21.15.16:0/1120762461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c074fe0 msgr2=0x7fd25c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c074fe0 0x7fd25c1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 -- 172.21.15.16:0/1120762461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250009520 con 0x7fd25c073a00 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd25bfff700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 0x7fd25c1990b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00eb30 tx=0x7fd24c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c009880 con 0x7fd25c073a00 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd24c0044b0 con 0x7fd25c073a00 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c009520 con 0x7fd25c073a00 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.496+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c003ce0 con 0x7fd25c073a00 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.497+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd254009d00 con 0x7fd25c073a00 2024-05-08T23:19:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.497+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c107b90 con 0x7fd25c073a00 2024-05-08T23:19:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.500+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7fd24c003e40 con 0x7fd25c073a00 2024-05-08T23:19:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.500+0000 7fd2597fa700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 0x7fd2440a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.500+0000 7fd25b7fe700 1 -- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 msgr2=0x7fd2440a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/781396085 2024-05-08T23:19:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.500+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd24c014070 con 0x7fd25c073a00 2024-05-08T23:19:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.500+0000 7fd25b7fe700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 0x7fd2440a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:19:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.504+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd24c096700 con 0x7fd25c073a00 2024-05-08T23:19:30.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:30.435+0000 7f421a34b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:19:30.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:30.563+0000 7f421a34b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:19:30.641 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:30.468+0000 7f8d1a598200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:19:30.641 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:30.555+0000 7f8d1a598200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:19:30.645 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:30.378+0000 7f06073be200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T23:19:30.645 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:30.470+0000 7f06073be200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T23:19:30.645 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:30.560+0000 7f06073be200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:19:30.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.701+0000 7fd25b7fe700 1 -- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 msgr2=0x7fd2440a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/781396085 2024-05-08T23:19:30.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.701+0000 7fd25b7fe700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 0x7fd2440a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:19:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.746+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fd25c10e7b0 con 0x7fd25c073a00 2024-05-08T23:19:30.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.750+0000 7fd2597fa700 1 -- 172.21.15.16:0/1120762461 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd24c099d20 con 0x7fd25c073a00 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 msgr2=0x7fd2440a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 0x7fd2440a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 msgr2=0x7fd25c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 0x7fd25c1990b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd24c00eb30 tx=0x7fd24c00ee40 comp rx=0 tx=0).stop 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 shutdown_connections 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7fd2440a1200 0x7fd2440a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd25c108dd0 0x7fd25c1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c073a00 0x7fd25c1990b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 --2- 172.21.15.16:0/1120762461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd25c074fe0 0x7fd25c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 >> 172.21.15.16:0/1120762461 conn(0x7fd25c0fc4a0 msgr2=0x7fd25c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 shutdown_connections 2024-05-08T23:19:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:30.753+0000 7fd262288700 1 -- 172.21.15.16:0/1120762461 wait complete. 2024-05-08T23:19:30.767 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-08T23:19:30.779 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-08T23:19:30.779 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-08T23:19:30.779 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-08T23:19:30.779 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T23:19:30.779 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:30.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:30.780 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-08T23:19:30.799 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T23:19:30.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:30.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T23:19:30.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:30.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-05-08T23:19:30.918 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:30.664+0000 7f421a34b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T23:19:30.918 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:30.761+0000 7f421a34b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:19:30.919 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:30.641+0000 7f8d1a598200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:19:30.926 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:30.646+0000 7f06073be200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T23:19:30.969 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-08T23:19:30.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T23:19:31.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:30 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2340535476' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:19:31.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:30 smithi155 ceph-mon[47105]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T23:19:31.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:30 smithi155 ceph-mon[47105]: from='client.? ' entity='client.admin' 2024-05-08T23:19:31.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:30 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1120762461' entity='client.admin' 2024-05-08T23:19:31.270 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:30 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2340535476' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:19:31.270 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:30 smithi189 ceph-mon[46919]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T23:19:31.270 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:30 smithi189 ceph-mon[46919]: from='client.? ' entity='client.admin' 2024-05-08T23:19:31.270 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:30 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1120762461' entity='client.admin' 2024-05-08T23:19:31.313 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:19:31.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:30 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2340535476' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T23:19:31.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:30 smithi016 ceph-mon[42541]: mgrmap e18: x(active, since 6m), standbys: z, y 2024-05-08T23:19:31.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:30 smithi016 ceph-mon[42541]: from='client.? ' entity='client.admin' 2024-05-08T23:19:31.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:30 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1120762461' entity='client.admin' 2024-05-08T23:19:31.523 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:31.269+0000 7f8d1a598200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:19:31.530 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:31.277+0000 7f06073be200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:19:31.792 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:31.529+0000 7f421a34b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:19:31.879 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:31.522+0000 7f8d1a598200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:19:31.879 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:31.610+0000 7f8d1a598200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:19:31.879 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:31.720+0000 7f8d1a598200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:19:31.886 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:31.531+0000 7f06073be200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:19:31.886 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:31.618+0000 7f06073be200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:19:31.886 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:31.727+0000 7f06073be200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:19:31.886 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:31.886+0000 7f06073be200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:19:32.064 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:31.825+0000 7f421a34b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T23:19:32.064 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:31.924+0000 7f421a34b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T23:19:32.064 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:32.063+0000 7f421a34b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T23:19:32.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:31.878+0000 7f8d1a598200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:19:32.169 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:32.007+0000 7f8d1a598200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:19:32.169 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:32.094+0000 7f8d1a598200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:19:32.215 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:32.015+0000 7f06073be200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:19:32.215 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:32.102+0000 7f06073be200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:19:32.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:32.268+0000 7f421a34b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T23:19:32.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.374+0000 7f9975693700 1 -- 172.21.15.16:0/1300689116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9970068950 msgr2=0x7f9970068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:32.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.374+0000 7f9975693700 1 --2- 172.21.15.16:0/1300689116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9970068950 0x7f9970068db0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9964009990 tx=0x7f9964009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 -- 172.21.15.16:0/1300689116 shutdown_connections 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 --2- 172.21.15.16:0/1300689116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99700692f0 0x7f997010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 --2- 172.21.15.16:0/1300689116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970107020 0x7f9970107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 --2- 172.21.15.16:0/1300689116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9970068950 0x7f9970068db0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 -- 172.21.15.16:0/1300689116 >> 172.21.15.16:0/1300689116 conn(0x7f9970075f80 msgr2=0x7f9970076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 -- 172.21.15.16:0/1300689116 shutdown_connections 2024-05-08T23:19:32.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 -- 172.21.15.16:0/1300689116 wait complete. 2024-05-08T23:19:32.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.375+0000 7f9975693700 1 Processor -- start 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 -- start start 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970068950 0x7f99700fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99700692f0 0x7f99701004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9970107020 0x7f9970100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701a5720 con 0x7f99700692f0 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701a5890 con 0x7f9970107020 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f9975693700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701a5a50 con 0x7f9970068950 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f996e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99700692f0 0x7f99701004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f996e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99700692f0 0x7f99701004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41800/0 (socket says 172.21.15.16:41800) 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f996effd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970068950 0x7f99700fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f996e7fc700 1 -- 172.21.15.16:0/3897999854 learned_addr learned my addr 172.21.15.16:0/3897999854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.376+0000 7f996f7fe700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9970107020 0x7f9970100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:32.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f996effd700 1 -- 172.21.15.16:0/3897999854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9970107020 msgr2=0x7f9970100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f996effd700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9970107020 0x7f9970100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f996effd700 1 -- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99700692f0 msgr2=0x7f99701004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f996effd700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99700692f0 0x7f99701004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f996effd700 1 -- 172.21.15.16:0/3897999854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99640095f0 con 0x7f9970068950 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f996effd700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970068950 0x7f99700fffa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f996000e9f0 tx=0x7f996000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000c9c0 con 0x7f9970068950 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9960010430 con 0x7f9970068950 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99701a5d30 con 0x7f9970068950 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960018680 con 0x7f9970068950 2024-05-08T23:19:32.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.377+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99701a6220 con 0x7f9970068950 2024-05-08T23:19:32.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.378+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970109540 con 0x7f9970068950 2024-05-08T23:19:32.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.379+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 18) v1 ==== 135941+0+0 (secure 0 0 0) 0x7f99600105a0 con 0x7f9970068950 2024-05-08T23:19:32.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.380+0000 7f9957fff700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 0x7f99580a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:32.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.380+0000 7f996e7fc700 1 -- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 msgr2=0x7f99580a3780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/781396085 2024-05-08T23:19:32.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.380+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9960014070 con 0x7f9970068950 2024-05-08T23:19:32.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.380+0000 7f996e7fc700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 0x7f99580a3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T23:19:32.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.383+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9960096200 con 0x7f9970068950 2024-05-08T23:19:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.581+0000 7f996e7fc700 1 -- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 msgr2=0x7f99580a3780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/781396085 2024-05-08T23:19:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.581+0000 7f996e7fc700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 0x7f99580a3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T23:19:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.645+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99701026d0 con 0x7f9970068950 2024-05-08T23:19:32.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.645+0000 7f9957fff700 1 -- 172.21.15.16:0/3897999854 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9960099820 con 0x7f9970068950 2024-05-08T23:19:32.646 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 msgr2=0x7f99580a3780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 0x7f99580a3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970068950 msgr2=0x7f99700fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970068950 0x7f99700fffa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f996000e9f0 tx=0x7f996000ed00 comp rx=0 tx=0).stop 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 shutdown_connections 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9970107020 0x7f9970100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:6800/781396085,v1:172.21.15.16:6801/781396085] conn(0x7f99580a12c0 0x7f99580a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9970068950 0x7f99700fffa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 --2- 172.21.15.16:0/3897999854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99700692f0 0x7f99701004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 >> 172.21.15.16:0/3897999854 conn(0x7f9970075f80 msgr2=0x7f99700fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 shutdown_connections 2024-05-08T23:19:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:32.648+0000 7f9975693700 1 -- 172.21.15.16:0/3897999854 wait complete. 2024-05-08T23:19:32.676 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:32.420+0000 7f421a34b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T23:19:32.677 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:32.519+0000 7f421a34b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T23:19:32.859 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:32.465+0000 7f8d1a598200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:19:32.860 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:32 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3897999854' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:19:32.867 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:32 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3897999854' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:19:32.867 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:32.473+0000 7f06073be200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:19:32.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:32 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3897999854' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T23:19:33.041 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T23:19:33.042 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:19:33.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.043 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:19:33.064 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41683sec preferred_lft 41683sec 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-05-08T23:19:33.065 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T23:19:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-05-08T23:19:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-05-08T23:19:33.066 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T23:19:33.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.066 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:19:33.066 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:19:33.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-05-08T23:19:33.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-05-08T23:19:33.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-05-08T23:19:33.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T23:19:33.067 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:19:33.151 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:32.859+0000 7f8d1a598200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:19:33.151 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:33.058+0000 7f8d1a598200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:19:33.157 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:32.868+0000 7f06073be200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:19:33.157 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:33.065+0000 7f06073be200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:19:33.196 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:32.962+0000 7f421a34b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T23:19:33.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.201 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T23:19:33.237 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-05-08T23:19:33.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.238 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-05-08T23:19:33.300 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-05-08T23:19:33.300 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T23:19:33.300 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T23:19:33.300 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T23:19:33.300 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T23:19:33.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.301 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:19:33.301 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:19:33.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T23:19:33.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T23:19:33.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T23:19:33.301 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:19:33.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.419 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-08T23:19:33.420 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:33.150+0000 7f8d1a598200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:19:33.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.439 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-05-08T23:19:33.464 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:33.158+0000 7f06073be200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:19:33.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.498 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:19:33.498 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:19:33.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:19:33.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T23:19:33.498 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:19:33.583 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:33.445+0000 7f421a34b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T23:19:33.603 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T23:19:33.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T23:19:33.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T23:19:33.604 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:19:33.837 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:33.672+0000 7f421a34b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T23:19:33.837 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:33.788+0000 7f421a34b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T23:19:33.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:33.839 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-05-08T23:19:33.839 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-05-08T23:19:33.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-05-08T23:19:33.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T23:19:33.839 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-05-08T23:19:33.946 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-08T23:19:33.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:19:33.947 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:33.965 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T23:19:33.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T23:19:33.965 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:34.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:19:34.095 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:19:34.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:19:34.165 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:19:34.166 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:19:34.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:34.167 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:19:34.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:33.917+0000 7f8d1a598200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:19:34.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:34.013+0000 7f8d1a598200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:19:34.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:34.164+0000 7f8d1a598200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:19:34.216 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:33.930+0000 7f06073be200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:19:34.216 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:34.026+0000 7f06073be200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:19:34.216 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:34.178+0000 7f06073be200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:19:34.226 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T23:19:34.226 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/9ee67809-27ca-4adc-9b10-dce1a6e369ab --client_fs=cephfs) 2024-05-08T23:19:34.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:19:34.301 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:19:34.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:19:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:19:34.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2024-05-08T23:19:34.376+0000 7efd1bd9c5c0 -1 init, newargv = 0x55e9e38d3a00 newargc=15 2024-05-08T23:19:34.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111907]: starting ceph client 2024-05-08T23:19:34.377 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:19:34.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:34.378 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:19:34.961 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:34.701+0000 7f421a34b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T23:19:34.961 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:34.813+0000 7f421a34b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T23:19:35.169 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:34.799+0000 7f8d1a598200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:19:35.169 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:34.982+0000 7f8d1a598200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:19:35.194 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:34.816+0000 7f06073be200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:19:35.194 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:35 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:34.999+0000 7f06073be200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:19:35.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:34.985+0000 7f421a34b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T23:19:35.465 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:35 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:35.194+0000 7f06073be200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:19:35.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:19:35.572 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:19:35.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:19:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:19:35.643 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:19:35.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:35.644 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:19:35.668 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:35.176+0000 7f8d1a598200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:19:35.863 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:35.720+0000 7f421a34b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T23:19:36.363 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:35.930+0000 7f421a34b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T23:19:36.364 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:36.156+0000 7f421a34b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T23:19:36.399 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:36.115+0000 7f8d1a598200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:19:36.421 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:36.134+0000 7f06073be200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:19:36.421 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:36.421+0000 7f06073be200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:19:36.668 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:36.399+0000 7f8d1a598200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:19:36.668 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:36.486+0000 7f8d1a598200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:19:36.703 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:36.510+0000 7f06073be200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:19:36.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:19:36.837 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:19:36.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:19:36.908 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:19:36.909 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:19:36.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:36.910 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:19:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y[48591]: 2024-05-08T23:19:36.704+0000 7f06073be200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: Standby manager daemon z restarted 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: Standby manager daemon z started 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: Standby manager daemon y restarted 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: Standby manager daemon y started 2024-05-08T23:19:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:19:36.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:19:36.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:19:36.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:36 smithi155 ceph-mon[47105]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:19:37.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: Standby manager daemon z restarted 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: Standby manager daemon z started 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: Standby manager daemon y restarted 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: Standby manager daemon y started 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:19:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:36 smithi016 ceph-mon[42541]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:19:37.168 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:19:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z[48780]: 2024-05-08T23:19:36.678+0000 7f8d1a598200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: Standby manager daemon z restarted 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: Standby manager daemon z started 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.189:0/3791676221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: Standby manager daemon y restarted 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: Standby manager daemon y started 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T23:19:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T23:19:37.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:36 smithi189 ceph-mon[46919]: from='mgr.? 172.21.15.155:0/3154364707' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T23:19:37.573 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:37.237+0000 7f421a34b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T23:19:37.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:37 smithi016 ceph-mon[42541]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-08T23:19:37.864 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:37.574+0000 7f421a34b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T23:19:37.864 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:37.679+0000 7f421a34b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T23:19:37.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111907]: starting fuse 2024-05-08T23:19:38.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-05-08T23:19:38.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T23:19:38.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T23:19:38.119 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:37 smithi155 ceph-mon[47105]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-08T23:19:38.120 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:37 smithi189 ceph-mon[46919]: mgrmap e19: x(active, since 6m), standbys: y, z 2024-05-08T23:19:38.186 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:37.903+0000 7f421a34b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T23:19:38.193 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T23:19:38.196 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T23:19:38.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:38.196 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-05-08T23:19:38.262 INFO:teuthology.orchestra.run.smithi016.stdout:124 2024-05-08T23:19:38.262 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-05-08T23:19:38.263 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> 2024-05-08T23:19:38.264 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2024-05-08T23:19:38.265 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-05-08T23:19:38.400 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.111907.asok 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.247+0000 7f417c61f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.247+0000 7f417c61f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.247+0000 7f417c61f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.247+0000 7f417c61f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.247+0000 7f417c61f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.250+0000 7f4178e18700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.580 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.250+0000 7f4178e18700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.581 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.250+0000 7f4178e18700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.581 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.250+0000 7f4178e18700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.581 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:19:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:19:38.250+0000 7f4178e18700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:19:38.582 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.111907.asok 2024-05-08T23:19:38.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:38.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.111907.asok status 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "111907", 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/volumes/qa/sv_0/9ee67809-27ca-4adc-9b10-dce1a6e369ab" 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:19:38.781 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 15495, 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 15495 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3569224694 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:19:38.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3569224694 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.15495 192.168.0.1:0/3569224694", 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3569224694", 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 5, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 24, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 83, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-05-08T23:19:38.783 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:38.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:38.793 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-05-08T23:19:38.794 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:38.815 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-05-08T23:19:38.815 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:38.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:19:38.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:38.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: Active manager daemon x restarted 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: Activating manager daemon x 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: mgrmap e20: x(active, starting, since 0.0120874s), standbys: y, z 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:19:38.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:19:38.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: Manager daemon x is now available 2024-05-08T23:19:38.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:38.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:19:38.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:19:38.889 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T23:19:38.899 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:19:38.899 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:19:38.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 -- 172.21.15.16:0/815423110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4104390 msgr2=0x7f20f4106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 --2- 172.21.15.16:0/815423110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4104390 0x7f20f4106780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009990 tx=0x7f20e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 -- 172.21.15.16:0/815423110 shutdown_connections 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 --2- 172.21.15.16:0/815423110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4106cc0 0x7f20f41090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 --2- 172.21.15.16:0/815423110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f4101a60 0x7f20f4103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 --2- 172.21.15.16:0/815423110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4104390 0x7f20f4106780 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 -- 172.21.15.16:0/815423110 >> 172.21.15.16:0/815423110 conn(0x7f20f40fb3b0 msgr2=0x7f20f40fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:39.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 -- 172.21.15.16:0/815423110 shutdown_connections 2024-05-08T23:19:39.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 -- 172.21.15.16:0/815423110 wait complete. 2024-05-08T23:19:39.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.084+0000 7f20fa43b700 1 Processor -- start 2024-05-08T23:19:39.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 -- start start 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 0x7f20f4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4104390 0x7f20f4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f4106cc0 0x7f20f41a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f41a0a70 con 0x7f20f4101a60 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f41a0bb0 con 0x7f20f4106cc0 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20fa43b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f41a0d40 con 0x7f20f4104390 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 0x7f20f4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 0x7f20f4198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60168/0 (socket says 172.21.15.16:60168) 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f3fff700 1 -- 172.21.15.16:0/383006953 learned_addr learned my addr 172.21.15.16:0/383006953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f37fe700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4104390 0x7f20f4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f89d8700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f4106cc0 0x7f20f41a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f37fe700 1 -- 172.21.15.16:0/383006953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f4106cc0 msgr2=0x7f20f41a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:39.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f37fe700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f4106cc0 0x7f20f41a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f37fe700 1 -- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 msgr2=0x7f20f4198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f37fe700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 0x7f20f4198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f37fe700 1 -- 172.21.15.16:0/383006953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80095f0 con 0x7f20f4104390 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.085+0000 7f20f3fff700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 0x7f20f4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.086+0000 7f20f37fe700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4104390 0x7f20f4199480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f20e8003f60 tx=0x7f20e8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.086+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e801d070 con 0x7f20f4104390 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.086+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20e800bdb0 con 0x7f20f4104390 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.086+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f41a0fc0 con 0x7f20f4104390 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.086+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8022ca0 con 0x7f20f4104390 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.086+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f41a1380 con 0x7f20f4104390 2024-05-08T23:19:39.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.087+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f4108840 con 0x7f20f4104390 2024-05-08T23:19:39.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.088+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7f20e8022470 con 0x7f20f4104390 2024-05-08T23:19:39.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.089+0000 7f20f17fa700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f20dc0a0f20 0x7f20dc0a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:39.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.089+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f20e80ce280 con 0x7f20f4104390 2024-05-08T23:19:39.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.089+0000 7f20f3fff700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f20dc0a0f20 0x7f20dc0a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:39.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.090+0000 7f20f3fff700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f20dc0a0f20 0x7f20dc0a33e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20e4011550 tx=0x7f20e4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:39.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.092+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f20e8097920 con 0x7f20f4104390 2024-05-08T23:19:39.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: Active manager daemon x restarted 2024-05-08T23:19:39.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: Activating manager daemon x 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: mgrmap e20: x(active, starting, since 0.0120874s), standbys: y, z 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:19:39.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:19:39.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: Manager daemon x is now available 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:19:39.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:19:39.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: Active manager daemon x restarted 2024-05-08T23:19:39.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: Activating manager daemon x 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: mgrmap e20: x(active, starting, since 0.0120874s), standbys: y, z 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T23:19:39.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T23:19:39.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: Manager daemon x is now available 2024-05-08T23:19:39.218 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:39.219 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T23:19:39.219 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T23:19:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.379+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f20f410c5d0 con 0x7f20f4104390 2024-05-08T23:19:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.936+0000 7f20f17fa700 1 -- 172.21.15.16:0/383006953 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) v1 ==== 117+0+0 (secure 0 0 0) 0x7f20e809af40 con 0x7f20f4104390 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f20dc0a0f20 msgr2=0x7f20dc0a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f20dc0a0f20 0x7f20dc0a33e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20e4011550 tx=0x7f20e4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4104390 msgr2=0x7f20f4199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4104390 0x7f20f4199480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f20e8003f60 tx=0x7f20e8004730 comp rx=0 tx=0).stop 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 shutdown_connections 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f20dc0a0f20 0x7f20dc0a33e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20f4106cc0 0x7f20f41a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f4101a60 0x7f20f4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 --2- 172.21.15.16:0/383006953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20f4104390 0x7f20f4199480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 >> 172.21.15.16:0/383006953 conn(0x7f20f40fb3b0 msgr2=0x7f20f4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 shutdown_connections 2024-05-08T23:19:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:39.939+0000 7f20fa43b700 1 -- 172.21.15.16:0/383006953 wait complete. 2024-05-08T23:19:39.954 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T23:19:39.965 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T23:19:39.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 -- 172.21.15.16:0/3555814472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 msgr2=0x7fe7f4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/3555814472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4102440 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009990 tx=0x7fe7e8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 -- 172.21.15.16:0/3555814472 shutdown_connections 2024-05-08T23:19:40.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/3555814472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7f4102980 0x7fe7f41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/3555814472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 0x7fe7f4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/3555814472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4102440 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 -- 172.21.15.16:0/3555814472 >> 172.21.15.16:0/3555814472 conn(0x7fe7f40fc400 msgr2=0x7fe7f40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:40.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 -- 172.21.15.16:0/3555814472 shutdown_connections 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 -- 172.21.15.16:0/3555814472 wait complete. 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.141+0000 7fe7f98ec700 1 Processor -- start 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 -- start start 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 0x7fe7f4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7f4102980 0x7fe7f40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4074070 con 0x7fe7f4101fc0 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40741e0 con 0x7fe7f4100e70 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f98ec700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f41a0d70 con 0x7fe7f4102980 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60194/0 (socket says 172.21.15.16:60194) 2024-05-08T23:19:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 -- 172.21.15.16:0/4149188965 learned_addr learned my addr 172.21.15.16:0/4149188965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f2ffd700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 0x7fe7f4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f37fe700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7f4102980 0x7fe7f40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 -- 172.21.15.16:0/4149188965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7f4102980 msgr2=0x7fe7f40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7f4102980 0x7fe7f40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 -- 172.21.15.16:0/4149188965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 msgr2=0x7fe7f4074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 0x7fe7f4074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f27fc700 1 -- 172.21.15.16:0/4149188965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80095f0 con 0x7fe7f4101fc0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.142+0000 7fe7f2ffd700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 0x7fe7f4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.143+0000 7fe7f27fc700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4073390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8006010 tx=0x7fe7e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.143+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e801d070 con 0x7fe7f4101fc0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.143+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7e800bd50 con 0x7fe7f4101fc0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.143+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f41a0fa0 con 0x7fe7f4101fc0 2024-05-08T23:19:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.143+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e800fd50 con 0x7fe7f4101fc0 2024-05-08T23:19:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.143+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41a1310 con 0x7fe7f4101fc0 2024-05-08T23:19:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.144+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f4108280 con 0x7fe7f4101fc0 2024-05-08T23:19:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.145+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fe7e800b8c0 con 0x7fe7f4101fc0 2024-05-08T23:19:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.146+0000 7fe7f88ea700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe7dc0a0f20 0x7fe7dc0a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.146+0000 7fe7f2ffd700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe7dc0a0f20 0x7fe7dc0a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.146+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fe7e80cdc30 con 0x7fe7f4101fc0 2024-05-08T23:19:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.146+0000 7fe7f2ffd700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe7dc0a0f20 0x7fe7dc0a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4011490 tx=0x7fe7e4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.150+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe7e80972d0 con 0x7fe7f4101fc0 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: mgrmap e21: x(active, since 1.01846s), standbys: y, z 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/383006953' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: mgrmap e21: x(active, since 1.01846s), standbys: y, z 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/383006953' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.384+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7f4068060 con 0x7fe7f4101fc0 2024-05-08T23:19:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.386+0000 7fe7f88ea700 1 -- 172.21.15.16:0/4149188965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24906 (secure 0 0 0) 0x7fe7e809a8f0 con 0x7fe7f4101fc0 2024-05-08T23:19:40.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:40.386 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:19:37.917761+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe7dc0a0f20 msgr2=0x7fe7dc0a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe7dc0a0f20 0x7fe7dc0a33e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4011490 tx=0x7fe7e4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 msgr2=0x7fe7f4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4073390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8006010 tx=0x7fe7e8004730 comp rx=0 tx=0).stop 2024-05-08T23:19:40.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 shutdown_connections 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe7dc0a0f20 0x7fe7dc0a33e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7f4102980 0x7fe7f40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7f4100e70 0x7fe7f4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.388+0000 7fe7f98ec700 1 --2- 172.21.15.16:0/4149188965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7f4101fc0 0x7fe7f4073390 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.389+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 >> 172.21.15.16:0/4149188965 conn(0x7fe7f40fc400 msgr2=0x7fe7f41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.389+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 shutdown_connections 2024-05-08T23:19:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.389+0000 7fe7f98ec700 1 -- 172.21.15.16:0/4149188965 wait complete. 2024-05-08T23:19:40.410 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:40.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:40.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: mgrmap e21: x(active, since 1.01846s), standbys: y, z 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/383006953' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 -- 172.21.15.16:0/3040235765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4102b10 msgr2=0x7fada41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 --2- 172.21.15.16:0/3040235765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4102b10 0x7fada41093c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009fc0 tx=0x7fad9c00b330 comp rx=0 tx=0).stop 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 -- 172.21.15.16:0/3040235765 shutdown_connections 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 --2- 172.21.15.16:0/3040235765 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4102b10 0x7fada41093c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 --2- 172.21.15.16:0/3040235765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4100f80 0x7fada41013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 --2- 172.21.15.16:0/3040235765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102170 0x7fada41025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 -- 172.21.15.16:0/3040235765 >> 172.21.15.16:0/3040235765 conn(0x7fada40fc4f0 msgr2=0x7fada40fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 -- 172.21.15.16:0/3040235765 shutdown_connections 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.559+0000 7fadab595700 1 -- 172.21.15.16:0/3040235765 wait complete. 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 Processor -- start 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 -- start start 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 0x7fada4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4102170 0x7fada4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102b10 0x7fada40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4074070 con 0x7fada4102b10 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40741e0 con 0x7fada4100f80 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fadab595700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada41a0f90 con 0x7fada4102170 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fada9331700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 0x7fada4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fada8b30700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4102170 0x7fada4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fada9331700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 0x7fada4074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46776/0 (socket says 172.21.15.16:46776) 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fada9331700 1 -- 172.21.15.16:0/3065765898 learned_addr learned my addr 172.21.15.16:0/3065765898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fada9b32700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102b10 0x7fada40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.560+0000 7fada9331700 1 -- 172.21.15.16:0/3065765898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4102170 msgr2=0x7fada4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada9331700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4102170 0x7fada4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada9331700 1 -- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102b10 msgr2=0x7fada40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada9331700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102b10 0x7fada40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada9331700 1 -- 172.21.15.16:0/3065765898 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940095f0 con 0x7fada4100f80 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada9b32700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102b10 0x7fada40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada8b30700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4102170 0x7fada4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fada9331700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 0x7fada4074d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fada000e970 tx=0x7fada000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0009790 con 0x7fada4100f80 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fada000c780 con 0x7fada4100f80 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c009c40 con 0x7fada4100f80 2024-05-08T23:19:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0003b40 con 0x7fada4100f80 2024-05-08T23:19:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.561+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada41a15a0 con 0x7fada4100f80 2024-05-08T23:19:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.562+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada4108440 con 0x7fada4100f80 2024-05-08T23:19:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.563+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 21) v1 ==== 135685+0+0 (secure 0 0 0) 0x7fada0044020 con 0x7fada4100f80 2024-05-08T23:19:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.564+0000 7fad9a7fc700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fad900a0f20 0x7fad900a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.564+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fada0014070 con 0x7fada4100f80 2024-05-08T23:19:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.564+0000 7fada8b30700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fad900a0f20 0x7fad900a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.565+0000 7fada8b30700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fad900a0f20 0x7fad900a33e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad94005fd0 tx=0x7fad94009310 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.568+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fada00963c0 con 0x7fada4100f80 2024-05-08T23:19:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.796+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fada410c8d0 con 0x7fada4100f80 2024-05-08T23:19:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.796+0000 7fad9a7fc700 1 -- 172.21.15.16:0/3065765898 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fada00999e0 con 0x7fada4100f80 2024-05-08T23:19:40.796 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fad900a0f20 msgr2=0x7fad900a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fad900a0f20 0x7fad900a33e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad94005fd0 tx=0x7fad94009310 comp rx=0 tx=0).stop 2024-05-08T23:19:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 msgr2=0x7fada4074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 0x7fada4074d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fada000e970 tx=0x7fada000ec80 comp rx=0 tx=0).stop 2024-05-08T23:19:40.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 shutdown_connections 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fad900a0f20 0x7fad900a33e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fada4102b10 0x7fada40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fada4100f80 0x7fada4074d00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 --2- 172.21.15.16:0/3065765898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada4102170 0x7fada4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 >> 172.21.15.16:0/3065765898 conn(0x7fada40fc4f0 msgr2=0x7fada4105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 shutdown_connections 2024-05-08T23:19:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.799+0000 7fadab595700 1 -- 172.21.15.16:0/3065765898 wait complete. 2024-05-08T23:19:40.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 -- 172.21.15.16:0/3194423694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 msgr2=0x7feb88101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/3194423694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 0x7feb88101360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb84009990 tx=0x7feb84009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 -- 172.21.15.16:0/3194423694 shutdown_connections 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/3194423694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb88102ab0 0x7feb881093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/3194423694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 0x7feb88101360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/3194423694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb881020f0 0x7feb88102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 -- 172.21.15.16:0/3194423694 >> 172.21.15.16:0/3194423694 conn(0x7feb880fc4e0 msgr2=0x7feb880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 -- 172.21.15.16:0/3194423694 shutdown_connections 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.970+0000 7feb8f6c1700 1 -- 172.21.15.16:0/3194423694 wait complete. 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 Processor -- start 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 -- start start 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 0x7feb88196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb881020f0 0x7feb881973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 0x7feb8819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8819e960 con 0x7feb881020f0 2024-05-08T23:19:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8819eaa0 con 0x7feb88102ab0 2024-05-08T23:19:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8f6c1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8819ec30 con 0x7feb88100f20 2024-05-08T23:19:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8dc5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 0x7feb8819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8cc5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb881020f0 0x7feb881973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8dc5e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 0x7feb8819e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:46802/0 (socket says 172.21.15.16:46802) 2024-05-08T23:19:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8dc5e700 1 -- 172.21.15.16:0/570161772 learned_addr learned my addr 172.21.15.16:0/570161772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.971+0000 7feb8d45d700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 0x7feb88196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8dc5e700 1 -- 172.21.15.16:0/570161772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 msgr2=0x7feb88196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8dc5e700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 0x7feb88196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8dc5e700 1 -- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb881020f0 msgr2=0x7feb881973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8dc5e700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb881020f0 0x7feb881973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8dc5e700 1 -- 172.21.15.16:0/570161772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb78009520 con 0x7feb88102ab0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8dc5e700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 0x7feb8819e390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb8000ef30 tx=0x7feb8000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80018070 con 0x7feb88102ab0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb8000f040 con 0x7feb88102ab0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8000cbc0 con 0x7feb88102ab0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb840095f0 con 0x7feb88102ab0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.972+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb88073500 con 0x7feb88102ab0 2024-05-08T23:19:40.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.973+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb881084a0 con 0x7feb88102ab0 2024-05-08T23:19:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.974+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7feb80003a00 con 0x7feb88102ab0 2024-05-08T23:19:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.975+0000 7feb7e7fc700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7feb740a1310 0x7feb740a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.976+0000 7feb8d45d700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7feb740a1310 0x7feb740a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:40.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.976+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7feb800ccb50 con 0x7feb88102ab0 2024-05-08T23:19:40.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.976+0000 7feb8d45d700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7feb740a1310 0x7feb740a37d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb84005910 tx=0x7feb8400b610 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:40.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:40.978+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb80095cc0 con 0x7feb88102ab0 2024-05-08T23:19:41.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: [08/May/2024:23:19:39] ENGINE Bus STARTING 2024-05-08T23:19:41.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: [08/May/2024:23:19:39] ENGINE Serving on https://172.21.15.16:7150 2024-05-08T23:19:41.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: [08/May/2024:23:19:39] ENGINE Serving on http://172.21.15.16:8765 2024-05-08T23:19:41.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: [08/May/2024:23:19:39] ENGINE Bus STARTED 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4149188965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:41.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:41.117 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:40 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3065765898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:41.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.208+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb8810c620 con 0x7feb88102ab0 2024-05-08T23:19:41.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.209+0000 7feb7e7fc700 1 -- 172.21.15.16:0/570161772 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb80003cb0 con 0x7feb88102ab0 2024-05-08T23:19:41.209 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7feb740a1310 msgr2=0x7feb740a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7feb740a1310 0x7feb740a37d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb84005910 tx=0x7feb8400b610 comp rx=0 tx=0).stop 2024-05-08T23:19:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 msgr2=0x7feb8819e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 0x7feb8819e390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb8000ef30 tx=0x7feb8000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 shutdown_connections 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7feb740a1310 0x7feb740a37d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb88102ab0 0x7feb8819e390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb88100f20 0x7feb88196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 --2- 172.21.15.16:0/570161772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb881020f0 0x7feb881973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 >> 172.21.15.16:0/570161772 conn(0x7feb880fc4e0 msgr2=0x7feb881055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 shutdown_connections 2024-05-08T23:19:41.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.211+0000 7feb8f6c1700 1 -- 172.21.15.16:0/570161772 wait complete. 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: [08/May/2024:23:19:39] ENGINE Bus STARTING 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: [08/May/2024:23:19:39] ENGINE Serving on https://172.21.15.16:7150 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: [08/May/2024:23:19:39] ENGINE Serving on http://172.21.15.16:8765 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: [08/May/2024:23:19:39] ENGINE Bus STARTED 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4149188965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:41.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:41.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:40 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3065765898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:41.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 -- 172.21.15.16:0/205800548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100a10 msgr2=0x7fd994104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 --2- 172.21.15.16:0/205800548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100a10 0x7fd994104a30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 -- 172.21.15.16:0/205800548 shutdown_connections 2024-05-08T23:19:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 --2- 172.21.15.16:0/205800548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd994105160 0x7fd99410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 --2- 172.21.15.16:0/205800548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd994100060 0x7fd994100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 --2- 172.21.15.16:0/205800548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994100a10 0x7fd994104a30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 -- 172.21.15.16:0/205800548 >> 172.21.15.16:0/205800548 conn(0x7fd994076f40 msgr2=0x7fd994077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 -- 172.21.15.16:0/205800548 shutdown_connections 2024-05-08T23:19:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 -- 172.21.15.16:0/205800548 wait complete. 2024-05-08T23:19:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 Processor -- start 2024-05-08T23:19:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.374+0000 7fd99a36a700 1 -- start start 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd99a36a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd994100060 0x7fd994197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd99a36a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 0x7fd9941978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd99a36a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd99419bc40 0x7fd99419c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd99a36a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99419c6c0 con 0x7fd994105160 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd99a36a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99419c830 con 0x7fd99419bc40 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd99a36a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99419c9f0 con 0x7fd994100060 2024-05-08T23:19:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 0x7fd9941978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 0x7fd9941978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60256/0 (socket says 172.21.15.16:60256) 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 -- 172.21.15.16:0/1195257644 learned_addr learned my addr 172.21.15.16:0/1195257644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd993fff700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd994100060 0x7fd994197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd998907700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd99419bc40 0x7fd99419c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 -- 172.21.15.16:0/1195257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd994100060 msgr2=0x7fd994197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd994100060 0x7fd994197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 -- 172.21.15.16:0/1195257644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd99419bc40 msgr2=0x7fd99419c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd99419bc40 0x7fd99419c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 -- 172.21.15.16:0/1195257644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880095f0 con 0x7fd994105160 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.375+0000 7fd9937fe700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 0x7fd9941978a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd988005910 tx=0x7fd988004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.376+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98801d070 con 0x7fd994105160 2024-05-08T23:19:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.376+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd98800bea0 con 0x7fd994105160 2024-05-08T23:19:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.376+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9941a3ef0 con 0x7fd994105160 2024-05-08T23:19:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.376+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988022df0 con 0x7fd994105160 2024-05-08T23:19:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.376+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9941a42b0 con 0x7fd994105160 2024-05-08T23:19:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.377+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd994109440 con 0x7fd994105160 2024-05-08T23:19:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.378+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7fd988022470 con 0x7fd994105160 2024-05-08T23:19:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.378+0000 7fd9917fa700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd97c0a1290 0x7fd97c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.379+0000 7fd993fff700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd97c0a1290 0x7fd97c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.379+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fd9880cea40 con 0x7fd994105160 2024-05-08T23:19:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.379+0000 7fd993fff700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd97c0a1290 0x7fd97c0a3750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd984009c60 tx=0x7fd984009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.382+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd988097bb0 con 0x7fd994105160 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: [08/May/2024:23:19:39] ENGINE Bus STARTING 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: [08/May/2024:23:19:39] ENGINE Serving on https://172.21.15.16:7150 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: [08/May/2024:23:19:39] ENGINE Serving on http://172.21.15.16:8765 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: [08/May/2024:23:19:39] ENGINE Bus STARTED 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4149188965' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:19:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:41.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:40 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3065765898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:41.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.692+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd994198660 con 0x7fd994105160 2024-05-08T23:19:41.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.692+0000 7fd9917fa700 1 -- 172.21.15.16:0/1195257644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd98809b1d0 con 0x7fd994105160 2024-05-08T23:19:41.692 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.694+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd97c0a1290 msgr2=0x7fd97c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.694+0000 7fd99a36a700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd97c0a1290 0x7fd97c0a3750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd984009c60 tx=0x7fd984009260 comp rx=0 tx=0).stop 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.694+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 msgr2=0x7fd9941978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.694+0000 7fd99a36a700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 0x7fd9941978a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd988005910 tx=0x7fd988004730 comp rx=0 tx=0).stop 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 shutdown_connections 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd97c0a1290 0x7fd97c0a3750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd99419bc40 0x7fd99419c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd994100060 0x7fd994197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 --2- 172.21.15.16:0/1195257644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd994105160 0x7fd9941978a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 >> 172.21.15.16:0/1195257644 conn(0x7fd994076f40 msgr2=0x7fd9940684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 shutdown_connections 2024-05-08T23:19:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.695+0000 7fd99a36a700 1 -- 172.21.15.16:0/1195257644 wait complete. 2024-05-08T23:19:41.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 -- 172.21.15.16:0/3090350278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c105160 msgr2=0x7fcf6c10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 --2- 172.21.15.16:0/3090350278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c105160 0x7fcf6c10cce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009f90 tx=0x7fcf6400b330 comp rx=0 tx=0).stop 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 -- 172.21.15.16:0/3090350278 shutdown_connections 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 --2- 172.21.15.16:0/3090350278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c105160 0x7fcf6c10cce0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 --2- 172.21.15.16:0/3090350278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf6c100060 0x7fcf6c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 --2- 172.21.15.16:0/3090350278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6c100a10 0x7fcf6c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 -- 172.21.15.16:0/3090350278 >> 172.21.15.16:0/3090350278 conn(0x7fcf6c076f40 msgr2=0x7fcf6c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 -- 172.21.15.16:0/3090350278 shutdown_connections 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.853+0000 7fcf724e8700 1 -- 172.21.15.16:0/3090350278 wait complete. 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.854+0000 7fcf724e8700 1 Processor -- start 2024-05-08T23:19:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.854+0000 7fcf724e8700 1 -- start start 2024-05-08T23:19:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 0x7fcf6c101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6c105160 0x7fcf6c102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf6c102550 0x7fcf6c1a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c1a5bf0 con 0x7fcf6c105160 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c1a5d60 con 0x7fcf6c102550 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c1a5f20 con 0x7fcf6c100060 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 0x7fcf6c101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6c105160 0x7fcf6c102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 0x7fcf6c101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:59042/0 (socket says 172.21.15.16:59042) 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 -- 172.21.15.16:0/412406971 learned_addr learned my addr 172.21.15.16:0/412406971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf70a85700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf6c102550 0x7fcf6c1a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 -- 172.21.15.16:0/412406971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf6c102550 msgr2=0x7fcf6c1a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf6c102550 0x7fcf6c1a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 -- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6c105160 msgr2=0x7fcf6c102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6c105160 0x7fcf6c102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:41.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 -- 172.21.15.16:0/412406971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64009c40 con 0x7fcf6c100060 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf6bfff700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 0x7fcf6c101ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e9a0 tx=0x7fcf5c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c009790 con 0x7fcf6c100060 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf5c00cbd0 con 0x7fcf6c100060 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.855+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c1a6200 con 0x7fcf6c100060 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.856+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00eee0 con 0x7fcf6c100060 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.856+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6c1a6710 con 0x7fcf6c100060 2024-05-08T23:19:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.857+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6c109440 con 0x7fcf6c100060 2024-05-08T23:19:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.858+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7fcf5c00cd40 con 0x7fcf6c100060 2024-05-08T23:19:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.859+0000 7fcf697fa700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcf540a1290 0x7fcf540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.859+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fcf5c014070 con 0x7fcf6c100060 2024-05-08T23:19:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.859+0000 7fcf6b7fe700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcf540a1290 0x7fcf540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.860+0000 7fcf6b7fe700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcf540a1290 0x7fcf540a3750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c103200 tx=0x7fcf60009310 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:41.862+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcf5c096350 con 0x7fcf6c100060 2024-05-08T23:19:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.087+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcf6c102fb0 con 0x7fcf6c100060 2024-05-08T23:19:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.088+0000 7fcf697fa700 1 -- 172.21.15.16:0/412406971 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcf5c099970 con 0x7fcf6c100060 2024-05-08T23:19:42.087 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcf540a1290 msgr2=0x7fcf540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcf540a1290 0x7fcf540a3750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c103200 tx=0x7fcf60009310 comp rx=0 tx=0).stop 2024-05-08T23:19:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 msgr2=0x7fcf6c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 0x7fcf6c101ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00e9a0 tx=0x7fcf5c00ed60 comp rx=0 tx=0).stop 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 shutdown_connections 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcf540a1290 0x7fcf540a3750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf6c102550 0x7fcf6c1a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf6c100060 0x7fcf6c101ad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 --2- 172.21.15.16:0/412406971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf6c105160 0x7fcf6c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 >> 172.21.15.16:0/412406971 conn(0x7fcf6c076f40 msgr2=0x7fcf6c0683f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 shutdown_connections 2024-05-08T23:19:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.090+0000 7fcf724e8700 1 -- 172.21.15.16:0/412406971 wait complete. 2024-05-08T23:19:42.102 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-05-08T23:19:42.102 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi189 to 182.5M 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Unable to set osd_memory_target on smithi189 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Unable to set osd_memory_target on smithi016 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Updating smithi155:/etc/ceph/ceph.conf 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Updating smithi189:/etc/ceph/ceph.conf 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/570161772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:42.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1195257644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:42.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:41 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-05-08T23:19:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:19:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi189 to 182.5M 2024-05-08T23:19:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Unable to set osd_memory_target on smithi189 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T23:19:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-05-08T23:19:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Unable to set osd_memory_target on smithi016 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Updating smithi155:/etc/ceph/ceph.conf 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Updating smithi189:/etc/ceph/ceph.conf 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/570161772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1195257644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:42.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:41 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 -- 172.21.15.16:0/2847459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c1050a0 msgr2=0x7f696c10cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 --2- 172.21.15.16:0/2847459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c1050a0 0x7f696c10cc20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f696400b0a0 tx=0x7f696400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 -- 172.21.15.16:0/2847459404 shutdown_connections 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 --2- 172.21.15.16:0/2847459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c1050a0 0x7f696c10cc20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 --2- 172.21.15.16:0/2847459404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f696c0fff80 0x7f696c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 --2- 172.21.15.16:0/2847459404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c100930 0x7f696c104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 -- 172.21.15.16:0/2847459404 >> 172.21.15.16:0/2847459404 conn(0x7f696c0fb840 msgr2=0x7f696c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 -- 172.21.15.16:0/2847459404 shutdown_connections 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.245+0000 7f697162b700 1 -- 172.21.15.16:0/2847459404 wait complete. 2024-05-08T23:19:42.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 Processor -- start 2024-05-08T23:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 -- start start 2024-05-08T23:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c0fff80 0x7f696c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f696c1050a0 0x7f696c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c19bc10 0x7f696c19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c19c690 con 0x7f696c0fff80 2024-05-08T23:19:42.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c19c800 con 0x7f696c1050a0 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f697162b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c19c9c0 con 0x7f696c19bc10 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.246+0000 7f696affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c0fff80 0x7f696c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c0fff80 0x7f696c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60290/0 (socket says 172.21.15.16:60290) 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696affd700 1 -- 172.21.15.16:0/2330102955 learned_addr learned my addr 172.21.15.16:0/2330102955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c19bc10 0x7f696c19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 -- 172.21.15.16:0/2330102955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f696c1050a0 msgr2=0x7f696c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f696c1050a0 0x7f696c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 -- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c0fff80 msgr2=0x7f696c197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c0fff80 0x7f696c197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 -- 172.21.15.16:0/2330102955 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964009d00 con 0x7f696c19bc10 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f696b7fe700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c19bc10 0x7f696c19c090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f696400b780 tx=0x7f6964009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.247+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696400e040 con 0x7f696c19bc10 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.248+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6964007e20 con 0x7f696c19bc10 2024-05-08T23:19:42.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.248+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c1a3ec0 con 0x7f696c19bc10 2024-05-08T23:19:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.248+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964005040 con 0x7f696c19bc10 2024-05-08T23:19:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.248+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c1a4340 con 0x7f696c19bc10 2024-05-08T23:19:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.249+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c109380 con 0x7f696c19bc10 2024-05-08T23:19:42.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.250+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f696401b450 con 0x7f696c19bc10 2024-05-08T23:19:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.250+0000 7f6968ff9700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f69580a1290 0x7f69580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.250+0000 7f696affd700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f69580a1290 0x7f69580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.250+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f69640ce470 con 0x7f696c19bc10 2024-05-08T23:19:42.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.251+0000 7f696affd700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f69580a1290 0x7f69580a3750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f695c005950 tx=0x7f695c00b500 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:42.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.254+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6964097640 con 0x7f696c19bc10 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi189 to 182.5M 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Unable to set osd_memory_target on smithi189 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Unable to set osd_memory_target on smithi016 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Updating smithi016:/etc/ceph/ceph.conf 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Updating smithi155:/etc/ceph/ceph.conf 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Updating smithi189:/etc/ceph/ceph.conf 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: mgrmap e22: x(active, since 3s), standbys: y, z 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/570161772' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1195257644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:19:42.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:19:42.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:41 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:19:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.472+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f696c198630 con 0x7f696c19bc10 2024-05-08T23:19:42.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.474+0000 7f6968ff9700 1 -- 172.21.15.16:0/2330102955 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24906 (secure 0 0 0) 0x7f696409ac60 con 0x7f696c19bc10 2024-05-08T23:19:42.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:42.474 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:19:37.917761+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.476+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f69580a1290 msgr2=0x7f69580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f69580a1290 0x7f69580a3750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f695c005950 tx=0x7f695c00b500 comp rx=0 tx=0).stop 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c19bc10 msgr2=0x7f696c19c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c19bc10 0x7f696c19c090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f696400b780 tx=0x7f6964009550 comp rx=0 tx=0).stop 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 shutdown_connections 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f69580a1290 0x7f69580a3750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f696c19bc10 0x7f696c19c090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f696c0fff80 0x7f696c197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 --2- 172.21.15.16:0/2330102955 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f696c1050a0 0x7f696c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 >> 172.21.15.16:0/2330102955 conn(0x7f696c0fb840 msgr2=0x7f696c0695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 shutdown_connections 2024-05-08T23:19:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.477+0000 7f697162b700 1 -- 172.21.15.16:0/2330102955 wait complete. 2024-05-08T23:19:42.490 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:42.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 -- 172.21.15.16:0/2654272920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248100a10 msgr2=0x7f3248104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 --2- 172.21.15.16:0/2654272920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248100a10 0x7f3248104a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f32440098f0 tx=0x7f3244009c00 comp rx=0 tx=0).stop 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 -- 172.21.15.16:0/2654272920 shutdown_connections 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 --2- 172.21.15.16:0/2654272920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3248105160 0x7f324810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 --2- 172.21.15.16:0/2654272920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3248100060 0x7f3248100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 --2- 172.21.15.16:0/2654272920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248100a10 0x7f3248104a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 -- 172.21.15.16:0/2654272920 >> 172.21.15.16:0/2654272920 conn(0x7f3248076f40 msgr2=0x7f3248077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 -- 172.21.15.16:0/2654272920 shutdown_connections 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.632+0000 7f3250059700 1 -- 172.21.15.16:0/2654272920 wait complete. 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 Processor -- start 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 -- start start 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3248100060 0x7f3248197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248105160 0x7f3248197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 0x7f324819c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819c6b0 con 0x7f324819bc30 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819c820 con 0x7f3248100060 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f3250059700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819c9e0 con 0x7f3248105160 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f324e5f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 0x7f324819c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f324e5f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 0x7f324819c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60298/0 (socket says 172.21.15.16:60298) 2024-05-08T23:19:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f324e5f6700 1 -- 172.21.15.16:0/4053870095 learned_addr learned my addr 172.21.15.16:0/4053870095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:42.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f324ddf5700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3248100060 0x7f3248197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f324d5f4700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248105160 0x7f3248197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.633+0000 7f324e5f6700 1 -- 172.21.15.16:0/4053870095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248105160 msgr2=0x7f3248197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f324e5f6700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248105160 0x7f3248197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f324e5f6700 1 -- 172.21.15.16:0/4053870095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3248100060 msgr2=0x7f3248197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f324e5f6700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3248100060 0x7f3248197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f324e5f6700 1 -- 172.21.15.16:0/4053870095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3244009520 con 0x7f324819bc30 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f324e5f6700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 0x7f324819c0b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f324000ef30 tx=0x7f324000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240018070 con 0x7f324819bc30 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f324000f040 con 0x7f324819bc30 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32481a3ee0 con 0x7f324819bc30 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324000cbb0 con 0x7f324819bc30 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.634+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32481a4360 con 0x7f324819bc30 2024-05-08T23:19:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.635+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3248109440 con 0x7f324819bc30 2024-05-08T23:19:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.636+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) v1 ==== 135993+0+0 (secure 0 0 0) 0x7f324000cd50 con 0x7f324819bc30 2024-05-08T23:19:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.637+0000 7f323effd700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32340a1290 0x7f32340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.637+0000 7f324ddf5700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32340a1290 0x7f32340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.637+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f32400cc950 con 0x7f324819bc30 2024-05-08T23:19:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.638+0000 7f324ddf5700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32340a1290 0x7f32340a3750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3238005fd0 tx=0x7f3238009310 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.642+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3240095b20 con 0x7f324819bc30 2024-05-08T23:19:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.859+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3248067ca0 con 0x7f324819bc30 2024-05-08T23:19:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.859+0000 7f323effd700 1 -- 172.21.15.16:0/4053870095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3240099140 con 0x7f324819bc30 2024-05-08T23:19:42.859 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32340a1290 msgr2=0x7f32340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32340a1290 0x7f32340a3750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3238005fd0 tx=0x7f3238009310 comp rx=0 tx=0).stop 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 msgr2=0x7f324819c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 0x7f324819c0b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f324000ef30 tx=0x7f324000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 shutdown_connections 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32340a1290 0x7f32340a3750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f324819bc30 0x7f324819c0b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3248100060 0x7f3248197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 --2- 172.21.15.16:0/4053870095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3248105160 0x7f3248197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 >> 172.21.15.16:0/4053870095 conn(0x7f3248076f40 msgr2=0x7f3248068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 shutdown_connections 2024-05-08T23:19:42.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:42.862+0000 7f3250059700 1 -- 172.21.15.16:0/4053870095 wait complete. 2024-05-08T23:19:42.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 -- 172.21.15.16:0/3486054038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc1000f0 msgr2=0x7fbccc100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 --2- 172.21.15.16:0/3486054038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc1000f0 0x7fbccc100570 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80099c0 tx=0x7fbcc8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 -- 172.21.15.16:0/3486054038 shutdown_connections 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 --2- 172.21.15.16:0/3486054038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbccc100ab0 0x7fbccc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 --2- 172.21.15.16:0/3486054038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbccc105f90 0x7fbccc106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 --2- 172.21.15.16:0/3486054038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc1000f0 0x7fbccc100570 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 -- 172.21.15.16:0/3486054038 >> 172.21.15.16:0/3486054038 conn(0x7fbccc076ba0 msgr2=0x7fbccc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 -- 172.21.15.16:0/3486054038 shutdown_connections 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.021+0000 7fbcd4530700 1 -- 172.21.15.16:0/3486054038 wait complete. 2024-05-08T23:19:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 Processor -- start 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 -- start start 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbccc1000f0 0x7fbccc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 0x7fbccc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbccc105f90 0x7fbccc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc19c1e0 con 0x7fbccc100ab0 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc19c350 con 0x7fbccc1000f0 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd4530700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc19c510 con 0x7fbccc105f90 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd1acb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 0x7fbccc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd1acb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 0x7fbccc197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60324/0 (socket says 172.21.15.16:60324) 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd1acb700 1 -- 172.21.15.16:0/492337927 learned_addr learned my addr 172.21.15.16:0/492337927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd22cc700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbccc1000f0 0x7fbccc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.022+0000 7fbcd2acd700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbccc105f90 0x7fbccc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd1acb700 1 -- 172.21.15.16:0/492337927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbccc105f90 msgr2=0x7fbccc19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd1acb700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbccc105f90 0x7fbccc19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd1acb700 1 -- 172.21.15.16:0/492337927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbccc1000f0 msgr2=0x7fbccc1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd1acb700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbccc1000f0 0x7fbccc1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd1acb700 1 -- 172.21.15.16:0/492337927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80095f0 con 0x7fbccc100ab0 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd1acb700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 0x7fbccc197800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800f690 tx=0x7fbcc800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc801c070 con 0x7fbccc100ab0 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc19c790 con 0x7fbccc100ab0 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcc800fdb0 con 0x7fbccc100ab0 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8004710 con 0x7fbccc100ab0 2024-05-08T23:19:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.023+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc1a4030 con 0x7fbccc100ab0 2024-05-08T23:19:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.024+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc101560 con 0x7fbccc100ab0 2024-05-08T23:19:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.025+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fbcc8020070 con 0x7fbccc100ab0 2024-05-08T23:19:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.026+0000 7fbcc37fe700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbcb80a1320 0x7fbcb80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.026+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fbcc80cdc00 con 0x7fbccc100ab0 2024-05-08T23:19:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.026+0000 7fbcd22cc700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbcb80a1320 0x7fbcb80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.026+0000 7fbcd22cc700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbcb80a1320 0x7fbcb80a37e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbccc067bb0 tx=0x7fbcbc009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:43.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.030+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbcc8096c00 con 0x7fbccc100ab0 2024-05-08T23:19:43.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/412406971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2330102955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4053870095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:42 smithi155 ceph-mon[47105]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:43.234 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/412406971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.235 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2330102955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:43.235 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4053870095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.235 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:42 smithi016 ceph-mon[42541]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-05-08T23:19:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.249+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbccc1985c0 con 0x7fbccc100ab0 2024-05-08T23:19:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.249+0000 7fbcc37fe700 1 -- 172.21.15.16:0/492337927 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbcc809a220 con 0x7fbccc100ab0 2024-05-08T23:19:43.250 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbcb80a1320 msgr2=0x7fbcb80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbcb80a1320 0x7fbcb80a37e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbccc067bb0 tx=0x7fbcbc009260 comp rx=0 tx=0).stop 2024-05-08T23:19:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 msgr2=0x7fbccc197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 0x7fbccc197800 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800f690 tx=0x7fbcc800f6c0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 shutdown_connections 2024-05-08T23:19:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbcb80a1320 0x7fbcb80a37e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbccc105f90 0x7fbccc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbccc1000f0 0x7fbccc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 --2- 172.21.15.16:0/492337927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc100ab0 0x7fbccc197800 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 >> 172.21.15.16:0/492337927 conn(0x7fbccc076ba0 msgr2=0x7fbccc069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 shutdown_connections 2024-05-08T23:19:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.252+0000 7fbcd4530700 1 -- 172.21.15.16:0/492337927 wait complete. 2024-05-08T23:19:43.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/1846192739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f40682e0 msgr2=0x7fd4f40686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/1846192739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f40682e0 0x7fd4f40686c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00099c0 tx=0x7fd4f0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/1846192739 shutdown_connections 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/1846192739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f4106290 0x7fd4f410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/1846192739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f40682e0 0x7fd4f40686c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/1846192739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f4068c90 0x7fd4f4105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/1846192739 >> 172.21.15.16:0/1846192739 conn(0x7fd4f4076f40 msgr2=0x7fd4f4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/1846192739 shutdown_connections 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.406+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/1846192739 wait complete. 2024-05-08T23:19:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 Processor -- start 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 -- start start 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f40682e0 0x7fd4f40ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 0x7fd4f41003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f41008e0 0x7fd4f41a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f41a5a90 con 0x7fd4f4106290 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f41a5c00 con 0x7fd4f40682e0 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fbbd9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f41a5dc0 con 0x7fd4f41008e0 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4f9174700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 0x7fd4f41003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4f9174700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 0x7fd4f41003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60338/0 (socket says 172.21.15.16:60338) 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4f9174700 1 -- 172.21.15.16:0/256430863 learned_addr learned my addr 172.21.15.16:0/256430863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4f9975700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f40682e0 0x7fd4f40ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4fa176700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f41008e0 0x7fd4f41a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4f9174700 1 -- 172.21.15.16:0/256430863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f41008e0 msgr2=0x7fd4f41a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.407+0000 7fd4f9174700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f41008e0 0x7fd4f41a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4f9174700 1 -- 172.21.15.16:0/256430863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f40682e0 msgr2=0x7fd4f40ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4f9174700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f40682e0 0x7fd4f40ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4f9174700 1 -- 172.21.15.16:0/256430863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00095f0 con 0x7fd4f4106290 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4fa176700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f41008e0 0x7fd4f41a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4f9174700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 0x7fd4f41003a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d980 tx=0x7fd4e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4009930 con 0x7fd4f4106290 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f41a60a0 con 0x7fd4f4106290 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e4010460 con 0x7fd4f4106290 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400f620 con 0x7fd4f4106290 2024-05-08T23:19:43.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.408+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f41a65b0 con 0x7fd4f4106290 2024-05-08T23:19:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.409+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f4109460 con 0x7fd4f4106290 2024-05-08T23:19:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.410+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fd4e40105d0 con 0x7fd4f4106290 2024-05-08T23:19:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.411+0000 7fd4eaffd700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd4e00a1320 0x7fd4e00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.411+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fd4e40ccc50 con 0x7fd4f4106290 2024-05-08T23:19:43.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.411+0000 7fd4f9975700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd4e00a1320 0x7fd4e00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.411+0000 7fd4f9975700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd4e00a1320 0x7fd4e00a37e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f0009f10 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:43.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.414+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd4e4095c20 con 0x7fd4f4106290 2024-05-08T23:19:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: Updating smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: Updating smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: Updating smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/412406971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2330102955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4053870095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:42 smithi189 ceph-mon[46919]: mgrmap e23: x(active, since 5s), standbys: y, z 2024-05-08T23:19:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.631+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4f4101340 con 0x7fd4f4106290 2024-05-08T23:19:43.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.631+0000 7fd4eaffd700 1 -- 172.21.15.16:0/256430863 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4e40170a0 con 0x7fd4f4106290 2024-05-08T23:19:43.632 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd4e00a1320 msgr2=0x7fd4e00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd4e00a1320 0x7fd4e00a37e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0000c00 tx=0x7fd4f0009f10 comp rx=0 tx=0).stop 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 msgr2=0x7fd4f41003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 0x7fd4f41003a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d980 tx=0x7fd4e400dc90 comp rx=0 tx=0).stop 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 shutdown_connections 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd4e00a1320 0x7fd4e00a37e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4f41008e0 0x7fd4f41a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4f40682e0 0x7fd4f40ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 --2- 172.21.15.16:0/256430863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f4106290 0x7fd4f41003a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 >> 172.21.15.16:0/256430863 conn(0x7fd4f4076f40 msgr2=0x7fd4f40fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 shutdown_connections 2024-05-08T23:19:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.633+0000 7fd4fbbd9700 1 -- 172.21.15.16:0/256430863 wait complete. 2024-05-08T23:19:43.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.788+0000 7fa2c4b91700 1 -- 172.21.15.16:0/142084776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0105160 msgr2=0x7fa2c010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.788+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/142084776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0105160 0x7fa2c010cce0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b0a0 tx=0x7fa2b800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 -- 172.21.15.16:0/142084776 shutdown_connections 2024-05-08T23:19:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/142084776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0105160 0x7fa2c010cce0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/142084776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100060 0x7fa2c0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/142084776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2c0100a10 0x7fa2c0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 -- 172.21.15.16:0/142084776 >> 172.21.15.16:0/142084776 conn(0x7fa2c0076f40 msgr2=0x7fa2c0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 -- 172.21.15.16:0/142084776 shutdown_connections 2024-05-08T23:19:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 -- 172.21.15.16:0/142084776 wait complete. 2024-05-08T23:19:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.789+0000 7fa2c4b91700 1 Processor -- start 2024-05-08T23:19:43.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 -- start start 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 0x7fa2c0101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0105160 0x7fa2c0102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2c0102550 0x7fa2c01a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01a5c30 con 0x7fa2c0100060 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01a5da0 con 0x7fa2c0102550 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2c4b91700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c01a5f60 con 0x7fa2c0105160 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 0x7fa2c0101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 0x7fa2c0101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60354/0 (socket says 172.21.15.16:60354) 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 -- 172.21.15.16:0/890899036 learned_addr learned my addr 172.21.15.16:0/890899036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:43.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2bdd9b700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0105160 0x7fa2c0102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2bed9d700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2c0102550 0x7fa2c01a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 -- 172.21.15.16:0/890899036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0105160 msgr2=0x7fa2c0102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0105160 0x7fa2c0102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 -- 172.21.15.16:0/890899036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2c0102550 msgr2=0x7fa2c01a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2c0102550 0x7fa2c01a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.790+0000 7fa2be59c700 1 -- 172.21.15.16:0/890899036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4009520 con 0x7fa2c0100060 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.791+0000 7fa2be59c700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 0x7fa2c0101ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000e9a0 tx=0x7fa2b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.791+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0009790 con 0x7fa2c0100060 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.791+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2b000cbd0 con 0x7fa2c0100060 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.791+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b8009d00 con 0x7fa2c0100060 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.791+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0018680 con 0x7fa2c0100060 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.791+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c01a6610 con 0x7fa2c0100060 2024-05-08T23:19:43.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.792+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0109440 con 0x7fa2c0100060 2024-05-08T23:19:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.793+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fa2b000cd40 con 0x7fa2c0100060 2024-05-08T23:19:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.794+0000 7fa2af7fe700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa2a80a13a0 0x7fa2a80a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.794+0000 7fa2bdd9b700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa2a80a13a0 0x7fa2a80a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.794+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fa2b0014070 con 0x7fa2c0100060 2024-05-08T23:19:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.795+0000 7fa2bdd9b700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa2a80a13a0 0x7fa2a80a3860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2c01031b0 tx=0x7fa2b4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:43.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:43.798+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa2b0096740 con 0x7fa2c0100060 2024-05-08T23:19:43.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:43 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/492337927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:43.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:43 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/256430863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:44.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.020+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa2c0102f60 con 0x7fa2c0100060 2024-05-08T23:19:44.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.021+0000 7fa2af7fe700 1 -- 172.21.15.16:0/890899036 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2b00260a0 con 0x7fa2c0100060 2024-05-08T23:19:44.022 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa2a80a13a0 msgr2=0x7fa2a80a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa2a80a13a0 0x7fa2a80a3860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2c01031b0 tx=0x7fa2b4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 msgr2=0x7fa2c0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 0x7fa2c0101ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000e9a0 tx=0x7fa2b000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 shutdown_connections 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa2a80a13a0 0x7fa2a80a3860 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2c0102550 0x7fa2c01a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0100060 0x7fa2c0101ad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 --2- 172.21.15.16:0/890899036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0105160 0x7fa2c0102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 >> 172.21.15.16:0/890899036 conn(0x7fa2c0076f40 msgr2=0x7fa2c0068370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 shutdown_connections 2024-05-08T23:19:44.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.023+0000 7fa2c4b91700 1 -- 172.21.15.16:0/890899036 wait complete. 2024-05-08T23:19:44.038 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T23:19:44.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 -- 172.21.15.16:0/813712365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101fb0 msgr2=0x7f6b60102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 --2- 172.21.15.16:0/813712365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101fb0 0x7f6b60102390 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009990 tx=0x7f6b50009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 -- 172.21.15.16:0/813712365 shutdown_connections 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 --2- 172.21.15.16:0/813712365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b6010b3d0 0x7f6b6010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 --2- 172.21.15.16:0/813712365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101fb0 0x7f6b60102390 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 --2- 172.21.15.16:0/813712365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b601028d0 0x7f6b6010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 -- 172.21.15.16:0/813712365 >> 172.21.15.16:0/813712365 conn(0x7f6b600fb810 msgr2=0x7f6b600fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 -- 172.21.15.16:0/813712365 shutdown_connections 2024-05-08T23:19:44.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.175+0000 7f6b65166700 1 -- 172.21.15.16:0/813712365 wait complete. 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 Processor -- start 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 -- start start 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b60101fb0 0x7f6b60197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 0x7f6b60197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b6010b3d0 0x7f6b6019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019c110 con 0x7f6b601028d0 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019c280 con 0x7f6b60101fb0 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.176+0000 7f6b65166700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019c440 con 0x7f6b6010b3d0 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 0x7f6b60197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 0x7f6b60197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60370/0 (socket says 172.21.15.16:60370) 2024-05-08T23:19:44.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 -- 172.21.15.16:0/4143482905 learned_addr learned my addr 172.21.15.16:0/4143482905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5ed9d700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b60101fb0 0x7f6b60197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5f59e700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b6010b3d0 0x7f6b6019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 -- 172.21.15.16:0/4143482905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b6010b3d0 msgr2=0x7f6b6019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b6010b3d0 0x7f6b6019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 -- 172.21.15.16:0/4143482905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b60101fb0 msgr2=0x7f6b60197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b60101fb0 0x7f6b60197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 -- 172.21.15.16:0/4143482905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500095f0 con 0x7f6b601028d0 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5f59e700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b6010b3d0 0x7f6b6019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b5e59c700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 0x7f6b60197780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c970 tx=0x7f6b5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54007b00 con 0x7f6b601028d0 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6019c720 con 0x7f6b601028d0 2024-05-08T23:19:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b5400ce60 con 0x7f6b601028d0 2024-05-08T23:19:44.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54005350 con 0x7f6b601028d0 2024-05-08T23:19:44.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.177+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b601a3f60 con 0x7f6b601028d0 2024-05-08T23:19:44.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.178+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b60106460 con 0x7f6b601028d0 2024-05-08T23:19:44.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.180+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f6b540054b0 con 0x7f6b601028d0 2024-05-08T23:19:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.180+0000 7f6b47fff700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6b480a1320 0x7f6b480a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.180+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f6b540cd0c0 con 0x7f6b601028d0 2024-05-08T23:19:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.180+0000 7f6b5ed9d700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6b480a1320 0x7f6b480a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.181+0000 7f6b5ed9d700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6b480a1320 0x7f6b480a37e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b5c0 tx=0x7f6b50005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.184+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b54096090 con 0x7f6b601028d0 2024-05-08T23:19:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:43 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/492337927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:43 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/256430863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:44.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:43 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/492337927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:43 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/256430863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.453+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b60069d50 con 0x7f6b601028d0 2024-05-08T23:19:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.454+0000 7f6b47fff700 1 -- 172.21.15.16:0/4143482905 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f6b5401d020 con 0x7f6b601028d0 2024-05-08T23:19:44.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:44.455 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6b480a1320 msgr2=0x7f6b480a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6b480a1320 0x7f6b480a37e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b5c0 tx=0x7f6b50005fb0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 msgr2=0x7f6b60197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 0x7f6b60197780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c970 tx=0x7f6b5400cc80 comp rx=0 tx=0).stop 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 shutdown_connections 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6b480a1320 0x7f6b480a37e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b6010b3d0 0x7f6b6019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b60101fb0 0x7f6b60197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 --2- 172.21.15.16:0/4143482905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b601028d0 0x7f6b60197780 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 >> 172.21.15.16:0/4143482905 conn(0x7f6b600fb810 msgr2=0x7f6b600fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 shutdown_connections 2024-05-08T23:19:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.457+0000 7f6b65166700 1 -- 172.21.15.16:0/4143482905 wait complete. 2024-05-08T23:19:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:19:44.472 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T23:19:44.472 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T23:19:44.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 -- 172.21.15.16:0/1402553129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff52410b3c0 msgr2=0x7ff52410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1402553129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff52410b3c0 0x7ff52410d880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b0a0 tx=0x7ff51c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 -- 172.21.15.16:0/1402553129 shutdown_connections 2024-05-08T23:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1402553129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff52410b3c0 0x7ff52410d880 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1402553129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff524101fa0 0x7ff524102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1402553129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5241028c0 0x7ff52410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 -- 172.21.15.16:0/1402553129 >> 172.21.15.16:0/1402553129 conn(0x7ff5240fb7e0 msgr2=0x7ff5240fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 -- 172.21.15.16:0/1402553129 shutdown_connections 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.608+0000 7ff52ad05700 1 -- 172.21.15.16:0/1402553129 wait complete. 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 Processor -- start 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 -- start start 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff524101fa0 0x7ff5241995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 0x7ff524199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff52410b3c0 0x7ff52419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52419e490 con 0x7ff5241028c0 2024-05-08T23:19:44.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52419e600 con 0x7ff524101fa0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff52ad05700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52419e7c0 con 0x7ff52410b3c0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 0x7ff524199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff51bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 0x7ff524199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60388/0 (socket says 172.21.15.16:60388) 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.609+0000 7ff51bfff700 1 -- 172.21.15.16:0/1427046647 learned_addr learned my addr 172.21.15.16:0/1427046647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff528aa1700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff524101fa0 0x7ff5241995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff5292a2700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff52410b3c0 0x7ff52419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff51bfff700 1 -- 172.21.15.16:0/1427046647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff52410b3c0 msgr2=0x7ff52419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff51bfff700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff52410b3c0 0x7ff52419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff51bfff700 1 -- 172.21.15.16:0/1427046647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff524101fa0 msgr2=0x7ff5241995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff51bfff700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff524101fa0 0x7ff5241995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff51bfff700 1 -- 172.21.15.16:0/1427046647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c009d00 con 0x7ff5241028c0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff51bfff700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 0x7ff524199b00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:44.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400f890 con 0x7ff5241028c0 2024-05-08T23:19:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff51400fe80 con 0x7ff5241028c0 2024-05-08T23:19:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52419eaa0 con 0x7ff5241028c0 2024-05-08T23:19:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400e3b0 con 0x7ff5241028c0 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.610+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5241a62e0 con 0x7ff5241028c0 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.612+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5241088f0 con 0x7ff5241028c0 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.613+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7ff514010460 con 0x7ff5241028c0 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.613+0000 7ff519ffb700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff50c0a13a0 0x7ff50c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.614+0000 7ff528aa1700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff50c0a13a0 0x7ff50c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.614+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7ff5140ccdd0 con 0x7ff5241028c0 2024-05-08T23:19:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.614+0000 7ff528aa1700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff50c0a13a0 0x7ff50c0a3860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff520007be0 tx=0x7ff5200073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.618+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff514095e30 con 0x7ff5241028c0 2024-05-08T23:19:44.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.831+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff52419a870 con 0x7ff50c0a13a0 2024-05-08T23:19:44.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.845+0000 7ff519ffb700 1 -- 172.21.15.16:0/1427046647 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+425554 (secure 0 0 0) 0x7ff52419a870 con 0x7ff50c0a13a0 2024-05-08T23:19:44.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff50c0a13a0 msgr2=0x7ff50c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff50c0a13a0 0x7ff50c0a3860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff520007be0 tx=0x7ff5200073d0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 msgr2=0x7ff524199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 0x7ff524199b00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 shutdown_connections 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff50c0a13a0 0x7ff50c0a3860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff52410b3c0 0x7ff52419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff524101fa0 0x7ff5241995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 --2- 172.21.15.16:0/1427046647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5241028c0 0x7ff524199b00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 >> 172.21.15.16:0/1427046647 conn(0x7ff5240fb7e0 msgr2=0x7ff524100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 shutdown_connections 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:44.849+0000 7ff52ad05700 1 -- 172.21.15.16:0/1427046647 wait complete. 2024-05-08T23:19:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:19:44.864 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-08T23:19:43.939958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184173,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":778,"num_read_kb":545,"num_write":461,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400548,"kb_used_data":14768,"kb_used_omap":18,"kb_used_meta":385517,"kb_avail":1124541276,"statfs":{"total":1151940427776,"available":1151530266624,"internally_reserved":0,"allocated":15122432,"data_stored":9200373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":394769736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989242"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979175+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:19:37.979175+0000","last_peered":"2024-05-08T23:19:37.979175+0000","last_clean":"2024-05-08T23:19:37.979175+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:19:37.979175+0000","last_undegraded":"2024-05-08T23:19:37.979175+0000","last_fullsized":"2024-05-08T23:19:37.979175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975013+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:19:37.975013+0000","last_peered":"2024-05-08T23:19:37.975013+0000","last_clean":"2024-05-08T23:19:37.975013+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:19:37.975013+0000","last_undegraded":"2024-05-08T23:19:37.975013+0000","last_fullsized":"2024-05-08T23:19:37.975013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978939+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:19:37.978939+0000","last_peered":"2024-05-08T23:19:37.978939+0000","last_clean":"2024-05-08T23:19:37.978939+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:19:37.978939+0000","last_undegraded":"2024-05-08T23:19:37.978939+0000","last_fullsized":"2024-05-08T23:19:37.978939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977387+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:19:37.977387+0000","last_peered":"2024-05-08T23:19:37.977387+0000","last_clean":"2024-05-08T23:19:37.977387+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:19:37.977387+0000","last_undegraded":"2024-05-08T23:19:37.977387+0000","last_fullsized":"2024-05-08T23:19:37.977387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974352+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:19:37.974352+0000","last_peered":"2024-05-08T23:19:37.974352+0000","last_clean":"2024-05-08T23:19:37.974352+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:19:37.974352+0000","last_undegraded":"2024-05-08T23:19:37.974352+0000","last_fullsized":"2024-05-08T23:19:37.974352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980276+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:19:37.980276+0000","last_peered":"2024-05-08T23:19:37.980276+0000","last_clean":"2024-05-08T23:19:37.980276+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:19:37.980276+0000","last_undegraded":"2024-05-08T23:19:37.980276+0000","last_fullsized":"2024-05-08T23:19:37.980276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978937+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:19:37.978937+0000","last_peered":"2024-05-08T23:19:37.978937+0000","last_clean":"2024-05-08T23:19:37.978937+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:19:37.978937+0000","last_undegraded":"2024-05-08T23:19:37.978937+0000","last_fullsized":"2024-05-08T23:19:37.978937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.970401+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:19:37.970401+0000","last_peered":"2024-05-08T23:19:37.970401+0000","last_clean":"2024-05-08T23:19:37.970401+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:19:37.970401+0000","last_undegraded":"2024-05-08T23:19:37.970401+0000","last_fullsized":"2024-05-08T23:19:37.970401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977019+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:19:37.977019+0000","last_peered":"2024-05-08T23:19:37.977019+0000","last_clean":"2024-05-08T23:19:37.977019+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:19:37.977019+0000","last_undegraded":"2024-05-08T23:19:37.977019+0000","last_fullsized":"2024-05-08T23:19:37.977019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.984147+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:19:37.984147+0000","last_peered":"2024-05-08T23:19:37.984147+0000","last_clean":"2024-05-08T23:19:37.984147+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:19:37.984147+0000","last_undegraded":"2024-05-08T23:19:37.984147+0000","last_fullsized":"2024-05-08T23:19:37.984147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980710+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:19:37.980710+0000","last_peered":"2024-05-08T23:19:37.980710+0000","last_clean":"2024-05-08T23:19:37.980710+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:19:37.980710+0000","last_undegraded":"2024-05-08T23:19:37.980710+0000","last_fullsized":"2024-05-08T23:19:37.980710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.967635+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:19:37.967635+0000","last_peered":"2024-05-08T23:19:37.967635+0000","last_clean":"2024-05-08T23:19:37.967635+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:19:37.967635+0000","last_undegraded":"2024-05-08T23:19:37.967635+0000","last_fullsized":"2024-05-08T23:19:37.967635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985493+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:19:37.985493+0000","last_peered":"2024-05-08T23:19:37.985493+0000","last_clean":"2024-05-08T23:19:37.985493+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:19:37.985493+0000","last_undegraded":"2024-05-08T23:19:37.985493+0000","last_fullsized":"2024-05-08T23:19:37.985493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.971337+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:19:37.971337+0000","last_peered":"2024-05-08T23:19:37.971337+0000","last_clean":"2024-05-08T23:19:37.971337+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:19:37.971337+0000","last_undegraded":"2024-05-08T23:19:37.971337+0000","last_fullsized":"2024-05-08T23:19:37.971337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978114+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:19:37.978114+0000","last_peered":"2024-05-08T23:19:37.978114+0000","last_clean":"2024-05-08T23:19:37.978114+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:19:37.978114+0000","last_undegraded":"2024-05-08T23:19:37.978114+0000","last_fullsized":"2024-05-08T23:19:37.978114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982276+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:19:37.982276+0000","last_peered":"2024-05-08T23:19:37.982276+0000","last_clean":"2024-05-08T23:19:37.982276+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:19:37.982276+0000","last_undegraded":"2024-05-08T23:19:37.982276+0000","last_fullsized":"2024-05-08T23:19:37.982276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979259+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:19:37.979259+0000","last_peered":"2024-05-08T23:19:37.979259+0000","last_clean":"2024-05-08T23:19:37.979259+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:19:37.979259+0000","last_undegraded":"2024-05-08T23:19:37.979259+0000","last_fullsized":"2024-05-08T23:19:37.979259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979567+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:19:37.979567+0000","last_peered":"2024-05-08T23:19:37.979567+0000","last_clean":"2024-05-08T23:19:37.979567+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:19:37.979567+0000","last_undegraded":"2024-05-08T23:19:37.979567+0000","last_fullsized":"2024-05-08T23:19:37.979567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.976723+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:19:37.976723+0000","last_peered":"2024-05-08T23:19:37.976723+0000","last_clean":"2024-05-08T23:19:37.976723+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:19:37.976723+0000","last_undegraded":"2024-05-08T23:19:37.976723+0000","last_fullsized":"2024-05-08T23:19:37.976723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796041+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:43.796041+0000","last_peered":"2024-05-08T23:18:43.796041+0000","last_clean":"2024-05-08T23:18:43.796041+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:43.796041+0000","last_undegraded":"2024-05-08T23:18:43.796041+0000","last_fullsized":"2024-05-08T23:18:43.796041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034288+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:44.034288+0000","last_peered":"2024-05-08T23:18:44.034288+0000","last_clean":"2024-05-08T23:18:44.034288+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:44.034288+0000","last_undegraded":"2024-05-08T23:18:44.034288+0000","last_fullsized":"2024-05-08T23:18:44.034288+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.964536+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:19:37.964536+0000","last_peered":"2024-05-08T23:19:37.964536+0000","last_clean":"2024-05-08T23:19:37.964536+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:19:37.964536+0000","last_undegraded":"2024-05-08T23:19:37.964536+0000","last_fullsized":"2024-05-08T23:19:37.964536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793688+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:43.793688+0000","last_peered":"2024-05-08T23:18:43.793688+0000","last_clean":"2024-05-08T23:18:43.793688+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:43.793688+0000","last_undegraded":"2024-05-08T23:18:43.793688+0000","last_fullsized":"2024-05-08T23:18:43.793688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794148+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:43.794148+0000","last_peered":"2024-05-08T23:18:43.794148+0000","last_clean":"2024-05-08T23:18:43.794148+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:43.794148+0000","last_undegraded":"2024-05-08T23:18:43.794148+0000","last_fullsized":"2024-05-08T23:18:43.794148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.983386+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:19:37.983386+0000","last_peered":"2024-05-08T23:19:37.983386+0000","last_clean":"2024-05-08T23:19:37.983386+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:19:37.983386+0000","last_undegraded":"2024-05-08T23:19:37.983386+0000","last_fullsized":"2024-05-08T23:19:37.983386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803701+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803701+0000","last_peered":"2024-05-08T23:18:43.803701+0000","last_clean":"2024-05-08T23:18:43.803701+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803701+0000","last_undegraded":"2024-05-08T23:18:43.803701+0000","last_fullsized":"2024-05-08T23:18:43.803701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.635572+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:45.635572+0000","last_peered":"2024-05-08T23:18:45.635572+0000","last_clean":"2024-05-08T23:18:45.635572+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:45.635572+0000","last_undegraded":"2024-05-08T23:18:45.635572+0000","last_fullsized":"2024-05-08T23:18:45.635572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979236+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:19:37.979236+0000","last_peered":"2024-05-08T23:19:37.979236+0000","last_clean":"2024-05-08T23:19:37.979236+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:19:37.979236+0000","last_undegraded":"2024-05-08T23:19:37.979236+0000","last_fullsized":"2024-05-08T23:19:37.979236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793735+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:43.793735+0000","last_peered":"2024-05-08T23:18:43.793735+0000","last_clean":"2024-05-08T23:18:43.793735+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:43.793735+0000","last_undegraded":"2024-05-08T23:18:43.793735+0000","last_fullsized":"2024-05-08T23:18:43.793735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804752+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:43.804752+0000","last_peered":"2024-05-08T23:18:43.804752+0000","last_clean":"2024-05-08T23:18:43.804752+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:43.804752+0000","last_undegraded":"2024-05-08T23:18:43.804752+0000","last_fullsized":"2024-05-08T23:18:43.804752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.963335+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:19:37.963335+0000","last_peered":"2024-05-08T23:19:37.963335+0000","last_clean":"2024-05-08T23:19:37.963335+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:19:37.963335+0000","last_undegraded":"2024-05-08T23:19:37.963335+0000","last_fullsized":"2024-05-08T23:19:37.963335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795805+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:43.795805+0000","last_peered":"2024-05-08T23:18:43.795805+0000","last_clean":"2024-05-08T23:18:43.795805+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:43.795805+0000","last_undegraded":"2024-05-08T23:18:43.795805+0000","last_fullsized":"2024-05-08T23:18:43.795805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":117,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818584+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:19:38.818584+0000","last_peered":"2024-05-08T23:19:38.818584+0000","last_clean":"2024-05-08T23:19:38.818584+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:19:38.818584+0000","last_undegraded":"2024-05-08T23:19:38.818584+0000","last_fullsized":"2024-05-08T23:19:38.818584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978785+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:19:37.978785+0000","last_peered":"2024-05-08T23:19:37.978785+0000","last_clean":"2024-05-08T23:19:37.978785+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:19:37.978785+0000","last_undegraded":"2024-05-08T23:19:37.978785+0000","last_fullsized":"2024-05-08T23:19:37.978785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803847+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803847+0000","last_peered":"2024-05-08T23:18:43.803847+0000","last_clean":"2024-05-08T23:18:43.803847+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803847+0000","last_undegraded":"2024-05-08T23:18:43.803847+0000","last_fullsized":"2024-05-08T23:18:43.803847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797144+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:43.797144+0000","last_peered":"2024-05-08T23:18:43.797144+0000","last_clean":"2024-05-08T23:18:43.797144+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:43.797144+0000","last_undegraded":"2024-05-08T23:18:43.797144+0000","last_fullsized":"2024-05-08T23:18:43.797144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.969192+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:19:37.969192+0000","last_peered":"2024-05-08T23:19:37.969192+0000","last_clean":"2024-05-08T23:19:37.969192+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:19:37.969192+0000","last_undegraded":"2024-05-08T23:19:37.969192+0000","last_fullsized":"2024-05-08T23:19:37.969192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798515+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:43.798515+0000","last_peered":"2024-05-08T23:18:43.798515+0000","last_clean":"2024-05-08T23:18:43.798515+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:43.798515+0000","last_undegraded":"2024-05-08T23:18:43.798515+0000","last_fullsized":"2024-05-08T23:18:43.798515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797721+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:43.797721+0000","last_peered":"2024-05-08T23:18:43.797721+0000","last_clean":"2024-05-08T23:18:43.797721+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:43.797721+0000","last_undegraded":"2024-05-08T23:18:43.797721+0000","last_fullsized":"2024-05-08T23:18:43.797721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977826+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:19:37.977826+0000","last_peered":"2024-05-08T23:19:37.977826+0000","last_clean":"2024-05-08T23:19:37.977826+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:19:37.977826+0000","last_undegraded":"2024-05-08T23:19:37.977826+0000","last_fullsized":"2024-05-08T23:19:37.977826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.091431+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:44.091431+0000","last_peered":"2024-05-08T23:18:44.091431+0000","last_clean":"2024-05-08T23:18:44.091431+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:44.091431+0000","last_undegraded":"2024-05-08T23:18:44.091431+0000","last_fullsized":"2024-05-08T23:18:44.091431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795525+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:43.795525+0000","last_peered":"2024-05-08T23:18:43.795525+0000","last_clean":"2024-05-08T23:18:43.795525+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:43.795525+0000","last_undegraded":"2024-05-08T23:18:43.795525+0000","last_fullsized":"2024-05-08T23:18:43.795525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982914+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:19:37.982914+0000","last_peered":"2024-05-08T23:19:37.982914+0000","last_clean":"2024-05-08T23:19:37.982914+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:19:37.982914+0000","last_undegraded":"2024-05-08T23:19:37.982914+0000","last_fullsized":"2024-05-08T23:19:37.982914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793690+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:43.793690+0000","last_peered":"2024-05-08T23:18:43.793690+0000","last_clean":"2024-05-08T23:18:43.793690+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:43.793690+0000","last_undegraded":"2024-05-08T23:18:43.793690+0000","last_fullsized":"2024-05-08T23:18:43.793690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":110,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.935026+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:19:37.935026+0000","last_peered":"2024-05-08T23:19:37.935026+0000","last_clean":"2024-05-08T23:19:37.935026+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:19:37.935026+0000","last_undegraded":"2024-05-08T23:19:37.935026+0000","last_fullsized":"2024-05-08T23:19:37.935026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.981390+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:19:37.981390+0000","last_peered":"2024-05-08T23:19:37.981390+0000","last_clean":"2024-05-08T23:19:37.981390+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:19:37.981390+0000","last_undegraded":"2024-05-08T23:19:37.981390+0000","last_fullsized":"2024-05-08T23:19:37.981390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.033348+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:44.033348+0000","last_peered":"2024-05-08T23:18:44.033348+0000","last_clean":"2024-05-08T23:18:44.033348+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:44.033348+0000","last_undegraded":"2024-05-08T23:18:44.033348+0000","last_fullsized":"2024-05-08T23:18:44.033348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797353+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:43.797353+0000","last_peered":"2024-05-08T23:18:43.797353+0000","last_clean":"2024-05-08T23:18:43.797353+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:43.797353+0000","last_undegraded":"2024-05-08T23:18:43.797353+0000","last_fullsized":"2024-05-08T23:18:43.797353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975947+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:19:37.975947+0000","last_peered":"2024-05-08T23:19:37.975947+0000","last_clean":"2024-05-08T23:19:37.975947+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:19:37.975947+0000","last_undegraded":"2024-05-08T23:19:37.975947+0000","last_fullsized":"2024-05-08T23:19:37.975947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794345+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:43.794345+0000","last_peered":"2024-05-08T23:18:43.794345+0000","last_clean":"2024-05-08T23:18:43.794345+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:43.794345+0000","last_undegraded":"2024-05-08T23:18:43.794345+0000","last_fullsized":"2024-05-08T23:18:43.794345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795045+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:43.795045+0000","last_peered":"2024-05-08T23:18:43.795045+0000","last_clean":"2024-05-08T23:18:43.795045+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:43.795045+0000","last_undegraded":"2024-05-08T23:18:43.795045+0000","last_fullsized":"2024-05-08T23:18:43.795045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982345+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:19:37.982345+0000","last_peered":"2024-05-08T23:19:37.982345+0000","last_clean":"2024-05-08T23:19:37.982345+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:19:37.982345+0000","last_undegraded":"2024-05-08T23:19:37.982345+0000","last_fullsized":"2024-05-08T23:19:37.982345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.033715+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:44.033715+0000","last_peered":"2024-05-08T23:18:44.033715+0000","last_clean":"2024-05-08T23:18:44.033715+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:44.033715+0000","last_undegraded":"2024-05-08T23:18:44.033715+0000","last_fullsized":"2024-05-08T23:18:44.033715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'38","reported_seq":240,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818239+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:19:38.818239+0000","last_peered":"2024-05-08T23:19:38.818239+0000","last_clean":"2024-05-08T23:19:38.818239+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:19:38.818239+0000","last_undegraded":"2024-05-08T23:19:38.818239+0000","last_fullsized":"2024-05-08T23:19:38.818239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62677,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":183,"num_read_kb":153,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973333+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:19:37.973333+0000","last_peered":"2024-05-08T23:19:37.973333+0000","last_clean":"2024-05-08T23:19:37.973333+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:19:37.973333+0000","last_undegraded":"2024-05-08T23:19:37.973333+0000","last_fullsized":"2024-05-08T23:19:37.973333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795420+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:18:43.795420+0000","last_peered":"2024-05-08T23:18:43.795420+0000","last_clean":"2024-05-08T23:18:43.795420+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:18:43.795420+0000","last_undegraded":"2024-05-08T23:18:43.795420+0000","last_fullsized":"2024-05-08T23:18:43.795420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796397+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:43.796397+0000","last_peered":"2024-05-08T23:18:43.796397+0000","last_clean":"2024-05-08T23:18:43.796397+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:43.796397+0000","last_undegraded":"2024-05-08T23:18:43.796397+0000","last_fullsized":"2024-05-08T23:18:43.796397+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985255+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:19:37.985255+0000","last_peered":"2024-05-08T23:19:37.985255+0000","last_clean":"2024-05-08T23:19:37.985255+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:19:37.985255+0000","last_undegraded":"2024-05-08T23:19:37.985255+0000","last_fullsized":"2024-05-08T23:19:37.985255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798559+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:43.798559+0000","last_peered":"2024-05-08T23:18:43.798559+0000","last_clean":"2024-05-08T23:18:43.798559+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:43.798559+0000","last_undegraded":"2024-05-08T23:18:43.798559+0000","last_fullsized":"2024-05-08T23:18:43.798559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795869+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:43.795869+0000","last_peered":"2024-05-08T23:18:43.795869+0000","last_clean":"2024-05-08T23:18:43.795869+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:43.795869+0000","last_undegraded":"2024-05-08T23:18:43.795869+0000","last_fullsized":"2024-05-08T23:18:43.795869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985393+0000","last_change":"2024-05-08T23:18:44.034932+0000","last_active":"2024-05-08T23:19:37.985393+0000","last_peered":"2024-05-08T23:19:37.985393+0000","last_clean":"2024-05-08T23:19:37.985393+0000","last_became_active":"2024-05-08T23:18:44.034439+0000","last_became_peered":"2024-05-08T23:18:44.034439+0000","last_unstale":"2024-05-08T23:19:37.985393+0000","last_undegraded":"2024-05-08T23:19:37.985393+0000","last_fullsized":"2024-05-08T23:19:37.985393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.536646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034119+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:44.034119+0000","last_peered":"2024-05-08T23:18:44.034119+0000","last_clean":"2024-05-08T23:18:44.034119+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:44.034119+0000","last_undegraded":"2024-05-08T23:18:44.034119+0000","last_fullsized":"2024-05-08T23:18:44.034119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818321+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:19:38.818321+0000","last_peered":"2024-05-08T23:19:38.818321+0000","last_clean":"2024-05-08T23:19:38.818321+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:19:38.818321+0000","last_undegraded":"2024-05-08T23:19:38.818321+0000","last_fullsized":"2024-05-08T23:19:38.818321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1822,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973136+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:19:37.973136+0000","last_peered":"2024-05-08T23:19:37.973136+0000","last_clean":"2024-05-08T23:19:37.973136+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:19:37.973136+0000","last_undegraded":"2024-05-08T23:19:37.973136+0000","last_fullsized":"2024-05-08T23:19:37.973136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798551+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:43.798551+0000","last_peered":"2024-05-08T23:18:43.798551+0000","last_clean":"2024-05-08T23:18:43.798551+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:43.798551+0000","last_undegraded":"2024-05-08T23:18:43.798551+0000","last_fullsized":"2024-05-08T23:18:43.798551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804250+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:43.804250+0000","last_peered":"2024-05-08T23:18:43.804250+0000","last_clean":"2024-05-08T23:18:43.804250+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:43.804250+0000","last_undegraded":"2024-05-08T23:18:43.804250+0000","last_fullsized":"2024-05-08T23:18:43.804250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.962016+0000","last_change":"2024-05-08T23:18:43.929400+0000","last_active":"2024-05-08T23:19:37.962016+0000","last_peered":"2024-05-08T23:19:37.962016+0000","last_clean":"2024-05-08T23:19:37.962016+0000","last_became_active":"2024-05-08T23:18:43.928980+0000","last_became_peered":"2024-05-08T23:18:43.928980+0000","last_unstale":"2024-05-08T23:19:37.962016+0000","last_undegraded":"2024-05-08T23:19:37.962016+0000","last_fullsized":"2024-05-08T23:19:37.962016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:10.433725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795293+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:18:43.795293+0000","last_peered":"2024-05-08T23:18:43.795293+0000","last_clean":"2024-05-08T23:18:43.795293+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:18:43.795293+0000","last_undegraded":"2024-05-08T23:18:43.795293+0000","last_fullsized":"2024-05-08T23:18:43.795293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804811+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:43.804811+0000","last_peered":"2024-05-08T23:18:43.804811+0000","last_clean":"2024-05-08T23:18:43.804811+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:43.804811+0000","last_undegraded":"2024-05-08T23:18:43.804811+0000","last_fullsized":"2024-05-08T23:18:43.804811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973319+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:19:37.973319+0000","last_peered":"2024-05-08T23:19:37.973319+0000","last_clean":"2024-05-08T23:19:37.973319+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:19:37.973319+0000","last_undegraded":"2024-05-08T23:19:37.973319+0000","last_fullsized":"2024-05-08T23:19:37.973319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804513+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:43.804513+0000","last_peered":"2024-05-08T23:18:43.804513+0000","last_clean":"2024-05-08T23:18:43.804513+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:43.804513+0000","last_undegraded":"2024-05-08T23:18:43.804513+0000","last_fullsized":"2024-05-08T23:18:43.804513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034555+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:44.034555+0000","last_peered":"2024-05-08T23:18:44.034555+0000","last_clean":"2024-05-08T23:18:44.034555+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:44.034555+0000","last_undegraded":"2024-05-08T23:18:44.034555+0000","last_fullsized":"2024-05-08T23:18:44.034555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979180+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:19:37.979180+0000","last_peered":"2024-05-08T23:19:37.979180+0000","last_clean":"2024-05-08T23:19:37.979180+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:19:37.979180+0000","last_undegraded":"2024-05-08T23:19:37.979180+0000","last_fullsized":"2024-05-08T23:19:37.979180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798335+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:43.798335+0000","last_peered":"2024-05-08T23:18:43.798335+0000","last_clean":"2024-05-08T23:18:43.798335+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:43.798335+0000","last_undegraded":"2024-05-08T23:18:43.798335+0000","last_fullsized":"2024-05-08T23:18:43.798335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797081+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:43.797081+0000","last_peered":"2024-05-08T23:18:43.797081+0000","last_clean":"2024-05-08T23:18:43.797081+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:43.797081+0000","last_undegraded":"2024-05-08T23:18:43.797081+0000","last_fullsized":"2024-05-08T23:18:43.797081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.970690+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:19:37.970690+0000","last_peered":"2024-05-08T23:19:37.970690+0000","last_clean":"2024-05-08T23:19:37.970690+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:19:37.970690+0000","last_undegraded":"2024-05-08T23:19:37.970690+0000","last_fullsized":"2024-05-08T23:19:37.970690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798132+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:43.798132+0000","last_peered":"2024-05-08T23:18:43.798132+0000","last_clean":"2024-05-08T23:18:43.798132+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:43.798132+0000","last_undegraded":"2024-05-08T23:18:43.798132+0000","last_fullsized":"2024-05-08T23:18:43.798132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973657+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:19:37.973657+0000","last_peered":"2024-05-08T23:19:37.973657+0000","last_clean":"2024-05-08T23:19:37.973657+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:19:37.973657+0000","last_undegraded":"2024-05-08T23:19:37.973657+0000","last_fullsized":"2024-05-08T23:19:37.973657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":111,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.987062+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:19:37.987062+0000","last_peered":"2024-05-08T23:19:37.987062+0000","last_clean":"2024-05-08T23:19:37.987062+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:19:37.987062+0000","last_undegraded":"2024-05-08T23:19:37.987062+0000","last_fullsized":"2024-05-08T23:19:37.987062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797552+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:43.797552+0000","last_peered":"2024-05-08T23:18:43.797552+0000","last_clean":"2024-05-08T23:18:43.797552+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:43.797552+0000","last_undegraded":"2024-05-08T23:18:43.797552+0000","last_fullsized":"2024-05-08T23:18:43.797552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979618+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:19:37.979618+0000","last_peered":"2024-05-08T23:19:37.979618+0000","last_clean":"2024-05-08T23:19:37.979618+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:19:37.979618+0000","last_undegraded":"2024-05-08T23:19:37.979618+0000","last_fullsized":"2024-05-08T23:19:37.979618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794337+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:43.794337+0000","last_peered":"2024-05-08T23:18:43.794337+0000","last_clean":"2024-05-08T23:18:43.794337+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:43.794337+0000","last_undegraded":"2024-05-08T23:18:43.794337+0000","last_fullsized":"2024-05-08T23:18:43.794337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":336,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796725+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:43.796725+0000","last_peered":"2024-05-08T23:18:43.796725+0000","last_clean":"2024-05-08T23:18:43.796725+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:43.796725+0000","last_undegraded":"2024-05-08T23:18:43.796725+0000","last_fullsized":"2024-05-08T23:18:43.796725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795283+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:18:43.795283+0000","last_peered":"2024-05-08T23:18:43.795283+0000","last_clean":"2024-05-08T23:18:43.795283+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:18:43.795283+0000","last_undegraded":"2024-05-08T23:18:43.795283+0000","last_fullsized":"2024-05-08T23:18:43.795283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.976441+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:19:37.976441+0000","last_peered":"2024-05-08T23:19:37.976441+0000","last_clean":"2024-05-08T23:19:37.976441+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:19:37.976441+0000","last_undegraded":"2024-05-08T23:19:37.976441+0000","last_fullsized":"2024-05-08T23:19:37.976441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796803+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:43.796803+0000","last_peered":"2024-05-08T23:18:43.796803+0000","last_clean":"2024-05-08T23:18:43.796803+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:43.796803+0000","last_undegraded":"2024-05-08T23:18:43.796803+0000","last_fullsized":"2024-05-08T23:18:43.796803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798937+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:43.798937+0000","last_peered":"2024-05-08T23:18:43.798937+0000","last_clean":"2024-05-08T23:18:43.798937+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:43.798937+0000","last_undegraded":"2024-05-08T23:18:43.798937+0000","last_fullsized":"2024-05-08T23:18:43.798937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804873+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:43.804873+0000","last_peered":"2024-05-08T23:18:43.804873+0000","last_clean":"2024-05-08T23:18:43.804873+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:43.804873+0000","last_undegraded":"2024-05-08T23:18:43.804873+0000","last_fullsized":"2024-05-08T23:18:43.804873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.983517+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:19:37.983517+0000","last_peered":"2024-05-08T23:19:37.983517+0000","last_clean":"2024-05-08T23:19:37.983517+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:19:37.983517+0000","last_undegraded":"2024-05-08T23:19:37.983517+0000","last_fullsized":"2024-05-08T23:19:37.983517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793564+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:43.793564+0000","last_peered":"2024-05-08T23:18:43.793564+0000","last_clean":"2024-05-08T23:18:43.793564+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:43.793564+0000","last_undegraded":"2024-05-08T23:18:43.793564+0000","last_fullsized":"2024-05-08T23:18:43.793564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'40","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.200521+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:19:38.200521+0000","last_peered":"2024-05-08T23:19:38.200521+0000","last_clean":"2024-05-08T23:19:38.200521+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:19:38.200521+0000","last_undegraded":"2024-05-08T23:19:38.200521+0000","last_fullsized":"2024-05-08T23:19:38.200521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":71,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974398+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:19:37.974398+0000","last_peered":"2024-05-08T23:19:37.974398+0000","last_clean":"2024-05-08T23:19:37.974398+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:19:37.974398+0000","last_undegraded":"2024-05-08T23:19:37.974398+0000","last_fullsized":"2024-05-08T23:19:37.974398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795943+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:43.795943+0000","last_peered":"2024-05-08T23:18:43.795943+0000","last_clean":"2024-05-08T23:18:43.795943+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:43.795943+0000","last_undegraded":"2024-05-08T23:18:43.795943+0000","last_fullsized":"2024-05-08T23:18:43.795943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794461+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:43.794461+0000","last_peered":"2024-05-08T23:18:43.794461+0000","last_clean":"2024-05-08T23:18:43.794461+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:43.794461+0000","last_undegraded":"2024-05-08T23:18:43.794461+0000","last_fullsized":"2024-05-08T23:18:43.794461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.964151+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:19:37.964151+0000","last_peered":"2024-05-08T23:19:37.964151+0000","last_clean":"2024-05-08T23:19:37.964151+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:19:37.964151+0000","last_undegraded":"2024-05-08T23:19:37.964151+0000","last_fullsized":"2024-05-08T23:19:37.964151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793979+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:43.793979+0000","last_peered":"2024-05-08T23:18:43.793979+0000","last_clean":"2024-05-08T23:18:43.793979+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:43.793979+0000","last_undegraded":"2024-05-08T23:18:43.793979+0000","last_fullsized":"2024-05-08T23:18:43.793979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.927256+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:43.927256+0000","last_peered":"2024-05-08T23:18:43.927256+0000","last_clean":"2024-05-08T23:18:43.927256+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:43.927256+0000","last_undegraded":"2024-05-08T23:18:43.927256+0000","last_fullsized":"2024-05-08T23:18:43.927256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.966650+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:19:37.966650+0000","last_peered":"2024-05-08T23:19:37.966650+0000","last_clean":"2024-05-08T23:19:37.966650+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:19:37.966650+0000","last_undegraded":"2024-05-08T23:19:37.966650+0000","last_fullsized":"2024-05-08T23:19:37.966650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034275+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:44.034275+0000","last_peered":"2024-05-08T23:18:44.034275+0000","last_clean":"2024-05-08T23:18:44.034275+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:44.034275+0000","last_undegraded":"2024-05-08T23:18:44.034275+0000","last_fullsized":"2024-05-08T23:18:44.034275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793711+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:43.793711+0000","last_peered":"2024-05-08T23:18:43.793711+0000","last_clean":"2024-05-08T23:18:43.793711+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:43.793711+0000","last_undegraded":"2024-05-08T23:18:43.793711+0000","last_fullsized":"2024-05-08T23:18:43.793711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975082+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:19:37.975082+0000","last_peered":"2024-05-08T23:19:37.975082+0000","last_clean":"2024-05-08T23:19:37.975082+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:19:37.975082+0000","last_undegraded":"2024-05-08T23:19:37.975082+0000","last_fullsized":"2024-05-08T23:19:37.975082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793856+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:43.793856+0000","last_peered":"2024-05-08T23:18:43.793856+0000","last_clean":"2024-05-08T23:18:43.793856+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:43.793856+0000","last_undegraded":"2024-05-08T23:18:43.793856+0000","last_fullsized":"2024-05-08T23:18:43.793856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796671+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:43.796671+0000","last_peered":"2024-05-08T23:18:43.796671+0000","last_clean":"2024-05-08T23:18:43.796671+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:43.796671+0000","last_undegraded":"2024-05-08T23:18:43.796671+0000","last_fullsized":"2024-05-08T23:18:43.796671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.967303+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:19:37.967303+0000","last_peered":"2024-05-08T23:19:37.967303+0000","last_clean":"2024-05-08T23:19:37.967303+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:19:37.967303+0000","last_undegraded":"2024-05-08T23:19:37.967303+0000","last_fullsized":"2024-05-08T23:19:37.967303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794095+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:43.794095+0000","last_peered":"2024-05-08T23:18:43.794095+0000","last_clean":"2024-05-08T23:18:43.794095+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:43.794095+0000","last_undegraded":"2024-05-08T23:18:43.794095+0000","last_fullsized":"2024-05-08T23:18:43.794095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.635532+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:45.635532+0000","last_peered":"2024-05-08T23:18:45.635532+0000","last_clean":"2024-05-08T23:18:45.635532+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:45.635532+0000","last_undegraded":"2024-05-08T23:18:45.635532+0000","last_fullsized":"2024-05-08T23:18:45.635532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985346+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:19:37.985346+0000","last_peered":"2024-05-08T23:19:37.985346+0000","last_clean":"2024-05-08T23:19:37.985346+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:19:37.985346+0000","last_undegraded":"2024-05-08T23:19:37.985346+0000","last_fullsized":"2024-05-08T23:19:37.985346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.927268+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:43.927268+0000","last_peered":"2024-05-08T23:18:43.927268+0000","last_clean":"2024-05-08T23:18:43.927268+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:43.927268+0000","last_undegraded":"2024-05-08T23:18:43.927268+0000","last_fullsized":"2024-05-08T23:18:43.927268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797148+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:43.797148+0000","last_peered":"2024-05-08T23:18:43.797148+0000","last_clean":"2024-05-08T23:18:43.797148+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:43.797148+0000","last_undegraded":"2024-05-08T23:18:43.797148+0000","last_fullsized":"2024-05-08T23:18:43.797148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980423+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:19:37.980423+0000","last_peered":"2024-05-08T23:19:37.980423+0000","last_clean":"2024-05-08T23:19:37.980423+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:19:37.980423+0000","last_undegraded":"2024-05-08T23:19:37.980423+0000","last_fullsized":"2024-05-08T23:19:37.980423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.091302+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:44.091302+0000","last_peered":"2024-05-08T23:18:44.091302+0000","last_clean":"2024-05-08T23:18:44.091302+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:44.091302+0000","last_undegraded":"2024-05-08T23:18:44.091302+0000","last_fullsized":"2024-05-08T23:18:44.091302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.197927+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:19:38.197927+0000","last_peered":"2024-05-08T23:19:38.197927+0000","last_clean":"2024-05-08T23:19:38.197927+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:19:38.197927+0000","last_undegraded":"2024-05-08T23:19:38.197927+0000","last_fullsized":"2024-05-08T23:19:38.197927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979495+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:19:37.979495+0000","last_peered":"2024-05-08T23:19:37.979495+0000","last_clean":"2024-05-08T23:19:37.979495+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:19:37.979495+0000","last_undegraded":"2024-05-08T23:19:37.979495+0000","last_fullsized":"2024-05-08T23:19:37.979495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794917+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:18:43.794917+0000","last_peered":"2024-05-08T23:18:43.794917+0000","last_clean":"2024-05-08T23:18:43.794917+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:18:43.794917+0000","last_undegraded":"2024-05-08T23:18:43.794917+0000","last_fullsized":"2024-05-08T23:18:43.794917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931822+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931822+0000","last_peered":"2024-05-08T23:18:43.931822+0000","last_clean":"2024-05-08T23:18:43.931822+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931822+0000","last_undegraded":"2024-05-08T23:18:43.931822+0000","last_fullsized":"2024-05-08T23:18:43.931822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.972088+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:19:37.972088+0000","last_peered":"2024-05-08T23:19:37.972088+0000","last_clean":"2024-05-08T23:19:37.972088+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:19:37.972088+0000","last_undegraded":"2024-05-08T23:19:37.972088+0000","last_fullsized":"2024-05-08T23:19:37.972088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974459+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:19:37.974459+0000","last_peered":"2024-05-08T23:19:37.974459+0000","last_clean":"2024-05-08T23:19:37.974459+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:19:37.974459+0000","last_undegraded":"2024-05-08T23:19:37.974459+0000","last_fullsized":"2024-05-08T23:19:37.974459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979844+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:19:37.979844+0000","last_peered":"2024-05-08T23:19:37.979844+0000","last_clean":"2024-05-08T23:19:37.979844+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:19:37.979844+0000","last_undegraded":"2024-05-08T23:19:37.979844+0000","last_fullsized":"2024-05-08T23:19:37.979844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.963986+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:19:37.963986+0000","last_peered":"2024-05-08T23:19:37.963986+0000","last_clean":"2024-05-08T23:19:37.963986+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:19:37.963986+0000","last_undegraded":"2024-05-08T23:19:37.963986+0000","last_fullsized":"2024-05-08T23:19:37.963986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985739+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:19:37.985739+0000","last_peered":"2024-05-08T23:19:37.985739+0000","last_clean":"2024-05-08T23:19:37.985739+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:19:37.985739+0000","last_undegraded":"2024-05-08T23:19:37.985739+0000","last_fullsized":"2024-05-08T23:19:37.985739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982612+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:19:37.982612+0000","last_peered":"2024-05-08T23:19:37.982612+0000","last_clean":"2024-05-08T23:19:37.982612+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:19:37.982612+0000","last_undegraded":"2024-05-08T23:19:37.982612+0000","last_fullsized":"2024-05-08T23:19:37.982612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973494+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:19:37.973494+0000","last_peered":"2024-05-08T23:19:37.973494+0000","last_clean":"2024-05-08T23:19:37.973494+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:19:37.973494+0000","last_undegraded":"2024-05-08T23:19:37.973494+0000","last_fullsized":"2024-05-08T23:19:37.973494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.972939+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:19:37.972939+0000","last_peered":"2024-05-08T23:19:37.972939+0000","last_clean":"2024-05-08T23:19:37.972939+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:19:37.972939+0000","last_undegraded":"2024-05-08T23:19:37.972939+0000","last_fullsized":"2024-05-08T23:19:37.972939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.960851+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:19:37.960851+0000","last_peered":"2024-05-08T23:19:37.960851+0000","last_clean":"2024-05-08T23:19:37.960851+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:19:37.960851+0000","last_undegraded":"2024-05-08T23:19:37.960851+0000","last_fullsized":"2024-05-08T23:19:37.960851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978702+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:19:37.978702+0000","last_peered":"2024-05-08T23:19:37.978702+0000","last_clean":"2024-05-08T23:19:37.978702+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:19:37.978702+0000","last_undegraded":"2024-05-08T23:19:37.978702+0000","last_fullsized":"2024-05-08T23:19:37.978702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.966116+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:19:37.966116+0000","last_peered":"2024-05-08T23:19:37.966116+0000","last_clean":"2024-05-08T23:19:37.966116+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:19:37.966116+0000","last_undegraded":"2024-05-08T23:19:37.966116+0000","last_fullsized":"2024-05-08T23:19:37.966116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982523+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:19:37.982523+0000","last_peered":"2024-05-08T23:19:37.982523+0000","last_clean":"2024-05-08T23:19:37.982523+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:19:37.982523+0000","last_undegraded":"2024-05-08T23:19:37.982523+0000","last_fullsized":"2024-05-08T23:19:37.982523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.986378+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:19:37.986378+0000","last_peered":"2024-05-08T23:19:37.986378+0000","last_clean":"2024-05-08T23:19:37.986378+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:19:37.986378+0000","last_undegraded":"2024-05-08T23:19:37.986378+0000","last_fullsized":"2024-05-08T23:19:37.986378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977581+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:19:37.977581+0000","last_peered":"2024-05-08T23:19:37.977581+0000","last_clean":"2024-05-08T23:19:37.977581+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:19:37.977581+0000","last_undegraded":"2024-05-08T23:19:37.977581+0000","last_fullsized":"2024-05-08T23:19:37.977581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593571,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":642,"num_read_kb":426,"num_write":278,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1780713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645344,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":950272,"data_stored":472912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48399999999999999}]},{"osd":1,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93300000000000005,"5min":1.163,"15min":1.163},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.749,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.379}]},{"osd":2,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45000000000000001},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.47299999999999998},"min":{"1min":0.37,"5min":0.311,"15min":0.311},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81399999999999995,"5min":1.181,"15min":1.181},"last":0.433}]},{"osd":5,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.45800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98599999999999999,"5min":1.075,"15min":1.075},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":1.038,"15min":1.038},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.59299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.875,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56299999999999994}]},{"osd":9,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76300000000000001,"15min":0.75800000000000001},"min":{"1min":0.55700000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.75,"15min":0.73699999999999999},"min":{"1min":0.71099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.032,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94899999999999995,"5min":1.216,"15min":1.216},"last":0.62},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.61599999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.61599999999999999}]}]},{"osd":10,"up_from":65,"seq":279172874276,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37060,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708092,"statfs":{"total":95995035648,"available":95957086208,"internally_reserved":0,"allocated":1773568,"data_stored":1255360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.374}]},{"osd":1,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.66800000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.61499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.871,"15min":0.871},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.62,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.746,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72999999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.77200000000000002,"5min":1.169,"15min":1.169},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81599999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.5},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.84299999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83699999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.77700000000000002,"5min":1.016,"15min":1.016},"last":0.60899999999999999}]},{"osd":9,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79800000000000004,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86199999999999999,"5min":1.3,"15min":1.3},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.76200000000000001,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83699999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56000000000000005}]},{"osd":11,"last update":"Wed May 8 23:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.57999999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.87,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95299999999999996,"5min":1.083,"15min":1.083},"last":0.63700000000000001}]}]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1363968,"data_stored":870460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42299999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.504},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.58299999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60499999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0660000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70599999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.66400000000000003}]}]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28344,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716808,"statfs":{"total":95995035648,"available":95966011392,"internally_reserved":0,"allocated":1359872,"data_stored":866284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83199999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.51600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.317},{"interface":"front","average":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":1.137,"15min":1.137},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84399999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.503,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.5}]},{"osd":5,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78400000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85599999999999998,"5min":0.876,"15min":0.876},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.65500000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95499999999999996,"5min":1.042,"15min":1.042},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.71199999999999997},"min":{"1min":0.57999999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94699999999999995,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.623},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94599999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.93600000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63300000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68300000000000005}]}]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1556480,"data_stored":1070723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.38},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.433},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81399999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.77800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.627}]},{"osd":6,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69399999999999995,"15min":0.70899999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56599999999999995}]},{"osd":8,"last update":"Wed May 8 23:19:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.59899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.82599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.622,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.81599999999999995,"5min":0.875,"15min":0.875},"last":0.63800000000000001}]},{"osd":9,"last update":"Wed May 8 23:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.878,"5min":0.997,"15min":0.997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.752}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86799999999999999,"5min":0.999,"15min":0.999},"last":0.59199999999999997}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68300000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":946176,"data_stored":468324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.623}]},{"osd":7,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.501,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0309999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.177,"15min":1.177},"last":0.53800000000000003}]},{"osd":9,"last update":"Wed May 8 23:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.433}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.79700000000000004,"15min":0.79000000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":1015808,"data_stored":530911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.495},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82499999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.50900000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.40999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44700000000000001,"15min":0.40400000000000003},"min":{"1min":0.376,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.45600000000000002,"15min":0.41199999999999998},"min":{"1min":0.371,"5min":0.248,"15min":0.248},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.627,"15min":0.56699999999999995},"min":{"1min":0.53300000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.63300000000000001,"15min":0.57199999999999995},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.58299999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.621,"15min":0.59299999999999997},"min":{"1min":0.54700000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.504},"min":{"1min":0.44700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.374},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.35899999999999999}]},{"osd":8,"last update":"Wed May 8 23:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64800000000000002},"min":{"1min":0.48799999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.57799999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.014,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.39600000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53700000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.59399999999999997}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":970752,"data_stored":469712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89000000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Wed May 8 23:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.55300000000000005},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.66000000000000003,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.63},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66100000000000003,"15min":0.629},"min":{"1min":0.502,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66000000000000003}]},{"osd":8,"last update":"Wed May 8 23:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72399999999999998,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.996,"5min":1.014,"15min":1.014},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.755}]},{"osd":10,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":0.997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0740000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68500000000000005}]},{"osd":11,"last update":"Wed May 8 23:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0820000000000001,"5min":1.246,"15min":1.246},"last":0.78000000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":531923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.873,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.60399999999999998,"15min":0.54000000000000004},"min":{"1min":0.54500000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 8 23:19:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83099999999999996,"5min":0.872,"15min":0.872},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.754,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.58699999999999997},"min":{"1min":0.52900000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":8,"last update":"Wed May 8 23:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.433,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.61299999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50800000000000001}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.55400000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.67700000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63600000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64500000000000002}]}]},{"osd":3,"up_from":23,"seq":98784247868,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36660,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708492,"statfs":{"total":95995035648,"available":95957495808,"internally_reserved":0,"allocated":1486848,"data_stored":993334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.436,"15min":0.36799999999999999},"min":{"1min":0.32600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.83199999999999996,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.441,"15min":0.38600000000000001},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59399999999999997},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.98499999999999999,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.57499999999999996},"min":{"1min":0.51400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.54500000000000004},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.105,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed May 8 23:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0469999999999999,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.497},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.621},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed May 8 23:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96099999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.997,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.624,"15min":0.58799999999999997},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91900000000000004,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.622},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.44700000000000001}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94499999999999995,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1879999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.73099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:19:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0920000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed May 8 23:18:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.92200000000000004,"15min":0.94799999999999995},"min":{"1min":0.65300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.135,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.97899999999999998,"15min":1.0049999999999999},"min":{"1min":0.67800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.171,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.74299999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65400000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018936,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":473422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.34100000000000003},"min":{"1min":0.32900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39500000000000002,"15min":0.36699999999999999},"min":{"1min":0.33600000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.58299999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.48799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.44500000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67200000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":0.874,"15min":0.874},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.44700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38}]},{"osd":5,"last update":"Wed May 8 23:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72299999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.28999999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65000000000000002,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.498}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60099999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81100000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41099999999999998}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55800000000000005}]}]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1703936,"data_stored":1197008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.372},{"interface":"front","average":{"1min":0.496,"5min":0.53300000000000003,"15min":0.56499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.434},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.27500000000000002}]},{"osd":3,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71999999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.35599999999999998}]},{"osd":6,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.443},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.433}]},{"osd":7,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79500000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40999999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47899999999999998}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55400000000000005}]},{"osd":10,"last update":"Wed May 8 23:18:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.56200000000000006,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":197089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:19:44.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:19:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 -- 172.21.15.16:0/3337160937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c102010 msgr2=0x7efc6c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 --2- 172.21.15.16:0/3337160937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c102010 0x7efc6c1023f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009990 tx=0x7efc5c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 -- 172.21.15.16:0/3337160937 shutdown_connections 2024-05-08T23:19:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 --2- 172.21.15.16:0/3337160937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc6c10b480 0x7efc6c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 --2- 172.21.15.16:0/3337160937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c102010 0x7efc6c1023f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 --2- 172.21.15.16:0/3337160937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc6c102930 0x7efc6c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 -- 172.21.15.16:0/3337160937 >> 172.21.15.16:0/3337160937 conn(0x7efc6c0fb7f0 msgr2=0x7efc6c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 -- 172.21.15.16:0/3337160937 shutdown_connections 2024-05-08T23:19:45.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.044+0000 7efc70b20700 1 -- 172.21.15.16:0/3337160937 wait complete. 2024-05-08T23:19:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 Processor -- start 2024-05-08T23:19:45.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 -- start start 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc6c102010 0x7efc6c0ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc6c102930 0x7efc6c1001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 0x7efc6c1006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c1a5610 con 0x7efc6c10b480 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c1a5780 con 0x7efc6c102930 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.045+0000 7efc70b20700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c1a5940 con 0x7efc6c102010 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 0x7efc6c1006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 0x7efc6c1006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60404/0 (socket says 172.21.15.16:60404) 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 -- 172.21.15.16:0/3092568591 learned_addr learned my addr 172.21.15.16:0/3092568591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6a59c700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc6c102010 0x7efc6c0ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc69d9b700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc6c102930 0x7efc6c1001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 -- 172.21.15.16:0/3092568591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc6c102010 msgr2=0x7efc6c0ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc6c102010 0x7efc6c0ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 -- 172.21.15.16:0/3092568591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc6c102930 msgr2=0x7efc6c1001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc6c102930 0x7efc6c1001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 -- 172.21.15.16:0/3092568591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0095f0 con 0x7efc6c10b480 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc6ad9d700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 0x7efc6c1006e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efc64007ec0 tx=0x7efc6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64017070 con 0x7efc6c10b480 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc6400f040 con 0x7efc6c10b480 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6c1a5c20 con 0x7efc6c10b480 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64004280 con 0x7efc6c10b480 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.046+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc6c1a6130 con 0x7efc6c10b480 2024-05-08T23:19:45.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.048+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6c106560 con 0x7efc6c10b480 2024-05-08T23:19:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.048+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7efc64004ad0 con 0x7efc6c10b480 2024-05-08T23:19:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.049+0000 7efc637fe700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc580a13a0 0x7efc580a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.049+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7efc640cc750 con 0x7efc6c10b480 2024-05-08T23:19:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.049+0000 7efc6a59c700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc580a13a0 0x7efc580a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.050+0000 7efc6a59c700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc580a13a0 0x7efc580a3860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b5c0 tx=0x7efc5c005fd0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:45.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.054+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc640957b0 con 0x7efc6c10b480 2024-05-08T23:19:45.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:44 smithi016 ceph-mon[42541]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/890899036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4143482905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:44 smithi155 ceph-mon[47105]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/890899036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4143482905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.267+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc6c1010f0 con 0x7efc580a13a0 2024-05-08T23:19:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.280+0000 7efc637fe700 1 -- 172.21.15.16:0/3092568591 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+425554 (secure 0 0 0) 0x7efc6c1010f0 con 0x7efc580a13a0 2024-05-08T23:19:45.286 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc580a13a0 msgr2=0x7efc580a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc580a13a0 0x7efc580a3860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b5c0 tx=0x7efc5c005fd0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 msgr2=0x7efc6c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 0x7efc6c1006e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efc64007ec0 tx=0x7efc6400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 shutdown_connections 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc580a13a0 0x7efc580a3860 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c10b480 0x7efc6c1006e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc6c102010 0x7efc6c0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 --2- 172.21.15.16:0/3092568591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc6c102930 0x7efc6c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 >> 172.21.15.16:0/3092568591 conn(0x7efc6c0fb7f0 msgr2=0x7efc6c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 shutdown_connections 2024-05-08T23:19:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.284+0000 7efc70b20700 1 -- 172.21.15.16:0/3092568591 wait complete. 2024-05-08T23:19:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:19:45.299 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-08T23:19:43.939958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184173,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":778,"num_read_kb":545,"num_write":461,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400548,"kb_used_data":14768,"kb_used_omap":18,"kb_used_meta":385517,"kb_avail":1124541276,"statfs":{"total":1151940427776,"available":1151530266624,"internally_reserved":0,"allocated":15122432,"data_stored":9200373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":394769736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989242"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979175+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:19:37.979175+0000","last_peered":"2024-05-08T23:19:37.979175+0000","last_clean":"2024-05-08T23:19:37.979175+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:19:37.979175+0000","last_undegraded":"2024-05-08T23:19:37.979175+0000","last_fullsized":"2024-05-08T23:19:37.979175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975013+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:19:37.975013+0000","last_peered":"2024-05-08T23:19:37.975013+0000","last_clean":"2024-05-08T23:19:37.975013+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:19:37.975013+0000","last_undegraded":"2024-05-08T23:19:37.975013+0000","last_fullsized":"2024-05-08T23:19:37.975013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978939+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:19:37.978939+0000","last_peered":"2024-05-08T23:19:37.978939+0000","last_clean":"2024-05-08T23:19:37.978939+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:19:37.978939+0000","last_undegraded":"2024-05-08T23:19:37.978939+0000","last_fullsized":"2024-05-08T23:19:37.978939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977387+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:19:37.977387+0000","last_peered":"2024-05-08T23:19:37.977387+0000","last_clean":"2024-05-08T23:19:37.977387+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:19:37.977387+0000","last_undegraded":"2024-05-08T23:19:37.977387+0000","last_fullsized":"2024-05-08T23:19:37.977387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974352+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:19:37.974352+0000","last_peered":"2024-05-08T23:19:37.974352+0000","last_clean":"2024-05-08T23:19:37.974352+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:19:37.974352+0000","last_undegraded":"2024-05-08T23:19:37.974352+0000","last_fullsized":"2024-05-08T23:19:37.974352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980276+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:19:37.980276+0000","last_peered":"2024-05-08T23:19:37.980276+0000","last_clean":"2024-05-08T23:19:37.980276+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:19:37.980276+0000","last_undegraded":"2024-05-08T23:19:37.980276+0000","last_fullsized":"2024-05-08T23:19:37.980276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978937+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:19:37.978937+0000","last_peered":"2024-05-08T23:19:37.978937+0000","last_clean":"2024-05-08T23:19:37.978937+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:19:37.978937+0000","last_undegraded":"2024-05-08T23:19:37.978937+0000","last_fullsized":"2024-05-08T23:19:37.978937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.970401+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:19:37.970401+0000","last_peered":"2024-05-08T23:19:37.970401+0000","last_clean":"2024-05-08T23:19:37.970401+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:19:37.970401+0000","last_undegraded":"2024-05-08T23:19:37.970401+0000","last_fullsized":"2024-05-08T23:19:37.970401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977019+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:19:37.977019+0000","last_peered":"2024-05-08T23:19:37.977019+0000","last_clean":"2024-05-08T23:19:37.977019+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:19:37.977019+0000","last_undegraded":"2024-05-08T23:19:37.977019+0000","last_fullsized":"2024-05-08T23:19:37.977019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.984147+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:19:37.984147+0000","last_peered":"2024-05-08T23:19:37.984147+0000","last_clean":"2024-05-08T23:19:37.984147+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:19:37.984147+0000","last_undegraded":"2024-05-08T23:19:37.984147+0000","last_fullsized":"2024-05-08T23:19:37.984147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980710+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:19:37.980710+0000","last_peered":"2024-05-08T23:19:37.980710+0000","last_clean":"2024-05-08T23:19:37.980710+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:19:37.980710+0000","last_undegraded":"2024-05-08T23:19:37.980710+0000","last_fullsized":"2024-05-08T23:19:37.980710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.967635+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:19:37.967635+0000","last_peered":"2024-05-08T23:19:37.967635+0000","last_clean":"2024-05-08T23:19:37.967635+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:19:37.967635+0000","last_undegraded":"2024-05-08T23:19:37.967635+0000","last_fullsized":"2024-05-08T23:19:37.967635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985493+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:19:37.985493+0000","last_peered":"2024-05-08T23:19:37.985493+0000","last_clean":"2024-05-08T23:19:37.985493+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:19:37.985493+0000","last_undegraded":"2024-05-08T23:19:37.985493+0000","last_fullsized":"2024-05-08T23:19:37.985493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.971337+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:19:37.971337+0000","last_peered":"2024-05-08T23:19:37.971337+0000","last_clean":"2024-05-08T23:19:37.971337+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:19:37.971337+0000","last_undegraded":"2024-05-08T23:19:37.971337+0000","last_fullsized":"2024-05-08T23:19:37.971337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978114+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:19:37.978114+0000","last_peered":"2024-05-08T23:19:37.978114+0000","last_clean":"2024-05-08T23:19:37.978114+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:19:37.978114+0000","last_undegraded":"2024-05-08T23:19:37.978114+0000","last_fullsized":"2024-05-08T23:19:37.978114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982276+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:19:37.982276+0000","last_peered":"2024-05-08T23:19:37.982276+0000","last_clean":"2024-05-08T23:19:37.982276+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:19:37.982276+0000","last_undegraded":"2024-05-08T23:19:37.982276+0000","last_fullsized":"2024-05-08T23:19:37.982276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979259+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:19:37.979259+0000","last_peered":"2024-05-08T23:19:37.979259+0000","last_clean":"2024-05-08T23:19:37.979259+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:19:37.979259+0000","last_undegraded":"2024-05-08T23:19:37.979259+0000","last_fullsized":"2024-05-08T23:19:37.979259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979567+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:19:37.979567+0000","last_peered":"2024-05-08T23:19:37.979567+0000","last_clean":"2024-05-08T23:19:37.979567+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:19:37.979567+0000","last_undegraded":"2024-05-08T23:19:37.979567+0000","last_fullsized":"2024-05-08T23:19:37.979567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.976723+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:19:37.976723+0000","last_peered":"2024-05-08T23:19:37.976723+0000","last_clean":"2024-05-08T23:19:37.976723+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:19:37.976723+0000","last_undegraded":"2024-05-08T23:19:37.976723+0000","last_fullsized":"2024-05-08T23:19:37.976723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796041+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:43.796041+0000","last_peered":"2024-05-08T23:18:43.796041+0000","last_clean":"2024-05-08T23:18:43.796041+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:43.796041+0000","last_undegraded":"2024-05-08T23:18:43.796041+0000","last_fullsized":"2024-05-08T23:18:43.796041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034288+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:44.034288+0000","last_peered":"2024-05-08T23:18:44.034288+0000","last_clean":"2024-05-08T23:18:44.034288+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:44.034288+0000","last_undegraded":"2024-05-08T23:18:44.034288+0000","last_fullsized":"2024-05-08T23:18:44.034288+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.964536+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:19:37.964536+0000","last_peered":"2024-05-08T23:19:37.964536+0000","last_clean":"2024-05-08T23:19:37.964536+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:19:37.964536+0000","last_undegraded":"2024-05-08T23:19:37.964536+0000","last_fullsized":"2024-05-08T23:19:37.964536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793688+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:43.793688+0000","last_peered":"2024-05-08T23:18:43.793688+0000","last_clean":"2024-05-08T23:18:43.793688+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:43.793688+0000","last_undegraded":"2024-05-08T23:18:43.793688+0000","last_fullsized":"2024-05-08T23:18:43.793688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794148+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:43.794148+0000","last_peered":"2024-05-08T23:18:43.794148+0000","last_clean":"2024-05-08T23:18:43.794148+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:43.794148+0000","last_undegraded":"2024-05-08T23:18:43.794148+0000","last_fullsized":"2024-05-08T23:18:43.794148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.983386+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:19:37.983386+0000","last_peered":"2024-05-08T23:19:37.983386+0000","last_clean":"2024-05-08T23:19:37.983386+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:19:37.983386+0000","last_undegraded":"2024-05-08T23:19:37.983386+0000","last_fullsized":"2024-05-08T23:19:37.983386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803701+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803701+0000","last_peered":"2024-05-08T23:18:43.803701+0000","last_clean":"2024-05-08T23:18:43.803701+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803701+0000","last_undegraded":"2024-05-08T23:18:43.803701+0000","last_fullsized":"2024-05-08T23:18:43.803701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.635572+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:45.635572+0000","last_peered":"2024-05-08T23:18:45.635572+0000","last_clean":"2024-05-08T23:18:45.635572+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:45.635572+0000","last_undegraded":"2024-05-08T23:18:45.635572+0000","last_fullsized":"2024-05-08T23:18:45.635572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979236+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:19:37.979236+0000","last_peered":"2024-05-08T23:19:37.979236+0000","last_clean":"2024-05-08T23:19:37.979236+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:19:37.979236+0000","last_undegraded":"2024-05-08T23:19:37.979236+0000","last_fullsized":"2024-05-08T23:19:37.979236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793735+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:43.793735+0000","last_peered":"2024-05-08T23:18:43.793735+0000","last_clean":"2024-05-08T23:18:43.793735+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:43.793735+0000","last_undegraded":"2024-05-08T23:18:43.793735+0000","last_fullsized":"2024-05-08T23:18:43.793735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804752+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:43.804752+0000","last_peered":"2024-05-08T23:18:43.804752+0000","last_clean":"2024-05-08T23:18:43.804752+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:43.804752+0000","last_undegraded":"2024-05-08T23:18:43.804752+0000","last_fullsized":"2024-05-08T23:18:43.804752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.963335+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:19:37.963335+0000","last_peered":"2024-05-08T23:19:37.963335+0000","last_clean":"2024-05-08T23:19:37.963335+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:19:37.963335+0000","last_undegraded":"2024-05-08T23:19:37.963335+0000","last_fullsized":"2024-05-08T23:19:37.963335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795805+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:43.795805+0000","last_peered":"2024-05-08T23:18:43.795805+0000","last_clean":"2024-05-08T23:18:43.795805+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:43.795805+0000","last_undegraded":"2024-05-08T23:18:43.795805+0000","last_fullsized":"2024-05-08T23:18:43.795805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":117,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818584+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:19:38.818584+0000","last_peered":"2024-05-08T23:19:38.818584+0000","last_clean":"2024-05-08T23:19:38.818584+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:19:38.818584+0000","last_undegraded":"2024-05-08T23:19:38.818584+0000","last_fullsized":"2024-05-08T23:19:38.818584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978785+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:19:37.978785+0000","last_peered":"2024-05-08T23:19:37.978785+0000","last_clean":"2024-05-08T23:19:37.978785+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:19:37.978785+0000","last_undegraded":"2024-05-08T23:19:37.978785+0000","last_fullsized":"2024-05-08T23:19:37.978785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803847+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803847+0000","last_peered":"2024-05-08T23:18:43.803847+0000","last_clean":"2024-05-08T23:18:43.803847+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803847+0000","last_undegraded":"2024-05-08T23:18:43.803847+0000","last_fullsized":"2024-05-08T23:18:43.803847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797144+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:43.797144+0000","last_peered":"2024-05-08T23:18:43.797144+0000","last_clean":"2024-05-08T23:18:43.797144+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:43.797144+0000","last_undegraded":"2024-05-08T23:18:43.797144+0000","last_fullsized":"2024-05-08T23:18:43.797144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.969192+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:19:37.969192+0000","last_peered":"2024-05-08T23:19:37.969192+0000","last_clean":"2024-05-08T23:19:37.969192+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:19:37.969192+0000","last_undegraded":"2024-05-08T23:19:37.969192+0000","last_fullsized":"2024-05-08T23:19:37.969192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798515+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:43.798515+0000","last_peered":"2024-05-08T23:18:43.798515+0000","last_clean":"2024-05-08T23:18:43.798515+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:43.798515+0000","last_undegraded":"2024-05-08T23:18:43.798515+0000","last_fullsized":"2024-05-08T23:18:43.798515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797721+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:43.797721+0000","last_peered":"2024-05-08T23:18:43.797721+0000","last_clean":"2024-05-08T23:18:43.797721+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:43.797721+0000","last_undegraded":"2024-05-08T23:18:43.797721+0000","last_fullsized":"2024-05-08T23:18:43.797721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977826+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:19:37.977826+0000","last_peered":"2024-05-08T23:19:37.977826+0000","last_clean":"2024-05-08T23:19:37.977826+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:19:37.977826+0000","last_undegraded":"2024-05-08T23:19:37.977826+0000","last_fullsized":"2024-05-08T23:19:37.977826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.091431+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:44.091431+0000","last_peered":"2024-05-08T23:18:44.091431+0000","last_clean":"2024-05-08T23:18:44.091431+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:44.091431+0000","last_undegraded":"2024-05-08T23:18:44.091431+0000","last_fullsized":"2024-05-08T23:18:44.091431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795525+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:43.795525+0000","last_peered":"2024-05-08T23:18:43.795525+0000","last_clean":"2024-05-08T23:18:43.795525+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:43.795525+0000","last_undegraded":"2024-05-08T23:18:43.795525+0000","last_fullsized":"2024-05-08T23:18:43.795525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982914+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:19:37.982914+0000","last_peered":"2024-05-08T23:19:37.982914+0000","last_clean":"2024-05-08T23:19:37.982914+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:19:37.982914+0000","last_undegraded":"2024-05-08T23:19:37.982914+0000","last_fullsized":"2024-05-08T23:19:37.982914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793690+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:43.793690+0000","last_peered":"2024-05-08T23:18:43.793690+0000","last_clean":"2024-05-08T23:18:43.793690+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:43.793690+0000","last_undegraded":"2024-05-08T23:18:43.793690+0000","last_fullsized":"2024-05-08T23:18:43.793690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":110,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.935026+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:19:37.935026+0000","last_peered":"2024-05-08T23:19:37.935026+0000","last_clean":"2024-05-08T23:19:37.935026+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:19:37.935026+0000","last_undegraded":"2024-05-08T23:19:37.935026+0000","last_fullsized":"2024-05-08T23:19:37.935026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.981390+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:19:37.981390+0000","last_peered":"2024-05-08T23:19:37.981390+0000","last_clean":"2024-05-08T23:19:37.981390+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:19:37.981390+0000","last_undegraded":"2024-05-08T23:19:37.981390+0000","last_fullsized":"2024-05-08T23:19:37.981390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.033348+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:44.033348+0000","last_peered":"2024-05-08T23:18:44.033348+0000","last_clean":"2024-05-08T23:18:44.033348+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:44.033348+0000","last_undegraded":"2024-05-08T23:18:44.033348+0000","last_fullsized":"2024-05-08T23:18:44.033348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797353+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:43.797353+0000","last_peered":"2024-05-08T23:18:43.797353+0000","last_clean":"2024-05-08T23:18:43.797353+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:43.797353+0000","last_undegraded":"2024-05-08T23:18:43.797353+0000","last_fullsized":"2024-05-08T23:18:43.797353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975947+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:19:37.975947+0000","last_peered":"2024-05-08T23:19:37.975947+0000","last_clean":"2024-05-08T23:19:37.975947+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:19:37.975947+0000","last_undegraded":"2024-05-08T23:19:37.975947+0000","last_fullsized":"2024-05-08T23:19:37.975947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794345+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:43.794345+0000","last_peered":"2024-05-08T23:18:43.794345+0000","last_clean":"2024-05-08T23:18:43.794345+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:43.794345+0000","last_undegraded":"2024-05-08T23:18:43.794345+0000","last_fullsized":"2024-05-08T23:18:43.794345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795045+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:43.795045+0000","last_peered":"2024-05-08T23:18:43.795045+0000","last_clean":"2024-05-08T23:18:43.795045+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:43.795045+0000","last_undegraded":"2024-05-08T23:18:43.795045+0000","last_fullsized":"2024-05-08T23:18:43.795045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982345+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:19:37.982345+0000","last_peered":"2024-05-08T23:19:37.982345+0000","last_clean":"2024-05-08T23:19:37.982345+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:19:37.982345+0000","last_undegraded":"2024-05-08T23:19:37.982345+0000","last_fullsized":"2024-05-08T23:19:37.982345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.033715+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:44.033715+0000","last_peered":"2024-05-08T23:18:44.033715+0000","last_clean":"2024-05-08T23:18:44.033715+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:44.033715+0000","last_undegraded":"2024-05-08T23:18:44.033715+0000","last_fullsized":"2024-05-08T23:18:44.033715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'38","reported_seq":240,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818239+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:19:38.818239+0000","last_peered":"2024-05-08T23:19:38.818239+0000","last_clean":"2024-05-08T23:19:38.818239+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:19:38.818239+0000","last_undegraded":"2024-05-08T23:19:38.818239+0000","last_fullsized":"2024-05-08T23:19:38.818239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62677,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":183,"num_read_kb":153,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973333+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:19:37.973333+0000","last_peered":"2024-05-08T23:19:37.973333+0000","last_clean":"2024-05-08T23:19:37.973333+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:19:37.973333+0000","last_undegraded":"2024-05-08T23:19:37.973333+0000","last_fullsized":"2024-05-08T23:19:37.973333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795420+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:18:43.795420+0000","last_peered":"2024-05-08T23:18:43.795420+0000","last_clean":"2024-05-08T23:18:43.795420+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:18:43.795420+0000","last_undegraded":"2024-05-08T23:18:43.795420+0000","last_fullsized":"2024-05-08T23:18:43.795420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796397+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:43.796397+0000","last_peered":"2024-05-08T23:18:43.796397+0000","last_clean":"2024-05-08T23:18:43.796397+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:43.796397+0000","last_undegraded":"2024-05-08T23:18:43.796397+0000","last_fullsized":"2024-05-08T23:18:43.796397+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985255+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:19:37.985255+0000","last_peered":"2024-05-08T23:19:37.985255+0000","last_clean":"2024-05-08T23:19:37.985255+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:19:37.985255+0000","last_undegraded":"2024-05-08T23:19:37.985255+0000","last_fullsized":"2024-05-08T23:19:37.985255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798559+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:43.798559+0000","last_peered":"2024-05-08T23:18:43.798559+0000","last_clean":"2024-05-08T23:18:43.798559+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:43.798559+0000","last_undegraded":"2024-05-08T23:18:43.798559+0000","last_fullsized":"2024-05-08T23:18:43.798559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795869+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:43.795869+0000","last_peered":"2024-05-08T23:18:43.795869+0000","last_clean":"2024-05-08T23:18:43.795869+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:43.795869+0000","last_undegraded":"2024-05-08T23:18:43.795869+0000","last_fullsized":"2024-05-08T23:18:43.795869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985393+0000","last_change":"2024-05-08T23:18:44.034932+0000","last_active":"2024-05-08T23:19:37.985393+0000","last_peered":"2024-05-08T23:19:37.985393+0000","last_clean":"2024-05-08T23:19:37.985393+0000","last_became_active":"2024-05-08T23:18:44.034439+0000","last_became_peered":"2024-05-08T23:18:44.034439+0000","last_unstale":"2024-05-08T23:19:37.985393+0000","last_undegraded":"2024-05-08T23:19:37.985393+0000","last_fullsized":"2024-05-08T23:19:37.985393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.536646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034119+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:44.034119+0000","last_peered":"2024-05-08T23:18:44.034119+0000","last_clean":"2024-05-08T23:18:44.034119+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:44.034119+0000","last_undegraded":"2024-05-08T23:18:44.034119+0000","last_fullsized":"2024-05-08T23:18:44.034119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818321+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:19:38.818321+0000","last_peered":"2024-05-08T23:19:38.818321+0000","last_clean":"2024-05-08T23:19:38.818321+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:19:38.818321+0000","last_undegraded":"2024-05-08T23:19:38.818321+0000","last_fullsized":"2024-05-08T23:19:38.818321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1822,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973136+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:19:37.973136+0000","last_peered":"2024-05-08T23:19:37.973136+0000","last_clean":"2024-05-08T23:19:37.973136+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:19:37.973136+0000","last_undegraded":"2024-05-08T23:19:37.973136+0000","last_fullsized":"2024-05-08T23:19:37.973136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798551+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:43.798551+0000","last_peered":"2024-05-08T23:18:43.798551+0000","last_clean":"2024-05-08T23:18:43.798551+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:43.798551+0000","last_undegraded":"2024-05-08T23:18:43.798551+0000","last_fullsized":"2024-05-08T23:18:43.798551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804250+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:43.804250+0000","last_peered":"2024-05-08T23:18:43.804250+0000","last_clean":"2024-05-08T23:18:43.804250+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:43.804250+0000","last_undegraded":"2024-05-08T23:18:43.804250+0000","last_fullsized":"2024-05-08T23:18:43.804250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.962016+0000","last_change":"2024-05-08T23:18:43.929400+0000","last_active":"2024-05-08T23:19:37.962016+0000","last_peered":"2024-05-08T23:19:37.962016+0000","last_clean":"2024-05-08T23:19:37.962016+0000","last_became_active":"2024-05-08T23:18:43.928980+0000","last_became_peered":"2024-05-08T23:18:43.928980+0000","last_unstale":"2024-05-08T23:19:37.962016+0000","last_undegraded":"2024-05-08T23:19:37.962016+0000","last_fullsized":"2024-05-08T23:19:37.962016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:10.433725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795293+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:18:43.795293+0000","last_peered":"2024-05-08T23:18:43.795293+0000","last_clean":"2024-05-08T23:18:43.795293+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:18:43.795293+0000","last_undegraded":"2024-05-08T23:18:43.795293+0000","last_fullsized":"2024-05-08T23:18:43.795293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804811+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:43.804811+0000","last_peered":"2024-05-08T23:18:43.804811+0000","last_clean":"2024-05-08T23:18:43.804811+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:43.804811+0000","last_undegraded":"2024-05-08T23:18:43.804811+0000","last_fullsized":"2024-05-08T23:18:43.804811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973319+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:19:37.973319+0000","last_peered":"2024-05-08T23:19:37.973319+0000","last_clean":"2024-05-08T23:19:37.973319+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:19:37.973319+0000","last_undegraded":"2024-05-08T23:19:37.973319+0000","last_fullsized":"2024-05-08T23:19:37.973319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804513+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:43.804513+0000","last_peered":"2024-05-08T23:18:43.804513+0000","last_clean":"2024-05-08T23:18:43.804513+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:43.804513+0000","last_undegraded":"2024-05-08T23:18:43.804513+0000","last_fullsized":"2024-05-08T23:18:43.804513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034555+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:44.034555+0000","last_peered":"2024-05-08T23:18:44.034555+0000","last_clean":"2024-05-08T23:18:44.034555+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:44.034555+0000","last_undegraded":"2024-05-08T23:18:44.034555+0000","last_fullsized":"2024-05-08T23:18:44.034555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979180+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:19:37.979180+0000","last_peered":"2024-05-08T23:19:37.979180+0000","last_clean":"2024-05-08T23:19:37.979180+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:19:37.979180+0000","last_undegraded":"2024-05-08T23:19:37.979180+0000","last_fullsized":"2024-05-08T23:19:37.979180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798335+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:43.798335+0000","last_peered":"2024-05-08T23:18:43.798335+0000","last_clean":"2024-05-08T23:18:43.798335+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:43.798335+0000","last_undegraded":"2024-05-08T23:18:43.798335+0000","last_fullsized":"2024-05-08T23:18:43.798335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797081+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:43.797081+0000","last_peered":"2024-05-08T23:18:43.797081+0000","last_clean":"2024-05-08T23:18:43.797081+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:43.797081+0000","last_undegraded":"2024-05-08T23:18:43.797081+0000","last_fullsized":"2024-05-08T23:18:43.797081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.970690+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:19:37.970690+0000","last_peered":"2024-05-08T23:19:37.970690+0000","last_clean":"2024-05-08T23:19:37.970690+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:19:37.970690+0000","last_undegraded":"2024-05-08T23:19:37.970690+0000","last_fullsized":"2024-05-08T23:19:37.970690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798132+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:43.798132+0000","last_peered":"2024-05-08T23:18:43.798132+0000","last_clean":"2024-05-08T23:18:43.798132+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:43.798132+0000","last_undegraded":"2024-05-08T23:18:43.798132+0000","last_fullsized":"2024-05-08T23:18:43.798132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973657+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:19:37.973657+0000","last_peered":"2024-05-08T23:19:37.973657+0000","last_clean":"2024-05-08T23:19:37.973657+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:19:37.973657+0000","last_undegraded":"2024-05-08T23:19:37.973657+0000","last_fullsized":"2024-05-08T23:19:37.973657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":111,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.987062+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:19:37.987062+0000","last_peered":"2024-05-08T23:19:37.987062+0000","last_clean":"2024-05-08T23:19:37.987062+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:19:37.987062+0000","last_undegraded":"2024-05-08T23:19:37.987062+0000","last_fullsized":"2024-05-08T23:19:37.987062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797552+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:43.797552+0000","last_peered":"2024-05-08T23:18:43.797552+0000","last_clean":"2024-05-08T23:18:43.797552+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:43.797552+0000","last_undegraded":"2024-05-08T23:18:43.797552+0000","last_fullsized":"2024-05-08T23:18:43.797552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979618+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:19:37.979618+0000","last_peered":"2024-05-08T23:19:37.979618+0000","last_clean":"2024-05-08T23:19:37.979618+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:19:37.979618+0000","last_undegraded":"2024-05-08T23:19:37.979618+0000","last_fullsized":"2024-05-08T23:19:37.979618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794337+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:43.794337+0000","last_peered":"2024-05-08T23:18:43.794337+0000","last_clean":"2024-05-08T23:18:43.794337+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:43.794337+0000","last_undegraded":"2024-05-08T23:18:43.794337+0000","last_fullsized":"2024-05-08T23:18:43.794337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":336,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796725+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:43.796725+0000","last_peered":"2024-05-08T23:18:43.796725+0000","last_clean":"2024-05-08T23:18:43.796725+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:43.796725+0000","last_undegraded":"2024-05-08T23:18:43.796725+0000","last_fullsized":"2024-05-08T23:18:43.796725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795283+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:18:43.795283+0000","last_peered":"2024-05-08T23:18:43.795283+0000","last_clean":"2024-05-08T23:18:43.795283+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:18:43.795283+0000","last_undegraded":"2024-05-08T23:18:43.795283+0000","last_fullsized":"2024-05-08T23:18:43.795283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.976441+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:19:37.976441+0000","last_peered":"2024-05-08T23:19:37.976441+0000","last_clean":"2024-05-08T23:19:37.976441+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:19:37.976441+0000","last_undegraded":"2024-05-08T23:19:37.976441+0000","last_fullsized":"2024-05-08T23:19:37.976441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796803+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:43.796803+0000","last_peered":"2024-05-08T23:18:43.796803+0000","last_clean":"2024-05-08T23:18:43.796803+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:43.796803+0000","last_undegraded":"2024-05-08T23:18:43.796803+0000","last_fullsized":"2024-05-08T23:18:43.796803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798937+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:43.798937+0000","last_peered":"2024-05-08T23:18:43.798937+0000","last_clean":"2024-05-08T23:18:43.798937+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:43.798937+0000","last_undegraded":"2024-05-08T23:18:43.798937+0000","last_fullsized":"2024-05-08T23:18:43.798937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804873+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:43.804873+0000","last_peered":"2024-05-08T23:18:43.804873+0000","last_clean":"2024-05-08T23:18:43.804873+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:43.804873+0000","last_undegraded":"2024-05-08T23:18:43.804873+0000","last_fullsized":"2024-05-08T23:18:43.804873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.983517+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:19:37.983517+0000","last_peered":"2024-05-08T23:19:37.983517+0000","last_clean":"2024-05-08T23:19:37.983517+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:19:37.983517+0000","last_undegraded":"2024-05-08T23:19:37.983517+0000","last_fullsized":"2024-05-08T23:19:37.983517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793564+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:43.793564+0000","last_peered":"2024-05-08T23:18:43.793564+0000","last_clean":"2024-05-08T23:18:43.793564+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:43.793564+0000","last_undegraded":"2024-05-08T23:18:43.793564+0000","last_fullsized":"2024-05-08T23:18:43.793564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'40","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.200521+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:19:38.200521+0000","last_peered":"2024-05-08T23:19:38.200521+0000","last_clean":"2024-05-08T23:19:38.200521+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:19:38.200521+0000","last_undegraded":"2024-05-08T23:19:38.200521+0000","last_fullsized":"2024-05-08T23:19:38.200521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":71,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974398+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:19:37.974398+0000","last_peered":"2024-05-08T23:19:37.974398+0000","last_clean":"2024-05-08T23:19:37.974398+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:19:37.974398+0000","last_undegraded":"2024-05-08T23:19:37.974398+0000","last_fullsized":"2024-05-08T23:19:37.974398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795943+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:43.795943+0000","last_peered":"2024-05-08T23:18:43.795943+0000","last_clean":"2024-05-08T23:18:43.795943+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:43.795943+0000","last_undegraded":"2024-05-08T23:18:43.795943+0000","last_fullsized":"2024-05-08T23:18:43.795943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794461+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:43.794461+0000","last_peered":"2024-05-08T23:18:43.794461+0000","last_clean":"2024-05-08T23:18:43.794461+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:43.794461+0000","last_undegraded":"2024-05-08T23:18:43.794461+0000","last_fullsized":"2024-05-08T23:18:43.794461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.964151+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:19:37.964151+0000","last_peered":"2024-05-08T23:19:37.964151+0000","last_clean":"2024-05-08T23:19:37.964151+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:19:37.964151+0000","last_undegraded":"2024-05-08T23:19:37.964151+0000","last_fullsized":"2024-05-08T23:19:37.964151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793979+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:43.793979+0000","last_peered":"2024-05-08T23:18:43.793979+0000","last_clean":"2024-05-08T23:18:43.793979+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:43.793979+0000","last_undegraded":"2024-05-08T23:18:43.793979+0000","last_fullsized":"2024-05-08T23:18:43.793979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.927256+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:43.927256+0000","last_peered":"2024-05-08T23:18:43.927256+0000","last_clean":"2024-05-08T23:18:43.927256+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:43.927256+0000","last_undegraded":"2024-05-08T23:18:43.927256+0000","last_fullsized":"2024-05-08T23:18:43.927256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.966650+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:19:37.966650+0000","last_peered":"2024-05-08T23:19:37.966650+0000","last_clean":"2024-05-08T23:19:37.966650+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:19:37.966650+0000","last_undegraded":"2024-05-08T23:19:37.966650+0000","last_fullsized":"2024-05-08T23:19:37.966650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034275+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:44.034275+0000","last_peered":"2024-05-08T23:18:44.034275+0000","last_clean":"2024-05-08T23:18:44.034275+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:44.034275+0000","last_undegraded":"2024-05-08T23:18:44.034275+0000","last_fullsized":"2024-05-08T23:18:44.034275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793711+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:43.793711+0000","last_peered":"2024-05-08T23:18:43.793711+0000","last_clean":"2024-05-08T23:18:43.793711+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:43.793711+0000","last_undegraded":"2024-05-08T23:18:43.793711+0000","last_fullsized":"2024-05-08T23:18:43.793711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975082+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:19:37.975082+0000","last_peered":"2024-05-08T23:19:37.975082+0000","last_clean":"2024-05-08T23:19:37.975082+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:19:37.975082+0000","last_undegraded":"2024-05-08T23:19:37.975082+0000","last_fullsized":"2024-05-08T23:19:37.975082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793856+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:43.793856+0000","last_peered":"2024-05-08T23:18:43.793856+0000","last_clean":"2024-05-08T23:18:43.793856+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:43.793856+0000","last_undegraded":"2024-05-08T23:18:43.793856+0000","last_fullsized":"2024-05-08T23:18:43.793856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796671+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:43.796671+0000","last_peered":"2024-05-08T23:18:43.796671+0000","last_clean":"2024-05-08T23:18:43.796671+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:43.796671+0000","last_undegraded":"2024-05-08T23:18:43.796671+0000","last_fullsized":"2024-05-08T23:18:43.796671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.967303+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:19:37.967303+0000","last_peered":"2024-05-08T23:19:37.967303+0000","last_clean":"2024-05-08T23:19:37.967303+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:19:37.967303+0000","last_undegraded":"2024-05-08T23:19:37.967303+0000","last_fullsized":"2024-05-08T23:19:37.967303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794095+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:43.794095+0000","last_peered":"2024-05-08T23:18:43.794095+0000","last_clean":"2024-05-08T23:18:43.794095+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:43.794095+0000","last_undegraded":"2024-05-08T23:18:43.794095+0000","last_fullsized":"2024-05-08T23:18:43.794095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.635532+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:45.635532+0000","last_peered":"2024-05-08T23:18:45.635532+0000","last_clean":"2024-05-08T23:18:45.635532+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:45.635532+0000","last_undegraded":"2024-05-08T23:18:45.635532+0000","last_fullsized":"2024-05-08T23:18:45.635532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985346+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:19:37.985346+0000","last_peered":"2024-05-08T23:19:37.985346+0000","last_clean":"2024-05-08T23:19:37.985346+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:19:37.985346+0000","last_undegraded":"2024-05-08T23:19:37.985346+0000","last_fullsized":"2024-05-08T23:19:37.985346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.927268+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:43.927268+0000","last_peered":"2024-05-08T23:18:43.927268+0000","last_clean":"2024-05-08T23:18:43.927268+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:43.927268+0000","last_undegraded":"2024-05-08T23:18:43.927268+0000","last_fullsized":"2024-05-08T23:18:43.927268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797148+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:43.797148+0000","last_peered":"2024-05-08T23:18:43.797148+0000","last_clean":"2024-05-08T23:18:43.797148+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:43.797148+0000","last_undegraded":"2024-05-08T23:18:43.797148+0000","last_fullsized":"2024-05-08T23:18:43.797148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980423+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:19:37.980423+0000","last_peered":"2024-05-08T23:19:37.980423+0000","last_clean":"2024-05-08T23:19:37.980423+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:19:37.980423+0000","last_undegraded":"2024-05-08T23:19:37.980423+0000","last_fullsized":"2024-05-08T23:19:37.980423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.091302+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:44.091302+0000","last_peered":"2024-05-08T23:18:44.091302+0000","last_clean":"2024-05-08T23:18:44.091302+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:44.091302+0000","last_undegraded":"2024-05-08T23:18:44.091302+0000","last_fullsized":"2024-05-08T23:18:44.091302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.197927+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:19:38.197927+0000","last_peered":"2024-05-08T23:19:38.197927+0000","last_clean":"2024-05-08T23:19:38.197927+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:19:38.197927+0000","last_undegraded":"2024-05-08T23:19:38.197927+0000","last_fullsized":"2024-05-08T23:19:38.197927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979495+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:19:37.979495+0000","last_peered":"2024-05-08T23:19:37.979495+0000","last_clean":"2024-05-08T23:19:37.979495+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:19:37.979495+0000","last_undegraded":"2024-05-08T23:19:37.979495+0000","last_fullsized":"2024-05-08T23:19:37.979495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794917+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:18:43.794917+0000","last_peered":"2024-05-08T23:18:43.794917+0000","last_clean":"2024-05-08T23:18:43.794917+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:18:43.794917+0000","last_undegraded":"2024-05-08T23:18:43.794917+0000","last_fullsized":"2024-05-08T23:18:43.794917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931822+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931822+0000","last_peered":"2024-05-08T23:18:43.931822+0000","last_clean":"2024-05-08T23:18:43.931822+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931822+0000","last_undegraded":"2024-05-08T23:18:43.931822+0000","last_fullsized":"2024-05-08T23:18:43.931822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.972088+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:19:37.972088+0000","last_peered":"2024-05-08T23:19:37.972088+0000","last_clean":"2024-05-08T23:19:37.972088+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:19:37.972088+0000","last_undegraded":"2024-05-08T23:19:37.972088+0000","last_fullsized":"2024-05-08T23:19:37.972088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974459+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:19:37.974459+0000","last_peered":"2024-05-08T23:19:37.974459+0000","last_clean":"2024-05-08T23:19:37.974459+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:19:37.974459+0000","last_undegraded":"2024-05-08T23:19:37.974459+0000","last_fullsized":"2024-05-08T23:19:37.974459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979844+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:19:37.979844+0000","last_peered":"2024-05-08T23:19:37.979844+0000","last_clean":"2024-05-08T23:19:37.979844+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:19:37.979844+0000","last_undegraded":"2024-05-08T23:19:37.979844+0000","last_fullsized":"2024-05-08T23:19:37.979844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.963986+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:19:37.963986+0000","last_peered":"2024-05-08T23:19:37.963986+0000","last_clean":"2024-05-08T23:19:37.963986+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:19:37.963986+0000","last_undegraded":"2024-05-08T23:19:37.963986+0000","last_fullsized":"2024-05-08T23:19:37.963986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985739+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:19:37.985739+0000","last_peered":"2024-05-08T23:19:37.985739+0000","last_clean":"2024-05-08T23:19:37.985739+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:19:37.985739+0000","last_undegraded":"2024-05-08T23:19:37.985739+0000","last_fullsized":"2024-05-08T23:19:37.985739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982612+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:19:37.982612+0000","last_peered":"2024-05-08T23:19:37.982612+0000","last_clean":"2024-05-08T23:19:37.982612+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:19:37.982612+0000","last_undegraded":"2024-05-08T23:19:37.982612+0000","last_fullsized":"2024-05-08T23:19:37.982612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973494+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:19:37.973494+0000","last_peered":"2024-05-08T23:19:37.973494+0000","last_clean":"2024-05-08T23:19:37.973494+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:19:37.973494+0000","last_undegraded":"2024-05-08T23:19:37.973494+0000","last_fullsized":"2024-05-08T23:19:37.973494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.972939+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:19:37.972939+0000","last_peered":"2024-05-08T23:19:37.972939+0000","last_clean":"2024-05-08T23:19:37.972939+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:19:37.972939+0000","last_undegraded":"2024-05-08T23:19:37.972939+0000","last_fullsized":"2024-05-08T23:19:37.972939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.960851+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:19:37.960851+0000","last_peered":"2024-05-08T23:19:37.960851+0000","last_clean":"2024-05-08T23:19:37.960851+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:19:37.960851+0000","last_undegraded":"2024-05-08T23:19:37.960851+0000","last_fullsized":"2024-05-08T23:19:37.960851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978702+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:19:37.978702+0000","last_peered":"2024-05-08T23:19:37.978702+0000","last_clean":"2024-05-08T23:19:37.978702+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:19:37.978702+0000","last_undegraded":"2024-05-08T23:19:37.978702+0000","last_fullsized":"2024-05-08T23:19:37.978702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.966116+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:19:37.966116+0000","last_peered":"2024-05-08T23:19:37.966116+0000","last_clean":"2024-05-08T23:19:37.966116+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:19:37.966116+0000","last_undegraded":"2024-05-08T23:19:37.966116+0000","last_fullsized":"2024-05-08T23:19:37.966116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982523+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:19:37.982523+0000","last_peered":"2024-05-08T23:19:37.982523+0000","last_clean":"2024-05-08T23:19:37.982523+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:19:37.982523+0000","last_undegraded":"2024-05-08T23:19:37.982523+0000","last_fullsized":"2024-05-08T23:19:37.982523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.986378+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:19:37.986378+0000","last_peered":"2024-05-08T23:19:37.986378+0000","last_clean":"2024-05-08T23:19:37.986378+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:19:37.986378+0000","last_undegraded":"2024-05-08T23:19:37.986378+0000","last_fullsized":"2024-05-08T23:19:37.986378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977581+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:19:37.977581+0000","last_peered":"2024-05-08T23:19:37.977581+0000","last_clean":"2024-05-08T23:19:37.977581+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:19:37.977581+0000","last_undegraded":"2024-05-08T23:19:37.977581+0000","last_fullsized":"2024-05-08T23:19:37.977581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593571,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":642,"num_read_kb":426,"num_write":278,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1780713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645344,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":950272,"data_stored":472912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48399999999999999}]},{"osd":1,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93300000000000005,"5min":1.163,"15min":1.163},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.749,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.379}]},{"osd":2,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45000000000000001},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.47299999999999998},"min":{"1min":0.37,"5min":0.311,"15min":0.311},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81399999999999995,"5min":1.181,"15min":1.181},"last":0.433}]},{"osd":5,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.45800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98599999999999999,"5min":1.075,"15min":1.075},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":1.038,"15min":1.038},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.59299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.875,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56299999999999994}]},{"osd":9,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76300000000000001,"15min":0.75800000000000001},"min":{"1min":0.55700000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.75,"15min":0.73699999999999999},"min":{"1min":0.71099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.032,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94899999999999995,"5min":1.216,"15min":1.216},"last":0.62},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.61599999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.61599999999999999}]}]},{"osd":10,"up_from":65,"seq":279172874276,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37060,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708092,"statfs":{"total":95995035648,"available":95957086208,"internally_reserved":0,"allocated":1773568,"data_stored":1255360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.374}]},{"osd":1,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.66800000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.61499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.871,"15min":0.871},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.62,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.746,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72999999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.77200000000000002,"5min":1.169,"15min":1.169},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81599999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.5},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.84299999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83699999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.77700000000000002,"5min":1.016,"15min":1.016},"last":0.60899999999999999}]},{"osd":9,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79800000000000004,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86199999999999999,"5min":1.3,"15min":1.3},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.76200000000000001,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83699999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56000000000000005}]},{"osd":11,"last update":"Wed May 8 23:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.57999999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.87,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95299999999999996,"5min":1.083,"15min":1.083},"last":0.63700000000000001}]}]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1363968,"data_stored":870460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42299999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.504},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.58299999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60499999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0660000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70599999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.66400000000000003}]}]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28344,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716808,"statfs":{"total":95995035648,"available":95966011392,"internally_reserved":0,"allocated":1359872,"data_stored":866284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83199999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.51600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.317},{"interface":"front","average":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":1.137,"15min":1.137},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84399999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.503,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.5}]},{"osd":5,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78400000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85599999999999998,"5min":0.876,"15min":0.876},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.65500000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95499999999999996,"5min":1.042,"15min":1.042},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.71199999999999997},"min":{"1min":0.57999999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94699999999999995,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.623},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94599999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.93600000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63300000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68300000000000005}]}]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1556480,"data_stored":1070723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.38},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.433},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81399999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.77800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.627}]},{"osd":6,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69399999999999995,"15min":0.70899999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56599999999999995}]},{"osd":8,"last update":"Wed May 8 23:19:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.59899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.82599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.622,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.81599999999999995,"5min":0.875,"15min":0.875},"last":0.63800000000000001}]},{"osd":9,"last update":"Wed May 8 23:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.878,"5min":0.997,"15min":0.997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.752}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86799999999999999,"5min":0.999,"15min":0.999},"last":0.59199999999999997}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68300000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":946176,"data_stored":468324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.623}]},{"osd":7,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.501,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0309999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.177,"15min":1.177},"last":0.53800000000000003}]},{"osd":9,"last update":"Wed May 8 23:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.433}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.79700000000000004,"15min":0.79000000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":1015808,"data_stored":530911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.495},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82499999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.50900000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.40999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44700000000000001,"15min":0.40400000000000003},"min":{"1min":0.376,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.45600000000000002,"15min":0.41199999999999998},"min":{"1min":0.371,"5min":0.248,"15min":0.248},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.627,"15min":0.56699999999999995},"min":{"1min":0.53300000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.63300000000000001,"15min":0.57199999999999995},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.58299999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.621,"15min":0.59299999999999997},"min":{"1min":0.54700000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.504},"min":{"1min":0.44700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.374},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.35899999999999999}]},{"osd":8,"last update":"Wed May 8 23:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64800000000000002},"min":{"1min":0.48799999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.57799999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.014,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.39600000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53700000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.59399999999999997}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":970752,"data_stored":469712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89000000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Wed May 8 23:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.55300000000000005},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.66000000000000003,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.63},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66100000000000003,"15min":0.629},"min":{"1min":0.502,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66000000000000003}]},{"osd":8,"last update":"Wed May 8 23:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72399999999999998,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.996,"5min":1.014,"15min":1.014},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.755}]},{"osd":10,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":0.997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0740000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68500000000000005}]},{"osd":11,"last update":"Wed May 8 23:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0820000000000001,"5min":1.246,"15min":1.246},"last":0.78000000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":531923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.873,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.60399999999999998,"15min":0.54000000000000004},"min":{"1min":0.54500000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 8 23:19:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83099999999999996,"5min":0.872,"15min":0.872},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.754,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.58699999999999997},"min":{"1min":0.52900000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":8,"last update":"Wed May 8 23:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.433,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.61299999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50800000000000001}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.55400000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.67700000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63600000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64500000000000002}]}]},{"osd":3,"up_from":23,"seq":98784247868,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36660,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708492,"statfs":{"total":95995035648,"available":95957495808,"internally_reserved":0,"allocated":1486848,"data_stored":993334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.436,"15min":0.36799999999999999},"min":{"1min":0.32600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.83199999999999996,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.441,"15min":0.38600000000000001},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59399999999999997},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.98499999999999999,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.57499999999999996},"min":{"1min":0.51400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.54500000000000004},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.105,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed May 8 23:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0469999999999999,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.497},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.621},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed May 8 23:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96099999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.997,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.624,"15min":0.58799999999999997},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91900000000000004,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.622},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.44700000000000001}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94499999999999995,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1879999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.73099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:19:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0920000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed May 8 23:18:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.92200000000000004,"15min":0.94799999999999995},"min":{"1min":0.65300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.135,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.97899999999999998,"15min":1.0049999999999999},"min":{"1min":0.67800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.171,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.74299999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65400000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018936,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":473422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.34100000000000003},"min":{"1min":0.32900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39500000000000002,"15min":0.36699999999999999},"min":{"1min":0.33600000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.58299999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.48799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.44500000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67200000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":0.874,"15min":0.874},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.44700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38}]},{"osd":5,"last update":"Wed May 8 23:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72299999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.28999999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65000000000000002,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.498}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60099999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81100000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41099999999999998}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55800000000000005}]}]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1703936,"data_stored":1197008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.372},{"interface":"front","average":{"1min":0.496,"5min":0.53300000000000003,"15min":0.56499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.434},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.27500000000000002}]},{"osd":3,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71999999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.35599999999999998}]},{"osd":6,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.443},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.433}]},{"osd":7,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79500000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40999999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47899999999999998}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55400000000000005}]},{"osd":10,"last update":"Wed May 8 23:18:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.56200000000000006,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":197089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:19:45.307 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T23:19:45.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T23:19:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:44 smithi189 ceph-mon[46919]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T23:19:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/890899036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4143482905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:45.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 -- 172.21.15.16:0/2106374167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec10b3f0 msgr2=0x7fcaec10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 --2- 172.21.15.16:0/2106374167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec10b3f0 0x7fcaec10d8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcae400b0a0 tx=0x7fcae400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 -- 172.21.15.16:0/2106374167 shutdown_connections 2024-05-08T23:19:45.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 --2- 172.21.15.16:0/2106374167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec10b3f0 0x7fcaec10d8b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 --2- 172.21.15.16:0/2106374167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec101fd0 0x7fcaec1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 --2- 172.21.15.16:0/2106374167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaec1028f0 0x7fcaec10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 -- 172.21.15.16:0/2106374167 >> 172.21.15.16:0/2106374167 conn(0x7fcaec0fb7d0 msgr2=0x7fcaec0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:45.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 -- 172.21.15.16:0/2106374167 shutdown_connections 2024-05-08T23:19:45.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.480+0000 7fcaf104f700 1 -- 172.21.15.16:0/2106374167 wait complete. 2024-05-08T23:19:45.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 Processor -- start 2024-05-08T23:19:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 -- start start 2024-05-08T23:19:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaec101fd0 0x7fcaec0ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 0x7fcaec100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec10b3f0 0x7fcaec1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1a5670 con 0x7fcaec1028f0 2024-05-08T23:19:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1a57e0 con 0x7fcaec101fd0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaf104f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1a59a0 con 0x7fcaec10b3f0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaea59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 0x7fcaec100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaea59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 0x7fcaec100160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60420/0 (socket says 172.21.15.16:60420) 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaea59c700 1 -- 172.21.15.16:0/3930093660 learned_addr learned my addr 172.21.15.16:0/3930093660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.481+0000 7fcaead9d700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaec101fd0 0x7fcaec0ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaeb59e700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec10b3f0 0x7fcaec1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaea59c700 1 -- 172.21.15.16:0/3930093660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec10b3f0 msgr2=0x7fcaec1006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaea59c700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec10b3f0 0x7fcaec1006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaea59c700 1 -- 172.21.15.16:0/3930093660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaec101fd0 msgr2=0x7fcaec0ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaea59c700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaec101fd0 0x7fcaec0ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaea59c700 1 -- 172.21.15.16:0/3930093660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4009d00 con 0x7fcaec1028f0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaea59c700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 0x7fcaec100160 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcae000d950 tx=0x7fcae000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000f890 con 0x7fcaec1028f0 2024-05-08T23:19:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae000fe80 con 0x7fcaec1028f0 2024-05-08T23:19:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec1a5c80 con 0x7fcaec1028f0 2024-05-08T23:19:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000e3b0 con 0x7fcaec1028f0 2024-05-08T23:19:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.482+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec1a6190 con 0x7fcaec1028f0 2024-05-08T23:19:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.483+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec106520 con 0x7fcaec1028f0 2024-05-08T23:19:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.485+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fcae0010460 con 0x7fcaec1028f0 2024-05-08T23:19:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.485+0000 7fcad3fff700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad40a1320 0x7fcad40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.485+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fcae00cce40 con 0x7fcaec1028f0 2024-05-08T23:19:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.485+0000 7fcaead9d700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad40a1320 0x7fcad40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.486+0000 7fcaead9d700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad40a1320 0x7fcad40a37e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009c60 tx=0x7fcadc009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.490+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcae0095ea0 con 0x7fcaec1028f0 2024-05-08T23:19:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.709+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcaec110c10 con 0x7fcad40a1320 2024-05-08T23:19:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.722+0000 7fcad3fff700 1 -- 172.21.15.16:0/3930093660 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+425554 (secure 0 0 0) 0x7fcaec110c10 con 0x7fcad40a1320 2024-05-08T23:19:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.726+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad40a1320 msgr2=0x7fcad40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.726+0000 7fcaf104f700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad40a1320 0x7fcad40a37e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcadc009c60 tx=0x7fcadc009260 comp rx=0 tx=0).stop 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.726+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 msgr2=0x7fcaec100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.726+0000 7fcaf104f700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 0x7fcaec100160 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcae000d950 tx=0x7fcae000dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 shutdown_connections 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad40a1320 0x7fcad40a37e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaec10b3f0 0x7fcaec1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaec101fd0 0x7fcaec0ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 --2- 172.21.15.16:0/3930093660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaec1028f0 0x7fcaec100160 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 >> 172.21.15.16:0/3930093660 conn(0x7fcaec0fb7d0 msgr2=0x7fcaec0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 shutdown_connections 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.727+0000 7fcaf104f700 1 -- 172.21.15.16:0/3930093660 wait complete. 2024-05-08T23:19:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-05-08T23:19:45.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-08T23:19:43.939958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1184173,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":778,"num_read_kb":545,"num_write":461,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":400548,"kb_used_data":14768,"kb_used_omap":18,"kb_used_meta":385517,"kb_avail":1124541276,"statfs":{"total":1151940427776,"available":1151530266624,"internally_reserved":0,"allocated":15122432,"data_stored":9200373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19128,"internal_metadata":394769736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989242"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979175+0000","last_change":"2024-05-08T23:17:46.106276+0000","last_active":"2024-05-08T23:19:37.979175+0000","last_peered":"2024-05-08T23:19:37.979175+0000","last_clean":"2024-05-08T23:19:37.979175+0000","last_became_active":"2024-05-08T23:17:46.105692+0000","last_became_peered":"2024-05-08T23:17:46.105692+0000","last_unstale":"2024-05-08T23:19:37.979175+0000","last_undegraded":"2024-05-08T23:19:37.979175+0000","last_fullsized":"2024-05-08T23:19:37.979175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:09:23.860710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975013+0000","last_change":"2024-05-08T23:17:46.096012+0000","last_active":"2024-05-08T23:19:37.975013+0000","last_peered":"2024-05-08T23:19:37.975013+0000","last_clean":"2024-05-08T23:19:37.975013+0000","last_became_active":"2024-05-08T23:17:46.095538+0000","last_became_peered":"2024-05-08T23:17:46.095538+0000","last_unstale":"2024-05-08T23:19:37.975013+0000","last_undegraded":"2024-05-08T23:19:37.975013+0000","last_fullsized":"2024-05-08T23:19:37.975013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:35:40.650497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978939+0000","last_change":"2024-05-08T23:17:46.108873+0000","last_active":"2024-05-08T23:19:37.978939+0000","last_peered":"2024-05-08T23:19:37.978939+0000","last_clean":"2024-05-08T23:19:37.978939+0000","last_became_active":"2024-05-08T23:17:46.108585+0000","last_became_peered":"2024-05-08T23:17:46.108585+0000","last_unstale":"2024-05-08T23:19:37.978939+0000","last_undegraded":"2024-05-08T23:19:37.978939+0000","last_fullsized":"2024-05-08T23:19:37.978939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:34.484480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977387+0000","last_change":"2024-05-08T23:17:46.102163+0000","last_active":"2024-05-08T23:19:37.977387+0000","last_peered":"2024-05-08T23:19:37.977387+0000","last_clean":"2024-05-08T23:19:37.977387+0000","last_became_active":"2024-05-08T23:17:46.101354+0000","last_became_peered":"2024-05-08T23:17:46.101354+0000","last_unstale":"2024-05-08T23:19:37.977387+0000","last_undegraded":"2024-05-08T23:19:37.977387+0000","last_fullsized":"2024-05-08T23:19:37.977387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:01.113546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974352+0000","last_change":"2024-05-08T23:17:46.099034+0000","last_active":"2024-05-08T23:19:37.974352+0000","last_peered":"2024-05-08T23:19:37.974352+0000","last_clean":"2024-05-08T23:19:37.974352+0000","last_became_active":"2024-05-08T23:17:46.096659+0000","last_became_peered":"2024-05-08T23:17:46.096659+0000","last_unstale":"2024-05-08T23:19:37.974352+0000","last_undegraded":"2024-05-08T23:19:37.974352+0000","last_fullsized":"2024-05-08T23:19:37.974352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:33.137809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980276+0000","last_change":"2024-05-08T23:17:46.103358+0000","last_active":"2024-05-08T23:19:37.980276+0000","last_peered":"2024-05-08T23:19:37.980276+0000","last_clean":"2024-05-08T23:19:37.980276+0000","last_became_active":"2024-05-08T23:17:46.102903+0000","last_became_peered":"2024-05-08T23:17:46.102903+0000","last_unstale":"2024-05-08T23:19:37.980276+0000","last_undegraded":"2024-05-08T23:19:37.980276+0000","last_fullsized":"2024-05-08T23:19:37.980276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:43:39.536118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978937+0000","last_change":"2024-05-08T23:17:46.107477+0000","last_active":"2024-05-08T23:19:37.978937+0000","last_peered":"2024-05-08T23:19:37.978937+0000","last_clean":"2024-05-08T23:19:37.978937+0000","last_became_active":"2024-05-08T23:17:46.107018+0000","last_became_peered":"2024-05-08T23:17:46.107018+0000","last_unstale":"2024-05-08T23:19:37.978937+0000","last_undegraded":"2024-05-08T23:19:37.978937+0000","last_fullsized":"2024-05-08T23:19:37.978937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:21.837998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.970401+0000","last_change":"2024-05-08T23:17:46.096234+0000","last_active":"2024-05-08T23:19:37.970401+0000","last_peered":"2024-05-08T23:19:37.970401+0000","last_clean":"2024-05-08T23:19:37.970401+0000","last_became_active":"2024-05-08T23:17:46.095678+0000","last_became_peered":"2024-05-08T23:17:46.095678+0000","last_unstale":"2024-05-08T23:19:37.970401+0000","last_undegraded":"2024-05-08T23:19:37.970401+0000","last_fullsized":"2024-05-08T23:19:37.970401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:25.156709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977019+0000","last_change":"2024-05-08T23:17:46.114317+0000","last_active":"2024-05-08T23:19:37.977019+0000","last_peered":"2024-05-08T23:19:37.977019+0000","last_clean":"2024-05-08T23:19:37.977019+0000","last_became_active":"2024-05-08T23:17:46.113762+0000","last_became_peered":"2024-05-08T23:17:46.113762+0000","last_unstale":"2024-05-08T23:19:37.977019+0000","last_undegraded":"2024-05-08T23:19:37.977019+0000","last_fullsized":"2024-05-08T23:19:37.977019+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:10.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":43,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.984147+0000","last_change":"2024-05-08T23:18:44.034666+0000","last_active":"2024-05-08T23:19:37.984147+0000","last_peered":"2024-05-08T23:19:37.984147+0000","last_clean":"2024-05-08T23:19:37.984147+0000","last_became_active":"2024-05-08T23:18:44.034211+0000","last_became_peered":"2024-05-08T23:18:44.034211+0000","last_unstale":"2024-05-08T23:19:37.984147+0000","last_undegraded":"2024-05-08T23:19:37.984147+0000","last_fullsized":"2024-05-08T23:19:37.984147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:17.808239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980710+0000","last_change":"2024-05-08T23:17:46.117326+0000","last_active":"2024-05-08T23:19:37.980710+0000","last_peered":"2024-05-08T23:19:37.980710+0000","last_clean":"2024-05-08T23:19:37.980710+0000","last_became_active":"2024-05-08T23:17:46.116917+0000","last_became_peered":"2024-05-08T23:17:46.116917+0000","last_unstale":"2024-05-08T23:19:37.980710+0000","last_undegraded":"2024-05-08T23:19:37.980710+0000","last_fullsized":"2024-05-08T23:19:37.980710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:17:30.643952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.967635+0000","last_change":"2024-05-08T23:17:46.102399+0000","last_active":"2024-05-08T23:19:37.967635+0000","last_peered":"2024-05-08T23:19:37.967635+0000","last_clean":"2024-05-08T23:19:37.967635+0000","last_became_active":"2024-05-08T23:17:46.101842+0000","last_became_peered":"2024-05-08T23:17:46.101842+0000","last_unstale":"2024-05-08T23:19:37.967635+0000","last_undegraded":"2024-05-08T23:19:37.967635+0000","last_fullsized":"2024-05-08T23:19:37.967635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:27:16.700124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985493+0000","last_change":"2024-05-08T23:17:46.104672+0000","last_active":"2024-05-08T23:19:37.985493+0000","last_peered":"2024-05-08T23:19:37.985493+0000","last_clean":"2024-05-08T23:19:37.985493+0000","last_became_active":"2024-05-08T23:17:46.104338+0000","last_became_peered":"2024-05-08T23:17:46.104338+0000","last_unstale":"2024-05-08T23:19:37.985493+0000","last_undegraded":"2024-05-08T23:19:37.985493+0000","last_fullsized":"2024-05-08T23:19:37.985493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:45:26.413942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.971337+0000","last_change":"2024-05-08T23:17:46.095707+0000","last_active":"2024-05-08T23:19:37.971337+0000","last_peered":"2024-05-08T23:19:37.971337+0000","last_clean":"2024-05-08T23:19:37.971337+0000","last_became_active":"2024-05-08T23:17:46.095321+0000","last_became_peered":"2024-05-08T23:17:46.095321+0000","last_unstale":"2024-05-08T23:19:37.971337+0000","last_undegraded":"2024-05-08T23:19:37.971337+0000","last_fullsized":"2024-05-08T23:19:37.971337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:11:38.935662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978114+0000","last_change":"2024-05-08T23:17:46.108270+0000","last_active":"2024-05-08T23:19:37.978114+0000","last_peered":"2024-05-08T23:19:37.978114+0000","last_clean":"2024-05-08T23:19:37.978114+0000","last_became_active":"2024-05-08T23:17:46.107933+0000","last_became_peered":"2024-05-08T23:17:46.107933+0000","last_unstale":"2024-05-08T23:19:37.978114+0000","last_undegraded":"2024-05-08T23:19:37.978114+0000","last_fullsized":"2024-05-08T23:19:37.978114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:18.814027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982276+0000","last_change":"2024-05-08T23:17:46.113448+0000","last_active":"2024-05-08T23:19:37.982276+0000","last_peered":"2024-05-08T23:19:37.982276+0000","last_clean":"2024-05-08T23:19:37.982276+0000","last_became_active":"2024-05-08T23:17:46.113117+0000","last_became_peered":"2024-05-08T23:17:46.113117+0000","last_unstale":"2024-05-08T23:19:37.982276+0000","last_undegraded":"2024-05-08T23:19:37.982276+0000","last_fullsized":"2024-05-08T23:19:37.982276+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:41.451452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979259+0000","last_change":"2024-05-08T23:17:46.106050+0000","last_active":"2024-05-08T23:19:37.979259+0000","last_peered":"2024-05-08T23:19:37.979259+0000","last_clean":"2024-05-08T23:19:37.979259+0000","last_became_active":"2024-05-08T23:17:46.105724+0000","last_became_peered":"2024-05-08T23:17:46.105724+0000","last_unstale":"2024-05-08T23:19:37.979259+0000","last_undegraded":"2024-05-08T23:19:37.979259+0000","last_fullsized":"2024-05-08T23:19:37.979259+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:03:56.898719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979567+0000","last_change":"2024-05-08T23:17:46.106331+0000","last_active":"2024-05-08T23:19:37.979567+0000","last_peered":"2024-05-08T23:19:37.979567+0000","last_clean":"2024-05-08T23:19:37.979567+0000","last_became_active":"2024-05-08T23:17:46.105970+0000","last_became_peered":"2024-05-08T23:17:46.105970+0000","last_unstale":"2024-05-08T23:19:37.979567+0000","last_undegraded":"2024-05-08T23:19:37.979567+0000","last_fullsized":"2024-05-08T23:19:37.979567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:46:33.929198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.976723+0000","last_change":"2024-05-08T23:17:46.108722+0000","last_active":"2024-05-08T23:19:37.976723+0000","last_peered":"2024-05-08T23:19:37.976723+0000","last_clean":"2024-05-08T23:19:37.976723+0000","last_became_active":"2024-05-08T23:17:46.108378+0000","last_became_peered":"2024-05-08T23:17:46.108378+0000","last_unstale":"2024-05-08T23:19:37.976723+0000","last_undegraded":"2024-05-08T23:19:37.976723+0000","last_fullsized":"2024-05-08T23:19:37.976723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:25:15.230597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796041+0000","last_change":"2024-05-08T23:17:50.131584+0000","last_active":"2024-05-08T23:18:43.796041+0000","last_peered":"2024-05-08T23:18:43.796041+0000","last_clean":"2024-05-08T23:18:43.796041+0000","last_became_active":"2024-05-08T23:17:50.131072+0000","last_became_peered":"2024-05-08T23:17:50.131072+0000","last_unstale":"2024-05-08T23:18:43.796041+0000","last_undegraded":"2024-05-08T23:18:43.796041+0000","last_fullsized":"2024-05-08T23:18:43.796041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:56:49.681911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034288+0000","last_change":"2024-05-08T23:17:45.617716+0000","last_active":"2024-05-08T23:18:44.034288+0000","last_peered":"2024-05-08T23:18:44.034288+0000","last_clean":"2024-05-08T23:18:44.034288+0000","last_became_active":"2024-05-08T23:17:45.617388+0000","last_became_peered":"2024-05-08T23:17:45.617388+0000","last_unstale":"2024-05-08T23:18:44.034288+0000","last_undegraded":"2024-05-08T23:18:44.034288+0000","last_fullsized":"2024-05-08T23:18:44.034288+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:06.486718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.964536+0000","last_change":"2024-05-08T23:17:46.110130+0000","last_active":"2024-05-08T23:19:37.964536+0000","last_peered":"2024-05-08T23:19:37.964536+0000","last_clean":"2024-05-08T23:19:37.964536+0000","last_became_active":"2024-05-08T23:17:46.109828+0000","last_became_peered":"2024-05-08T23:17:46.109828+0000","last_unstale":"2024-05-08T23:19:37.964536+0000","last_undegraded":"2024-05-08T23:19:37.964536+0000","last_fullsized":"2024-05-08T23:19:37.964536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:06:09.488746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793688+0000","last_change":"2024-05-08T23:17:50.131224+0000","last_active":"2024-05-08T23:18:43.793688+0000","last_peered":"2024-05-08T23:18:43.793688+0000","last_clean":"2024-05-08T23:18:43.793688+0000","last_became_active":"2024-05-08T23:17:50.130824+0000","last_became_peered":"2024-05-08T23:17:50.130824+0000","last_unstale":"2024-05-08T23:18:43.793688+0000","last_undegraded":"2024-05-08T23:18:43.793688+0000","last_fullsized":"2024-05-08T23:18:43.793688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:36:40.298832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794148+0000","last_change":"2024-05-08T23:17:45.623265+0000","last_active":"2024-05-08T23:18:43.794148+0000","last_peered":"2024-05-08T23:18:43.794148+0000","last_clean":"2024-05-08T23:18:43.794148+0000","last_became_active":"2024-05-08T23:17:45.622927+0000","last_became_peered":"2024-05-08T23:17:45.622927+0000","last_unstale":"2024-05-08T23:18:43.794148+0000","last_undegraded":"2024-05-08T23:18:43.794148+0000","last_fullsized":"2024-05-08T23:18:43.794148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:22.246932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.983386+0000","last_change":"2024-05-08T23:17:46.108001+0000","last_active":"2024-05-08T23:19:37.983386+0000","last_peered":"2024-05-08T23:19:37.983386+0000","last_clean":"2024-05-08T23:19:37.983386+0000","last_became_active":"2024-05-08T23:17:46.107267+0000","last_became_peered":"2024-05-08T23:17:46.107267+0000","last_unstale":"2024-05-08T23:19:37.983386+0000","last_undegraded":"2024-05-08T23:19:37.983386+0000","last_fullsized":"2024-05-08T23:19:37.983386+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:04.803827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803701+0000","last_change":"2024-05-08T23:18:43.803307+0000","last_active":"2024-05-08T23:18:43.803701+0000","last_peered":"2024-05-08T23:18:43.803701+0000","last_clean":"2024-05-08T23:18:43.803701+0000","last_became_active":"2024-05-08T23:18:43.802963+0000","last_became_peered":"2024-05-08T23:18:43.802963+0000","last_unstale":"2024-05-08T23:18:43.803701+0000","last_undegraded":"2024-05-08T23:18:43.803701+0000","last_fullsized":"2024-05-08T23:18:43.803701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:29.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.635572+0000","last_change":"2024-05-08T23:18:43.813167+0000","last_active":"2024-05-08T23:18:45.635572+0000","last_peered":"2024-05-08T23:18:45.635572+0000","last_clean":"2024-05-08T23:18:45.635572+0000","last_became_active":"2024-05-08T23:18:43.801702+0000","last_became_peered":"2024-05-08T23:18:43.801702+0000","last_unstale":"2024-05-08T23:18:45.635572+0000","last_undegraded":"2024-05-08T23:18:45.635572+0000","last_fullsized":"2024-05-08T23:18:45.635572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:10.511009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979236+0000","last_change":"2024-05-08T23:17:46.116557+0000","last_active":"2024-05-08T23:19:37.979236+0000","last_peered":"2024-05-08T23:19:37.979236+0000","last_clean":"2024-05-08T23:19:37.979236+0000","last_became_active":"2024-05-08T23:17:46.115671+0000","last_became_peered":"2024-05-08T23:17:46.115671+0000","last_unstale":"2024-05-08T23:19:37.979236+0000","last_undegraded":"2024-05-08T23:19:37.979236+0000","last_fullsized":"2024-05-08T23:19:37.979236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:38.011122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793735+0000","last_change":"2024-05-08T23:17:50.126271+0000","last_active":"2024-05-08T23:18:43.793735+0000","last_peered":"2024-05-08T23:18:43.793735+0000","last_clean":"2024-05-08T23:18:43.793735+0000","last_became_active":"2024-05-08T23:17:50.125858+0000","last_became_peered":"2024-05-08T23:17:50.125858+0000","last_unstale":"2024-05-08T23:18:43.793735+0000","last_undegraded":"2024-05-08T23:18:43.793735+0000","last_fullsized":"2024-05-08T23:18:43.793735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:16.824749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"76'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804752+0000","last_change":"2024-05-08T23:17:45.612232+0000","last_active":"2024-05-08T23:18:43.804752+0000","last_peered":"2024-05-08T23:18:43.804752+0000","last_clean":"2024-05-08T23:18:43.804752+0000","last_became_active":"2024-05-08T23:17:45.611896+0000","last_became_peered":"2024-05-08T23:17:45.611896+0000","last_unstale":"2024-05-08T23:18:43.804752+0000","last_undegraded":"2024-05-08T23:18:43.804752+0000","last_fullsized":"2024-05-08T23:18:43.804752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:40.466394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.963335+0000","last_change":"2024-05-08T23:17:46.108094+0000","last_active":"2024-05-08T23:19:37.963335+0000","last_peered":"2024-05-08T23:19:37.963335+0000","last_clean":"2024-05-08T23:19:37.963335+0000","last_became_active":"2024-05-08T23:17:46.107762+0000","last_became_peered":"2024-05-08T23:17:46.107762+0000","last_unstale":"2024-05-08T23:19:37.963335+0000","last_undegraded":"2024-05-08T23:19:37.963335+0000","last_fullsized":"2024-05-08T23:19:37.963335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:51.632270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795805+0000","last_change":"2024-05-08T23:17:50.125588+0000","last_active":"2024-05-08T23:18:43.795805+0000","last_peered":"2024-05-08T23:18:43.795805+0000","last_clean":"2024-05-08T23:18:43.795805+0000","last_became_active":"2024-05-08T23:17:50.125183+0000","last_became_peered":"2024-05-08T23:17:50.125183+0000","last_unstale":"2024-05-08T23:18:43.795805+0000","last_undegraded":"2024-05-08T23:18:43.795805+0000","last_fullsized":"2024-05-08T23:18:43.795805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:49:08.421912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":117,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818584+0000","last_change":"2024-05-08T23:17:45.617408+0000","last_active":"2024-05-08T23:19:38.818584+0000","last_peered":"2024-05-08T23:19:38.818584+0000","last_clean":"2024-05-08T23:19:38.818584+0000","last_became_active":"2024-05-08T23:17:45.617067+0000","last_became_peered":"2024-05-08T23:17:45.617067+0000","last_unstale":"2024-05-08T23:19:38.818584+0000","last_undegraded":"2024-05-08T23:19:38.818584+0000","last_fullsized":"2024-05-08T23:19:38.818584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:44:23.327199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978785+0000","last_change":"2024-05-08T23:17:46.105703+0000","last_active":"2024-05-08T23:19:37.978785+0000","last_peered":"2024-05-08T23:19:37.978785+0000","last_clean":"2024-05-08T23:19:37.978785+0000","last_became_active":"2024-05-08T23:17:46.105336+0000","last_became_peered":"2024-05-08T23:17:46.105336+0000","last_unstale":"2024-05-08T23:19:37.978785+0000","last_undegraded":"2024-05-08T23:19:37.978785+0000","last_fullsized":"2024-05-08T23:19:37.978785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:51:09.005770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.803847+0000","last_change":"2024-05-08T23:18:43.803558+0000","last_active":"2024-05-08T23:18:43.803847+0000","last_peered":"2024-05-08T23:18:43.803847+0000","last_clean":"2024-05-08T23:18:43.803847+0000","last_became_active":"2024-05-08T23:18:43.803167+0000","last_became_peered":"2024-05-08T23:18:43.803167+0000","last_unstale":"2024-05-08T23:18:43.803847+0000","last_undegraded":"2024-05-08T23:18:43.803847+0000","last_fullsized":"2024-05-08T23:18:43.803847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:13:06.378054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797144+0000","last_change":"2024-05-08T23:17:45.618406+0000","last_active":"2024-05-08T23:18:43.797144+0000","last_peered":"2024-05-08T23:18:43.797144+0000","last_clean":"2024-05-08T23:18:43.797144+0000","last_became_active":"2024-05-08T23:17:45.618031+0000","last_became_peered":"2024-05-08T23:17:45.618031+0000","last_unstale":"2024-05-08T23:18:43.797144+0000","last_undegraded":"2024-05-08T23:18:43.797144+0000","last_fullsized":"2024-05-08T23:18:43.797144+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:29:43.958447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.969192+0000","last_change":"2024-05-08T23:17:46.102844+0000","last_active":"2024-05-08T23:19:37.969192+0000","last_peered":"2024-05-08T23:19:37.969192+0000","last_clean":"2024-05-08T23:19:37.969192+0000","last_became_active":"2024-05-08T23:17:46.102500+0000","last_became_peered":"2024-05-08T23:17:46.102500+0000","last_unstale":"2024-05-08T23:19:37.969192+0000","last_undegraded":"2024-05-08T23:19:37.969192+0000","last_fullsized":"2024-05-08T23:19:37.969192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:35.289689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798515+0000","last_change":"2024-05-08T23:17:50.122835+0000","last_active":"2024-05-08T23:18:43.798515+0000","last_peered":"2024-05-08T23:18:43.798515+0000","last_clean":"2024-05-08T23:18:43.798515+0000","last_became_active":"2024-05-08T23:17:50.122380+0000","last_became_peered":"2024-05-08T23:17:50.122380+0000","last_unstale":"2024-05-08T23:18:43.798515+0000","last_undegraded":"2024-05-08T23:18:43.798515+0000","last_fullsized":"2024-05-08T23:18:43.798515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:41.663850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797721+0000","last_change":"2024-05-08T23:17:45.617972+0000","last_active":"2024-05-08T23:18:43.797721+0000","last_peered":"2024-05-08T23:18:43.797721+0000","last_clean":"2024-05-08T23:18:43.797721+0000","last_became_active":"2024-05-08T23:17:45.616044+0000","last_became_peered":"2024-05-08T23:17:45.616044+0000","last_unstale":"2024-05-08T23:18:43.797721+0000","last_undegraded":"2024-05-08T23:18:43.797721+0000","last_fullsized":"2024-05-08T23:18:43.797721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:17:32.785754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977826+0000","last_change":"2024-05-08T23:17:46.104970+0000","last_active":"2024-05-08T23:19:37.977826+0000","last_peered":"2024-05-08T23:19:37.977826+0000","last_clean":"2024-05-08T23:19:37.977826+0000","last_became_active":"2024-05-08T23:17:46.104634+0000","last_became_peered":"2024-05-08T23:17:46.104634+0000","last_unstale":"2024-05-08T23:19:37.977826+0000","last_undegraded":"2024-05-08T23:19:37.977826+0000","last_fullsized":"2024-05-08T23:19:37.977826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:39:18.215075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.091431+0000","last_change":"2024-05-08T23:17:50.129488+0000","last_active":"2024-05-08T23:18:44.091431+0000","last_peered":"2024-05-08T23:18:44.091431+0000","last_clean":"2024-05-08T23:18:44.091431+0000","last_became_active":"2024-05-08T23:17:50.128902+0000","last_became_peered":"2024-05-08T23:17:50.128902+0000","last_unstale":"2024-05-08T23:18:44.091431+0000","last_undegraded":"2024-05-08T23:18:44.091431+0000","last_fullsized":"2024-05-08T23:18:44.091431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:18.125078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795525+0000","last_change":"2024-05-08T23:17:45.613936+0000","last_active":"2024-05-08T23:18:43.795525+0000","last_peered":"2024-05-08T23:18:43.795525+0000","last_clean":"2024-05-08T23:18:43.795525+0000","last_became_active":"2024-05-08T23:17:45.613387+0000","last_became_peered":"2024-05-08T23:17:45.613387+0000","last_unstale":"2024-05-08T23:18:43.795525+0000","last_undegraded":"2024-05-08T23:18:43.795525+0000","last_fullsized":"2024-05-08T23:18:43.795525+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:27.114651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982914+0000","last_change":"2024-05-08T23:17:46.094253+0000","last_active":"2024-05-08T23:19:37.982914+0000","last_peered":"2024-05-08T23:19:37.982914+0000","last_clean":"2024-05-08T23:19:37.982914+0000","last_became_active":"2024-05-08T23:17:46.093538+0000","last_became_peered":"2024-05-08T23:17:46.093538+0000","last_unstale":"2024-05-08T23:19:37.982914+0000","last_undegraded":"2024-05-08T23:19:37.982914+0000","last_fullsized":"2024-05-08T23:19:37.982914+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:52:11.797388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793690+0000","last_change":"2024-05-08T23:17:50.126332+0000","last_active":"2024-05-08T23:18:43.793690+0000","last_peered":"2024-05-08T23:18:43.793690+0000","last_clean":"2024-05-08T23:18:43.793690+0000","last_became_active":"2024-05-08T23:17:50.125750+0000","last_became_peered":"2024-05-08T23:17:50.125750+0000","last_unstale":"2024-05-08T23:18:43.793690+0000","last_undegraded":"2024-05-08T23:18:43.793690+0000","last_fullsized":"2024-05-08T23:18:43.793690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:57:52.008806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":110,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.935026+0000","last_change":"2024-05-08T23:17:45.612094+0000","last_active":"2024-05-08T23:19:37.935026+0000","last_peered":"2024-05-08T23:19:37.935026+0000","last_clean":"2024-05-08T23:19:37.935026+0000","last_became_active":"2024-05-08T23:17:45.611695+0000","last_became_peered":"2024-05-08T23:17:45.611695+0000","last_unstale":"2024-05-08T23:19:37.935026+0000","last_undegraded":"2024-05-08T23:19:37.935026+0000","last_fullsized":"2024-05-08T23:19:37.935026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:16.991873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.981390+0000","last_change":"2024-05-08T23:17:46.115188+0000","last_active":"2024-05-08T23:19:37.981390+0000","last_peered":"2024-05-08T23:19:37.981390+0000","last_clean":"2024-05-08T23:19:37.981390+0000","last_became_active":"2024-05-08T23:17:46.114909+0000","last_became_peered":"2024-05-08T23:17:46.114909+0000","last_unstale":"2024-05-08T23:19:37.981390+0000","last_undegraded":"2024-05-08T23:19:37.981390+0000","last_fullsized":"2024-05-08T23:19:37.981390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:40.650896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.033348+0000","last_change":"2024-05-08T23:17:50.133104+0000","last_active":"2024-05-08T23:18:44.033348+0000","last_peered":"2024-05-08T23:18:44.033348+0000","last_clean":"2024-05-08T23:18:44.033348+0000","last_became_active":"2024-05-08T23:17:50.132763+0000","last_became_peered":"2024-05-08T23:17:50.132763+0000","last_unstale":"2024-05-08T23:18:44.033348+0000","last_undegraded":"2024-05-08T23:18:44.033348+0000","last_fullsized":"2024-05-08T23:18:44.033348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:23:54.043567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797353+0000","last_change":"2024-05-08T23:17:45.618421+0000","last_active":"2024-05-08T23:18:43.797353+0000","last_peered":"2024-05-08T23:18:43.797353+0000","last_clean":"2024-05-08T23:18:43.797353+0000","last_became_active":"2024-05-08T23:17:45.618043+0000","last_became_peered":"2024-05-08T23:17:45.618043+0000","last_unstale":"2024-05-08T23:18:43.797353+0000","last_undegraded":"2024-05-08T23:18:43.797353+0000","last_fullsized":"2024-05-08T23:18:43.797353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:09.420505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975947+0000","last_change":"2024-05-08T23:17:46.098341+0000","last_active":"2024-05-08T23:19:37.975947+0000","last_peered":"2024-05-08T23:19:37.975947+0000","last_clean":"2024-05-08T23:19:37.975947+0000","last_became_active":"2024-05-08T23:17:46.098032+0000","last_became_peered":"2024-05-08T23:17:46.098032+0000","last_unstale":"2024-05-08T23:19:37.975947+0000","last_undegraded":"2024-05-08T23:19:37.975947+0000","last_fullsized":"2024-05-08T23:19:37.975947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:04.799359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794345+0000","last_change":"2024-05-08T23:17:50.134124+0000","last_active":"2024-05-08T23:18:43.794345+0000","last_peered":"2024-05-08T23:18:43.794345+0000","last_clean":"2024-05-08T23:18:43.794345+0000","last_became_active":"2024-05-08T23:17:50.133747+0000","last_became_peered":"2024-05-08T23:17:50.133747+0000","last_unstale":"2024-05-08T23:18:43.794345+0000","last_undegraded":"2024-05-08T23:18:43.794345+0000","last_fullsized":"2024-05-08T23:18:43.794345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:15.922466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795045+0000","last_change":"2024-05-08T23:17:45.616376+0000","last_active":"2024-05-08T23:18:43.795045+0000","last_peered":"2024-05-08T23:18:43.795045+0000","last_clean":"2024-05-08T23:18:43.795045+0000","last_became_active":"2024-05-08T23:17:45.616058+0000","last_became_peered":"2024-05-08T23:17:45.616058+0000","last_unstale":"2024-05-08T23:18:43.795045+0000","last_undegraded":"2024-05-08T23:18:43.795045+0000","last_fullsized":"2024-05-08T23:18:43.795045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:44:23.709375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982345+0000","last_change":"2024-05-08T23:17:46.114793+0000","last_active":"2024-05-08T23:19:37.982345+0000","last_peered":"2024-05-08T23:19:37.982345+0000","last_clean":"2024-05-08T23:19:37.982345+0000","last_became_active":"2024-05-08T23:17:46.114475+0000","last_became_peered":"2024-05-08T23:17:46.114475+0000","last_unstale":"2024-05-08T23:19:37.982345+0000","last_undegraded":"2024-05-08T23:19:37.982345+0000","last_fullsized":"2024-05-08T23:19:37.982345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:18:45.156426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.033715+0000","last_change":"2024-05-08T23:17:50.125365+0000","last_active":"2024-05-08T23:18:44.033715+0000","last_peered":"2024-05-08T23:18:44.033715+0000","last_clean":"2024-05-08T23:18:44.033715+0000","last_became_active":"2024-05-08T23:17:50.124961+0000","last_became_peered":"2024-05-08T23:17:50.124961+0000","last_unstale":"2024-05-08T23:18:44.033715+0000","last_undegraded":"2024-05-08T23:18:44.033715+0000","last_fullsized":"2024-05-08T23:18:44.033715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:15.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'38","reported_seq":240,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818239+0000","last_change":"2024-05-08T23:17:45.618611+0000","last_active":"2024-05-08T23:19:38.818239+0000","last_peered":"2024-05-08T23:19:38.818239+0000","last_clean":"2024-05-08T23:19:38.818239+0000","last_became_active":"2024-05-08T23:17:45.618221+0000","last_became_peered":"2024-05-08T23:17:45.618221+0000","last_unstale":"2024-05-08T23:19:38.818239+0000","last_undegraded":"2024-05-08T23:19:38.818239+0000","last_fullsized":"2024-05-08T23:19:38.818239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:03.423014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62677,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":183,"num_read_kb":153,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973333+0000","last_change":"2024-05-08T23:17:46.112600+0000","last_active":"2024-05-08T23:19:37.973333+0000","last_peered":"2024-05-08T23:19:37.973333+0000","last_clean":"2024-05-08T23:19:37.973333+0000","last_became_active":"2024-05-08T23:17:46.111059+0000","last_became_peered":"2024-05-08T23:17:46.111059+0000","last_unstale":"2024-05-08T23:19:37.973333+0000","last_undegraded":"2024-05-08T23:19:37.973333+0000","last_fullsized":"2024-05-08T23:19:37.973333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:07:31.243699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795420+0000","last_change":"2024-05-08T23:17:50.128089+0000","last_active":"2024-05-08T23:18:43.795420+0000","last_peered":"2024-05-08T23:18:43.795420+0000","last_clean":"2024-05-08T23:18:43.795420+0000","last_became_active":"2024-05-08T23:17:50.127722+0000","last_became_peered":"2024-05-08T23:17:50.127722+0000","last_unstale":"2024-05-08T23:18:43.795420+0000","last_undegraded":"2024-05-08T23:18:43.795420+0000","last_fullsized":"2024-05-08T23:18:43.795420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:37:57.567780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796397+0000","last_change":"2024-05-08T23:17:45.617712+0000","last_active":"2024-05-08T23:18:43.796397+0000","last_peered":"2024-05-08T23:18:43.796397+0000","last_clean":"2024-05-08T23:18:43.796397+0000","last_became_active":"2024-05-08T23:17:45.617304+0000","last_became_peered":"2024-05-08T23:17:45.617304+0000","last_unstale":"2024-05-08T23:18:43.796397+0000","last_undegraded":"2024-05-08T23:18:43.796397+0000","last_fullsized":"2024-05-08T23:18:43.796397+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:49:33.308759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985255+0000","last_change":"2024-05-08T23:17:46.109549+0000","last_active":"2024-05-08T23:19:37.985255+0000","last_peered":"2024-05-08T23:19:37.985255+0000","last_clean":"2024-05-08T23:19:37.985255+0000","last_became_active":"2024-05-08T23:17:46.109006+0000","last_became_peered":"2024-05-08T23:17:46.109006+0000","last_unstale":"2024-05-08T23:19:37.985255+0000","last_undegraded":"2024-05-08T23:19:37.985255+0000","last_fullsized":"2024-05-08T23:19:37.985255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:03.921954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798559+0000","last_change":"2024-05-08T23:17:50.128269+0000","last_active":"2024-05-08T23:18:43.798559+0000","last_peered":"2024-05-08T23:18:43.798559+0000","last_clean":"2024-05-08T23:18:43.798559+0000","last_became_active":"2024-05-08T23:17:50.127906+0000","last_became_peered":"2024-05-08T23:17:50.127906+0000","last_unstale":"2024-05-08T23:18:43.798559+0000","last_undegraded":"2024-05-08T23:18:43.798559+0000","last_fullsized":"2024-05-08T23:18:43.798559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:26:56.990293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795869+0000","last_change":"2024-05-08T23:17:45.617742+0000","last_active":"2024-05-08T23:18:43.795869+0000","last_peered":"2024-05-08T23:18:43.795869+0000","last_clean":"2024-05-08T23:18:43.795869+0000","last_became_active":"2024-05-08T23:17:45.617271+0000","last_became_peered":"2024-05-08T23:17:45.617271+0000","last_unstale":"2024-05-08T23:18:43.795869+0000","last_undegraded":"2024-05-08T23:18:43.795869+0000","last_fullsized":"2024-05-08T23:18:43.795869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.163613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985393+0000","last_change":"2024-05-08T23:18:44.034932+0000","last_active":"2024-05-08T23:19:37.985393+0000","last_peered":"2024-05-08T23:19:37.985393+0000","last_clean":"2024-05-08T23:19:37.985393+0000","last_became_active":"2024-05-08T23:18:44.034439+0000","last_became_peered":"2024-05-08T23:18:44.034439+0000","last_unstale":"2024-05-08T23:19:37.985393+0000","last_undegraded":"2024-05-08T23:19:37.985393+0000","last_fullsized":"2024-05-08T23:19:37.985393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:45.536646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034119+0000","last_change":"2024-05-08T23:17:50.126100+0000","last_active":"2024-05-08T23:18:44.034119+0000","last_peered":"2024-05-08T23:18:44.034119+0000","last_clean":"2024-05-08T23:18:44.034119+0000","last_became_active":"2024-05-08T23:17:50.125708+0000","last_became_peered":"2024-05-08T23:17:50.125708+0000","last_unstale":"2024-05-08T23:18:44.034119+0000","last_undegraded":"2024-05-08T23:18:44.034119+0000","last_fullsized":"2024-05-08T23:18:44.034119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:03:10.704350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.818321+0000","last_change":"2024-05-08T23:17:45.618161+0000","last_active":"2024-05-08T23:19:38.818321+0000","last_peered":"2024-05-08T23:19:38.818321+0000","last_clean":"2024-05-08T23:19:38.818321+0000","last_became_active":"2024-05-08T23:17:45.617834+0000","last_became_peered":"2024-05-08T23:17:45.617834+0000","last_unstale":"2024-05-08T23:19:38.818321+0000","last_undegraded":"2024-05-08T23:19:38.818321+0000","last_fullsized":"2024-05-08T23:19:38.818321+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:44.009874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1822,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973136+0000","last_change":"2024-05-08T23:17:46.109629+0000","last_active":"2024-05-08T23:19:37.973136+0000","last_peered":"2024-05-08T23:19:37.973136+0000","last_clean":"2024-05-08T23:19:37.973136+0000","last_became_active":"2024-05-08T23:17:46.109026+0000","last_became_peered":"2024-05-08T23:17:46.109026+0000","last_unstale":"2024-05-08T23:19:37.973136+0000","last_undegraded":"2024-05-08T23:19:37.973136+0000","last_fullsized":"2024-05-08T23:19:37.973136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:20:18.366736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798551+0000","last_change":"2024-05-08T23:17:50.131608+0000","last_active":"2024-05-08T23:18:43.798551+0000","last_peered":"2024-05-08T23:18:43.798551+0000","last_clean":"2024-05-08T23:18:43.798551+0000","last_became_active":"2024-05-08T23:17:50.131116+0000","last_became_peered":"2024-05-08T23:17:50.131116+0000","last_unstale":"2024-05-08T23:18:43.798551+0000","last_undegraded":"2024-05-08T23:18:43.798551+0000","last_fullsized":"2024-05-08T23:18:43.798551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:59.698102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804250+0000","last_change":"2024-05-08T23:17:45.612476+0000","last_active":"2024-05-08T23:18:43.804250+0000","last_peered":"2024-05-08T23:18:43.804250+0000","last_clean":"2024-05-08T23:18:43.804250+0000","last_became_active":"2024-05-08T23:17:45.611442+0000","last_became_peered":"2024-05-08T23:17:45.611442+0000","last_unstale":"2024-05-08T23:18:43.804250+0000","last_undegraded":"2024-05-08T23:18:43.804250+0000","last_fullsized":"2024-05-08T23:18:43.804250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:50.811709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.962016+0000","last_change":"2024-05-08T23:18:43.929400+0000","last_active":"2024-05-08T23:19:37.962016+0000","last_peered":"2024-05-08T23:19:37.962016+0000","last_clean":"2024-05-08T23:19:37.962016+0000","last_became_active":"2024-05-08T23:18:43.928980+0000","last_became_peered":"2024-05-08T23:18:43.928980+0000","last_unstale":"2024-05-08T23:19:37.962016+0000","last_undegraded":"2024-05-08T23:19:37.962016+0000","last_fullsized":"2024-05-08T23:19:37.962016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:10.433725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795293+0000","last_change":"2024-05-08T23:17:50.122491+0000","last_active":"2024-05-08T23:18:43.795293+0000","last_peered":"2024-05-08T23:18:43.795293+0000","last_clean":"2024-05-08T23:18:43.795293+0000","last_became_active":"2024-05-08T23:17:50.122136+0000","last_became_peered":"2024-05-08T23:17:50.122136+0000","last_unstale":"2024-05-08T23:18:43.795293+0000","last_undegraded":"2024-05-08T23:18:43.795293+0000","last_fullsized":"2024-05-08T23:18:43.795293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:23:45.344316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804811+0000","last_change":"2024-05-08T23:17:45.617814+0000","last_active":"2024-05-08T23:18:43.804811+0000","last_peered":"2024-05-08T23:18:43.804811+0000","last_clean":"2024-05-08T23:18:43.804811+0000","last_became_active":"2024-05-08T23:17:45.616543+0000","last_became_peered":"2024-05-08T23:17:45.616543+0000","last_unstale":"2024-05-08T23:18:43.804811+0000","last_undegraded":"2024-05-08T23:18:43.804811+0000","last_fullsized":"2024-05-08T23:18:43.804811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:51:21.291461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973319+0000","last_change":"2024-05-08T23:17:46.097972+0000","last_active":"2024-05-08T23:19:37.973319+0000","last_peered":"2024-05-08T23:19:37.973319+0000","last_clean":"2024-05-08T23:19:37.973319+0000","last_became_active":"2024-05-08T23:17:46.097587+0000","last_became_peered":"2024-05-08T23:17:46.097587+0000","last_unstale":"2024-05-08T23:19:37.973319+0000","last_undegraded":"2024-05-08T23:19:37.973319+0000","last_fullsized":"2024-05-08T23:19:37.973319+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:31:27.786509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804513+0000","last_change":"2024-05-08T23:17:50.244053+0000","last_active":"2024-05-08T23:18:43.804513+0000","last_peered":"2024-05-08T23:18:43.804513+0000","last_clean":"2024-05-08T23:18:43.804513+0000","last_became_active":"2024-05-08T23:17:50.243639+0000","last_became_peered":"2024-05-08T23:17:50.243639+0000","last_unstale":"2024-05-08T23:18:43.804513+0000","last_undegraded":"2024-05-08T23:18:43.804513+0000","last_fullsized":"2024-05-08T23:18:43.804513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.515072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034555+0000","last_change":"2024-05-08T23:17:45.615383+0000","last_active":"2024-05-08T23:18:44.034555+0000","last_peered":"2024-05-08T23:18:44.034555+0000","last_clean":"2024-05-08T23:18:44.034555+0000","last_became_active":"2024-05-08T23:17:45.614956+0000","last_became_peered":"2024-05-08T23:17:45.614956+0000","last_unstale":"2024-05-08T23:18:44.034555+0000","last_undegraded":"2024-05-08T23:18:44.034555+0000","last_fullsized":"2024-05-08T23:18:44.034555+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.416195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979180+0000","last_change":"2024-05-08T23:17:46.109009+0000","last_active":"2024-05-08T23:19:37.979180+0000","last_peered":"2024-05-08T23:19:37.979180+0000","last_clean":"2024-05-08T23:19:37.979180+0000","last_became_active":"2024-05-08T23:17:46.108656+0000","last_became_peered":"2024-05-08T23:17:46.108656+0000","last_unstale":"2024-05-08T23:19:37.979180+0000","last_undegraded":"2024-05-08T23:19:37.979180+0000","last_fullsized":"2024-05-08T23:19:37.979180+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:53.588954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798335+0000","last_change":"2024-05-08T23:17:50.134838+0000","last_active":"2024-05-08T23:18:43.798335+0000","last_peered":"2024-05-08T23:18:43.798335+0000","last_clean":"2024-05-08T23:18:43.798335+0000","last_became_active":"2024-05-08T23:17:50.134321+0000","last_became_peered":"2024-05-08T23:17:50.134321+0000","last_unstale":"2024-05-08T23:18:43.798335+0000","last_undegraded":"2024-05-08T23:18:43.798335+0000","last_fullsized":"2024-05-08T23:18:43.798335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:27.243169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797081+0000","last_change":"2024-05-08T23:17:45.621837+0000","last_active":"2024-05-08T23:18:43.797081+0000","last_peered":"2024-05-08T23:18:43.797081+0000","last_clean":"2024-05-08T23:18:43.797081+0000","last_became_active":"2024-05-08T23:17:45.621491+0000","last_became_peered":"2024-05-08T23:17:45.621491+0000","last_unstale":"2024-05-08T23:18:43.797081+0000","last_undegraded":"2024-05-08T23:18:43.797081+0000","last_fullsized":"2024-05-08T23:18:43.797081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:10:12.178175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.970690+0000","last_change":"2024-05-08T23:17:46.104955+0000","last_active":"2024-05-08T23:19:37.970690+0000","last_peered":"2024-05-08T23:19:37.970690+0000","last_clean":"2024-05-08T23:19:37.970690+0000","last_became_active":"2024-05-08T23:17:46.104609+0000","last_became_peered":"2024-05-08T23:17:46.104609+0000","last_unstale":"2024-05-08T23:19:37.970690+0000","last_undegraded":"2024-05-08T23:19:37.970690+0000","last_fullsized":"2024-05-08T23:19:37.970690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:27:17.246778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798132+0000","last_change":"2024-05-08T23:17:50.244063+0000","last_active":"2024-05-08T23:18:43.798132+0000","last_peered":"2024-05-08T23:18:43.798132+0000","last_clean":"2024-05-08T23:18:43.798132+0000","last_became_active":"2024-05-08T23:17:50.243585+0000","last_became_peered":"2024-05-08T23:17:50.243585+0000","last_unstale":"2024-05-08T23:18:43.798132+0000","last_undegraded":"2024-05-08T23:18:43.798132+0000","last_fullsized":"2024-05-08T23:18:43.798132+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:07:24.578327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973657+0000","last_change":"2024-05-08T23:17:46.110613+0000","last_active":"2024-05-08T23:19:37.973657+0000","last_peered":"2024-05-08T23:19:37.973657+0000","last_clean":"2024-05-08T23:19:37.973657+0000","last_became_active":"2024-05-08T23:17:46.110215+0000","last_became_peered":"2024-05-08T23:17:46.110215+0000","last_unstale":"2024-05-08T23:19:37.973657+0000","last_undegraded":"2024-05-08T23:19:37.973657+0000","last_fullsized":"2024-05-08T23:19:37.973657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:37.684004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":111,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.987062+0000","last_change":"2024-05-08T23:17:45.623933+0000","last_active":"2024-05-08T23:19:37.987062+0000","last_peered":"2024-05-08T23:19:37.987062+0000","last_clean":"2024-05-08T23:19:37.987062+0000","last_became_active":"2024-05-08T23:17:45.623666+0000","last_became_peered":"2024-05-08T23:17:45.623666+0000","last_unstale":"2024-05-08T23:19:37.987062+0000","last_undegraded":"2024-05-08T23:19:37.987062+0000","last_fullsized":"2024-05-08T23:19:37.987062+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:16.314961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797552+0000","last_change":"2024-05-08T23:17:50.124275+0000","last_active":"2024-05-08T23:18:43.797552+0000","last_peered":"2024-05-08T23:18:43.797552+0000","last_clean":"2024-05-08T23:18:43.797552+0000","last_became_active":"2024-05-08T23:17:50.123825+0000","last_became_peered":"2024-05-08T23:17:50.123825+0000","last_unstale":"2024-05-08T23:18:43.797552+0000","last_undegraded":"2024-05-08T23:18:43.797552+0000","last_fullsized":"2024-05-08T23:18:43.797552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:48.222555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979618+0000","last_change":"2024-05-08T23:17:46.106437+0000","last_active":"2024-05-08T23:19:37.979618+0000","last_peered":"2024-05-08T23:19:37.979618+0000","last_clean":"2024-05-08T23:19:37.979618+0000","last_became_active":"2024-05-08T23:17:46.106127+0000","last_became_peered":"2024-05-08T23:17:46.106127+0000","last_unstale":"2024-05-08T23:19:37.979618+0000","last_undegraded":"2024-05-08T23:19:37.979618+0000","last_fullsized":"2024-05-08T23:19:37.979618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:29.164227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794337+0000","last_change":"2024-05-08T23:17:45.604145+0000","last_active":"2024-05-08T23:18:43.794337+0000","last_peered":"2024-05-08T23:18:43.794337+0000","last_clean":"2024-05-08T23:18:43.794337+0000","last_became_active":"2024-05-08T23:17:45.603708+0000","last_became_peered":"2024-05-08T23:17:45.603708+0000","last_unstale":"2024-05-08T23:18:43.794337+0000","last_undegraded":"2024-05-08T23:18:43.794337+0000","last_fullsized":"2024-05-08T23:18:43.794337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:54:22.826894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":336,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796725+0000","last_change":"2024-05-08T23:16:54.541099+0000","last_active":"2024-05-08T23:18:43.796725+0000","last_peered":"2024-05-08T23:18:43.796725+0000","last_clean":"2024-05-08T23:18:43.796725+0000","last_became_active":"2024-05-08T23:16:54.540676+0000","last_became_peered":"2024-05-08T23:16:54.540676+0000","last_unstale":"2024-05-08T23:18:43.796725+0000","last_undegraded":"2024-05-08T23:18:43.796725+0000","last_fullsized":"2024-05-08T23:18:43.796725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:14:33.043966+0000","last_clean_scrub_stamp":"2024-05-08T23:14:33.043966+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:07.353593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795283+0000","last_change":"2024-05-08T23:17:50.244514+0000","last_active":"2024-05-08T23:18:43.795283+0000","last_peered":"2024-05-08T23:18:43.795283+0000","last_clean":"2024-05-08T23:18:43.795283+0000","last_became_active":"2024-05-08T23:17:50.244085+0000","last_became_peered":"2024-05-08T23:17:50.244085+0000","last_unstale":"2024-05-08T23:18:43.795283+0000","last_undegraded":"2024-05-08T23:18:43.795283+0000","last_fullsized":"2024-05-08T23:18:43.795283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:04.335753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.976441+0000","last_change":"2024-05-08T23:17:46.108532+0000","last_active":"2024-05-08T23:19:37.976441+0000","last_peered":"2024-05-08T23:19:37.976441+0000","last_clean":"2024-05-08T23:19:37.976441+0000","last_became_active":"2024-05-08T23:17:46.107953+0000","last_became_peered":"2024-05-08T23:17:46.107953+0000","last_unstale":"2024-05-08T23:19:37.976441+0000","last_undegraded":"2024-05-08T23:19:37.976441+0000","last_fullsized":"2024-05-08T23:19:37.976441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:05:30.307887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796803+0000","last_change":"2024-05-08T23:17:45.619091+0000","last_active":"2024-05-08T23:18:43.796803+0000","last_peered":"2024-05-08T23:18:43.796803+0000","last_clean":"2024-05-08T23:18:43.796803+0000","last_became_active":"2024-05-08T23:17:45.618620+0000","last_became_peered":"2024-05-08T23:17:45.618620+0000","last_unstale":"2024-05-08T23:18:43.796803+0000","last_undegraded":"2024-05-08T23:18:43.796803+0000","last_fullsized":"2024-05-08T23:18:43.796803+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:26.377818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.798937+0000","last_change":"2024-05-08T23:17:50.131448+0000","last_active":"2024-05-08T23:18:43.798937+0000","last_peered":"2024-05-08T23:18:43.798937+0000","last_clean":"2024-05-08T23:18:43.798937+0000","last_became_active":"2024-05-08T23:17:50.131029+0000","last_became_peered":"2024-05-08T23:17:50.131029+0000","last_unstale":"2024-05-08T23:18:43.798937+0000","last_undegraded":"2024-05-08T23:18:43.798937+0000","last_fullsized":"2024-05-08T23:18:43.798937+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:18.672933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.804873+0000","last_change":"2024-05-08T23:17:45.615666+0000","last_active":"2024-05-08T23:18:43.804873+0000","last_peered":"2024-05-08T23:18:43.804873+0000","last_clean":"2024-05-08T23:18:43.804873+0000","last_became_active":"2024-05-08T23:17:45.615155+0000","last_became_peered":"2024-05-08T23:17:45.615155+0000","last_unstale":"2024-05-08T23:18:43.804873+0000","last_undegraded":"2024-05-08T23:18:43.804873+0000","last_fullsized":"2024-05-08T23:18:43.804873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:45.505718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.983517+0000","last_change":"2024-05-08T23:17:46.108161+0000","last_active":"2024-05-08T23:19:37.983517+0000","last_peered":"2024-05-08T23:19:37.983517+0000","last_clean":"2024-05-08T23:19:37.983517+0000","last_became_active":"2024-05-08T23:17:46.107869+0000","last_became_peered":"2024-05-08T23:17:46.107869+0000","last_unstale":"2024-05-08T23:19:37.983517+0000","last_undegraded":"2024-05-08T23:19:37.983517+0000","last_fullsized":"2024-05-08T23:19:37.983517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:50.616054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793564+0000","last_change":"2024-05-08T23:17:50.244236+0000","last_active":"2024-05-08T23:18:43.793564+0000","last_peered":"2024-05-08T23:18:43.793564+0000","last_clean":"2024-05-08T23:18:43.793564+0000","last_became_active":"2024-05-08T23:17:50.243797+0000","last_became_peered":"2024-05-08T23:17:50.243797+0000","last_unstale":"2024-05-08T23:18:43.793564+0000","last_undegraded":"2024-05-08T23:18:43.793564+0000","last_fullsized":"2024-05-08T23:18:43.793564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:20.830901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'40","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.200521+0000","last_change":"2024-05-08T23:17:45.612063+0000","last_active":"2024-05-08T23:19:38.200521+0000","last_peered":"2024-05-08T23:19:38.200521+0000","last_clean":"2024-05-08T23:19:38.200521+0000","last_became_active":"2024-05-08T23:17:45.611685+0000","last_became_peered":"2024-05-08T23:17:45.611685+0000","last_unstale":"2024-05-08T23:19:38.200521+0000","last_undegraded":"2024-05-08T23:19:38.200521+0000","last_fullsized":"2024-05-08T23:19:38.200521+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:15:33.119188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":71,"num_write_kb":834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974398+0000","last_change":"2024-05-08T23:17:46.107302+0000","last_active":"2024-05-08T23:19:37.974398+0000","last_peered":"2024-05-08T23:19:37.974398+0000","last_clean":"2024-05-08T23:19:37.974398+0000","last_became_active":"2024-05-08T23:17:46.106929+0000","last_became_peered":"2024-05-08T23:17:46.106929+0000","last_unstale":"2024-05-08T23:19:37.974398+0000","last_undegraded":"2024-05-08T23:19:37.974398+0000","last_fullsized":"2024-05-08T23:19:37.974398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:55.543820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.795943+0000","last_change":"2024-05-08T23:17:50.125917+0000","last_active":"2024-05-08T23:18:43.795943+0000","last_peered":"2024-05-08T23:18:43.795943+0000","last_clean":"2024-05-08T23:18:43.795943+0000","last_became_active":"2024-05-08T23:17:50.125534+0000","last_became_peered":"2024-05-08T23:17:50.125534+0000","last_unstale":"2024-05-08T23:18:43.795943+0000","last_undegraded":"2024-05-08T23:18:43.795943+0000","last_fullsized":"2024-05-08T23:18:43.795943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:22.507462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794461+0000","last_change":"2024-05-08T23:17:45.619848+0000","last_active":"2024-05-08T23:18:43.794461+0000","last_peered":"2024-05-08T23:18:43.794461+0000","last_clean":"2024-05-08T23:18:43.794461+0000","last_became_active":"2024-05-08T23:17:45.615731+0000","last_became_peered":"2024-05-08T23:17:45.615731+0000","last_unstale":"2024-05-08T23:18:43.794461+0000","last_undegraded":"2024-05-08T23:18:43.794461+0000","last_fullsized":"2024-05-08T23:18:43.794461+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:08.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.964151+0000","last_change":"2024-05-08T23:17:46.091959+0000","last_active":"2024-05-08T23:19:37.964151+0000","last_peered":"2024-05-08T23:19:37.964151+0000","last_clean":"2024-05-08T23:19:37.964151+0000","last_became_active":"2024-05-08T23:17:46.091637+0000","last_became_peered":"2024-05-08T23:17:46.091637+0000","last_unstale":"2024-05-08T23:19:37.964151+0000","last_undegraded":"2024-05-08T23:19:37.964151+0000","last_fullsized":"2024-05-08T23:19:37.964151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:47.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793979+0000","last_change":"2024-05-08T23:17:50.244993+0000","last_active":"2024-05-08T23:18:43.793979+0000","last_peered":"2024-05-08T23:18:43.793979+0000","last_clean":"2024-05-08T23:18:43.793979+0000","last_became_active":"2024-05-08T23:17:50.244008+0000","last_became_peered":"2024-05-08T23:17:50.244008+0000","last_unstale":"2024-05-08T23:18:43.793979+0000","last_undegraded":"2024-05-08T23:18:43.793979+0000","last_fullsized":"2024-05-08T23:18:43.793979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:12:27.341712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.927256+0000","last_change":"2024-05-08T23:17:45.616093+0000","last_active":"2024-05-08T23:18:43.927256+0000","last_peered":"2024-05-08T23:18:43.927256+0000","last_clean":"2024-05-08T23:18:43.927256+0000","last_became_active":"2024-05-08T23:17:45.615561+0000","last_became_peered":"2024-05-08T23:17:45.615561+0000","last_unstale":"2024-05-08T23:18:43.927256+0000","last_undegraded":"2024-05-08T23:18:43.927256+0000","last_fullsized":"2024-05-08T23:18:43.927256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:00.091776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.966650+0000","last_change":"2024-05-08T23:17:46.099356+0000","last_active":"2024-05-08T23:19:37.966650+0000","last_peered":"2024-05-08T23:19:37.966650+0000","last_clean":"2024-05-08T23:19:37.966650+0000","last_became_active":"2024-05-08T23:17:46.099037+0000","last_became_peered":"2024-05-08T23:17:46.099037+0000","last_unstale":"2024-05-08T23:19:37.966650+0000","last_undegraded":"2024-05-08T23:19:37.966650+0000","last_fullsized":"2024-05-08T23:19:37.966650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:22:21.183095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.034275+0000","last_change":"2024-05-08T23:17:50.126042+0000","last_active":"2024-05-08T23:18:44.034275+0000","last_peered":"2024-05-08T23:18:44.034275+0000","last_clean":"2024-05-08T23:18:44.034275+0000","last_became_active":"2024-05-08T23:17:50.125652+0000","last_became_peered":"2024-05-08T23:17:50.125652+0000","last_unstale":"2024-05-08T23:18:44.034275+0000","last_undegraded":"2024-05-08T23:18:44.034275+0000","last_fullsized":"2024-05-08T23:18:44.034275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:38:32.016698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793711+0000","last_change":"2024-05-08T23:17:45.613509+0000","last_active":"2024-05-08T23:18:43.793711+0000","last_peered":"2024-05-08T23:18:43.793711+0000","last_clean":"2024-05-08T23:18:43.793711+0000","last_became_active":"2024-05-08T23:17:45.613113+0000","last_became_peered":"2024-05-08T23:17:45.613113+0000","last_unstale":"2024-05-08T23:18:43.793711+0000","last_undegraded":"2024-05-08T23:18:43.793711+0000","last_fullsized":"2024-05-08T23:18:43.793711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:11.936384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.975082+0000","last_change":"2024-05-08T23:17:46.103467+0000","last_active":"2024-05-08T23:19:37.975082+0000","last_peered":"2024-05-08T23:19:37.975082+0000","last_clean":"2024-05-08T23:19:37.975082+0000","last_became_active":"2024-05-08T23:17:46.102994+0000","last_became_peered":"2024-05-08T23:17:46.102994+0000","last_unstale":"2024-05-08T23:19:37.975082+0000","last_undegraded":"2024-05-08T23:19:37.975082+0000","last_fullsized":"2024-05-08T23:19:37.975082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:45.468689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"81'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.793856+0000","last_change":"2024-05-08T23:17:50.131644+0000","last_active":"2024-05-08T23:18:43.793856+0000","last_peered":"2024-05-08T23:18:43.793856+0000","last_clean":"2024-05-08T23:18:43.793856+0000","last_became_active":"2024-05-08T23:17:50.131209+0000","last_became_peered":"2024-05-08T23:17:50.131209+0000","last_unstale":"2024-05-08T23:18:43.793856+0000","last_undegraded":"2024-05-08T23:18:43.793856+0000","last_fullsized":"2024-05-08T23:18:43.793856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:17:43.900411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.796671+0000","last_change":"2024-05-08T23:17:45.615644+0000","last_active":"2024-05-08T23:18:43.796671+0000","last_peered":"2024-05-08T23:18:43.796671+0000","last_clean":"2024-05-08T23:18:43.796671+0000","last_became_active":"2024-05-08T23:17:45.614285+0000","last_became_peered":"2024-05-08T23:17:45.614285+0000","last_unstale":"2024-05-08T23:18:43.796671+0000","last_undegraded":"2024-05-08T23:18:43.796671+0000","last_fullsized":"2024-05-08T23:18:43.796671+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:08.462402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.967303+0000","last_change":"2024-05-08T23:17:46.093807+0000","last_active":"2024-05-08T23:19:37.967303+0000","last_peered":"2024-05-08T23:19:37.967303+0000","last_clean":"2024-05-08T23:19:37.967303+0000","last_became_active":"2024-05-08T23:17:46.093493+0000","last_became_peered":"2024-05-08T23:17:46.093493+0000","last_unstale":"2024-05-08T23:19:37.967303+0000","last_undegraded":"2024-05-08T23:19:37.967303+0000","last_fullsized":"2024-05-08T23:19:37.967303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:55.064437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794095+0000","last_change":"2024-05-08T23:17:50.245420+0000","last_active":"2024-05-08T23:18:43.794095+0000","last_peered":"2024-05-08T23:18:43.794095+0000","last_clean":"2024-05-08T23:18:43.794095+0000","last_became_active":"2024-05-08T23:17:50.243854+0000","last_became_peered":"2024-05-08T23:17:50.243854+0000","last_unstale":"2024-05-08T23:18:43.794095+0000","last_undegraded":"2024-05-08T23:18:43.794095+0000","last_fullsized":"2024-05-08T23:18:43.794095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:10:49.081717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:45.635532+0000","last_change":"2024-05-08T23:18:43.807691+0000","last_active":"2024-05-08T23:18:45.635532+0000","last_peered":"2024-05-08T23:18:45.635532+0000","last_clean":"2024-05-08T23:18:45.635532+0000","last_became_active":"2024-05-08T23:18:43.801858+0000","last_became_peered":"2024-05-08T23:18:43.801858+0000","last_unstale":"2024-05-08T23:18:45.635532+0000","last_undegraded":"2024-05-08T23:18:45.635532+0000","last_fullsized":"2024-05-08T23:18:45.635532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:34:13.427876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985346+0000","last_change":"2024-05-08T23:17:46.107072+0000","last_active":"2024-05-08T23:19:37.985346+0000","last_peered":"2024-05-08T23:19:37.985346+0000","last_clean":"2024-05-08T23:19:37.985346+0000","last_became_active":"2024-05-08T23:17:46.106752+0000","last_became_peered":"2024-05-08T23:17:46.106752+0000","last_unstale":"2024-05-08T23:19:37.985346+0000","last_undegraded":"2024-05-08T23:19:37.985346+0000","last_fullsized":"2024-05-08T23:19:37.985346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:54.189276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.927268+0000","last_change":"2024-05-08T23:17:50.130800+0000","last_active":"2024-05-08T23:18:43.927268+0000","last_peered":"2024-05-08T23:18:43.927268+0000","last_clean":"2024-05-08T23:18:43.927268+0000","last_became_active":"2024-05-08T23:17:50.130463+0000","last_became_peered":"2024-05-08T23:17:50.130463+0000","last_unstale":"2024-05-08T23:18:43.927268+0000","last_undegraded":"2024-05-08T23:18:43.927268+0000","last_fullsized":"2024-05-08T23:18:43.927268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:05.753309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.797148+0000","last_change":"2024-05-08T23:17:45.615761+0000","last_active":"2024-05-08T23:18:43.797148+0000","last_peered":"2024-05-08T23:18:43.797148+0000","last_clean":"2024-05-08T23:18:43.797148+0000","last_became_active":"2024-05-08T23:17:45.615385+0000","last_became_peered":"2024-05-08T23:17:45.615385+0000","last_unstale":"2024-05-08T23:18:43.797148+0000","last_undegraded":"2024-05-08T23:18:43.797148+0000","last_fullsized":"2024-05-08T23:18:43.797148+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:42:09.737467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.980423+0000","last_change":"2024-05-08T23:17:46.107539+0000","last_active":"2024-05-08T23:19:37.980423+0000","last_peered":"2024-05-08T23:19:37.980423+0000","last_clean":"2024-05-08T23:19:37.980423+0000","last_became_active":"2024-05-08T23:17:46.107223+0000","last_became_peered":"2024-05-08T23:17:46.107223+0000","last_unstale":"2024-05-08T23:19:37.980423+0000","last_undegraded":"2024-05-08T23:19:37.980423+0000","last_fullsized":"2024-05-08T23:19:37.980423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:19.922417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:44.091302+0000","last_change":"2024-05-08T23:17:50.134388+0000","last_active":"2024-05-08T23:18:44.091302+0000","last_peered":"2024-05-08T23:18:44.091302+0000","last_clean":"2024-05-08T23:18:44.091302+0000","last_became_active":"2024-05-08T23:17:50.133899+0000","last_became_peered":"2024-05-08T23:17:50.133899+0000","last_unstale":"2024-05-08T23:18:44.091302+0000","last_undegraded":"2024-05-08T23:18:44.091302+0000","last_fullsized":"2024-05-08T23:18:44.091302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:31:57.764000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:38.197927+0000","last_change":"2024-05-08T23:17:45.614255+0000","last_active":"2024-05-08T23:19:38.197927+0000","last_peered":"2024-05-08T23:19:38.197927+0000","last_clean":"2024-05-08T23:19:38.197927+0000","last_became_active":"2024-05-08T23:17:45.611251+0000","last_became_peered":"2024-05-08T23:17:45.611251+0000","last_unstale":"2024-05-08T23:19:38.197927+0000","last_undegraded":"2024-05-08T23:19:38.197927+0000","last_fullsized":"2024-05-08T23:19:38.197927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:42.013128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979495+0000","last_change":"2024-05-08T23:17:46.112997+0000","last_active":"2024-05-08T23:19:37.979495+0000","last_peered":"2024-05-08T23:19:37.979495+0000","last_clean":"2024-05-08T23:19:37.979495+0000","last_became_active":"2024-05-08T23:17:46.112534+0000","last_became_peered":"2024-05-08T23:17:46.112534+0000","last_unstale":"2024-05-08T23:19:37.979495+0000","last_undegraded":"2024-05-08T23:19:37.979495+0000","last_fullsized":"2024-05-08T23:19:37.979495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:28:30.700155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.794917+0000","last_change":"2024-05-08T23:17:50.125471+0000","last_active":"2024-05-08T23:18:43.794917+0000","last_peered":"2024-05-08T23:18:43.794917+0000","last_clean":"2024-05-08T23:18:43.794917+0000","last_became_active":"2024-05-08T23:17:50.125044+0000","last_became_peered":"2024-05-08T23:17:50.125044+0000","last_unstale":"2024-05-08T23:18:43.794917+0000","last_undegraded":"2024-05-08T23:18:43.794917+0000","last_fullsized":"2024-05-08T23:18:43.794917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:49.094411+0000","last_clean_scrub_stamp":"2024-05-08T23:17:49.094411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.220835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"76'2","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-08T23:18:43.931822+0000","last_change":"2024-05-08T23:18:43.931402+0000","last_active":"2024-05-08T23:18:43.931822+0000","last_peered":"2024-05-08T23:18:43.931822+0000","last_clean":"2024-05-08T23:18:43.931822+0000","last_became_active":"2024-05-08T23:18:43.928793+0000","last_became_peered":"2024-05-08T23:18:43.928793+0000","last_unstale":"2024-05-08T23:18:43.931822+0000","last_undegraded":"2024-05-08T23:18:43.931822+0000","last_fullsized":"2024-05-08T23:18:43.931822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:44.586943+0000","last_clean_scrub_stamp":"2024-05-08T23:17:44.586943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:37:52.588272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.972088+0000","last_change":"2024-05-08T23:17:46.115502+0000","last_active":"2024-05-08T23:19:37.972088+0000","last_peered":"2024-05-08T23:19:37.972088+0000","last_clean":"2024-05-08T23:19:37.972088+0000","last_became_active":"2024-05-08T23:17:46.115187+0000","last_became_peered":"2024-05-08T23:17:46.115187+0000","last_unstale":"2024-05-08T23:19:37.972088+0000","last_undegraded":"2024-05-08T23:19:37.972088+0000","last_fullsized":"2024-05-08T23:19:37.972088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:37.002250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.974459+0000","last_change":"2024-05-08T23:17:46.103810+0000","last_active":"2024-05-08T23:19:37.974459+0000","last_peered":"2024-05-08T23:19:37.974459+0000","last_clean":"2024-05-08T23:19:37.974459+0000","last_became_active":"2024-05-08T23:17:46.103493+0000","last_became_peered":"2024-05-08T23:17:46.103493+0000","last_unstale":"2024-05-08T23:19:37.974459+0000","last_undegraded":"2024-05-08T23:19:37.974459+0000","last_fullsized":"2024-05-08T23:19:37.974459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:06.320580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.979844+0000","last_change":"2024-05-08T23:18:44.034679+0000","last_active":"2024-05-08T23:19:37.979844+0000","last_peered":"2024-05-08T23:19:37.979844+0000","last_clean":"2024-05-08T23:19:37.979844+0000","last_became_active":"2024-05-08T23:18:44.034370+0000","last_became_peered":"2024-05-08T23:18:44.034370+0000","last_unstale":"2024-05-08T23:19:37.979844+0000","last_undegraded":"2024-05-08T23:19:37.979844+0000","last_fullsized":"2024-05-08T23:19:37.979844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:43:47.843111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.963986+0000","last_change":"2024-05-08T23:17:46.107941+0000","last_active":"2024-05-08T23:19:37.963986+0000","last_peered":"2024-05-08T23:19:37.963986+0000","last_clean":"2024-05-08T23:19:37.963986+0000","last_became_active":"2024-05-08T23:17:46.107582+0000","last_became_peered":"2024-05-08T23:17:46.107582+0000","last_unstale":"2024-05-08T23:19:37.963986+0000","last_undegraded":"2024-05-08T23:19:37.963986+0000","last_fullsized":"2024-05-08T23:19:37.963986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:25:25.115188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.985739+0000","last_change":"2024-05-08T23:17:46.092596+0000","last_active":"2024-05-08T23:19:37.985739+0000","last_peered":"2024-05-08T23:19:37.985739+0000","last_clean":"2024-05-08T23:19:37.985739+0000","last_became_active":"2024-05-08T23:17:46.092223+0000","last_became_peered":"2024-05-08T23:17:46.092223+0000","last_unstale":"2024-05-08T23:19:37.985739+0000","last_undegraded":"2024-05-08T23:19:37.985739+0000","last_fullsized":"2024-05-08T23:19:37.985739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:30.238182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982612+0000","last_change":"2024-05-08T23:17:46.098837+0000","last_active":"2024-05-08T23:19:37.982612+0000","last_peered":"2024-05-08T23:19:37.982612+0000","last_clean":"2024-05-08T23:19:37.982612+0000","last_became_active":"2024-05-08T23:17:46.098327+0000","last_became_peered":"2024-05-08T23:17:46.098327+0000","last_unstale":"2024-05-08T23:19:37.982612+0000","last_undegraded":"2024-05-08T23:19:37.982612+0000","last_fullsized":"2024-05-08T23:19:37.982612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:28:38.697736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.973494+0000","last_change":"2024-05-08T23:17:46.102759+0000","last_active":"2024-05-08T23:19:37.973494+0000","last_peered":"2024-05-08T23:19:37.973494+0000","last_clean":"2024-05-08T23:19:37.973494+0000","last_became_active":"2024-05-08T23:17:46.101850+0000","last_became_peered":"2024-05-08T23:17:46.101850+0000","last_unstale":"2024-05-08T23:19:37.973494+0000","last_undegraded":"2024-05-08T23:19:37.973494+0000","last_fullsized":"2024-05-08T23:19:37.973494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:36:21.319139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.972939+0000","last_change":"2024-05-08T23:17:46.112556+0000","last_active":"2024-05-08T23:19:37.972939+0000","last_peered":"2024-05-08T23:19:37.972939+0000","last_clean":"2024-05-08T23:19:37.972939+0000","last_became_active":"2024-05-08T23:17:46.112233+0000","last_became_peered":"2024-05-08T23:17:46.112233+0000","last_unstale":"2024-05-08T23:19:37.972939+0000","last_undegraded":"2024-05-08T23:19:37.972939+0000","last_fullsized":"2024-05-08T23:19:37.972939+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:33.114350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.960851+0000","last_change":"2024-05-08T23:17:46.109490+0000","last_active":"2024-05-08T23:19:37.960851+0000","last_peered":"2024-05-08T23:19:37.960851+0000","last_clean":"2024-05-08T23:19:37.960851+0000","last_became_active":"2024-05-08T23:17:46.109120+0000","last_became_peered":"2024-05-08T23:17:46.109120+0000","last_unstale":"2024-05-08T23:19:37.960851+0000","last_undegraded":"2024-05-08T23:19:37.960851+0000","last_fullsized":"2024-05-08T23:19:37.960851+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:53:08.393874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.978702+0000","last_change":"2024-05-08T23:17:46.102237+0000","last_active":"2024-05-08T23:19:37.978702+0000","last_peered":"2024-05-08T23:19:37.978702+0000","last_clean":"2024-05-08T23:19:37.978702+0000","last_became_active":"2024-05-08T23:17:46.101869+0000","last_became_peered":"2024-05-08T23:17:46.101869+0000","last_unstale":"2024-05-08T23:19:37.978702+0000","last_undegraded":"2024-05-08T23:19:37.978702+0000","last_fullsized":"2024-05-08T23:19:37.978702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:24.070600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.966116+0000","last_change":"2024-05-08T23:17:46.110364+0000","last_active":"2024-05-08T23:19:37.966116+0000","last_peered":"2024-05-08T23:19:37.966116+0000","last_clean":"2024-05-08T23:19:37.966116+0000","last_became_active":"2024-05-08T23:17:46.108610+0000","last_became_peered":"2024-05-08T23:17:46.108610+0000","last_unstale":"2024-05-08T23:19:37.966116+0000","last_undegraded":"2024-05-08T23:19:37.966116+0000","last_fullsized":"2024-05-08T23:19:37.966116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:41.956912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.982523+0000","last_change":"2024-05-08T23:17:46.112500+0000","last_active":"2024-05-08T23:19:37.982523+0000","last_peered":"2024-05-08T23:19:37.982523+0000","last_clean":"2024-05-08T23:19:37.982523+0000","last_became_active":"2024-05-08T23:17:46.112048+0000","last_became_peered":"2024-05-08T23:17:46.112048+0000","last_unstale":"2024-05-08T23:19:37.982523+0000","last_undegraded":"2024-05-08T23:19:37.982523+0000","last_fullsized":"2024-05-08T23:19:37.982523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:17:41.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.986378+0000","last_change":"2024-05-08T23:17:46.104443+0000","last_active":"2024-05-08T23:19:37.986378+0000","last_peered":"2024-05-08T23:19:37.986378+0000","last_clean":"2024-05-08T23:19:37.986378+0000","last_became_active":"2024-05-08T23:17:46.103893+0000","last_became_peered":"2024-05-08T23:17:46.103893+0000","last_unstale":"2024-05-08T23:19:37.986378+0000","last_undegraded":"2024-05-08T23:19:37.986378+0000","last_fullsized":"2024-05-08T23:19:37.986378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:04:20.214307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T23:19:37.977581+0000","last_change":"2024-05-08T23:17:46.101912+0000","last_active":"2024-05-08T23:19:37.977581+0000","last_peered":"2024-05-08T23:19:37.977581+0000","last_clean":"2024-05-08T23:19:37.977581+0000","last_became_active":"2024-05-08T23:17:46.101506+0000","last_became_peered":"2024-05-08T23:17:46.101506+0000","last_unstale":"2024-05-08T23:19:37.977581+0000","last_undegraded":"2024-05-08T23:19:37.977581+0000","last_fullsized":"2024-05-08T23:19:37.977581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T23:17:45.592191+0000","last_clean_scrub_stamp":"2024-05-08T23:17:45.592191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:42.902090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593571,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":642,"num_read_kb":426,"num_write":278,"num_write_kb":1432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2015232,"data_stored":1780713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645344,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":950272,"data_stored":472912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.48399999999999999}]},{"osd":1,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93300000000000005,"5min":1.163,"15min":1.163},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.749,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.379}]},{"osd":2,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45000000000000001},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77400000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.47299999999999998},"min":{"1min":0.37,"5min":0.311,"15min":0.311},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85899999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81399999999999995,"5min":1.181,"15min":1.181},"last":0.433}]},{"osd":5,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.45800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98599999999999999,"5min":1.075,"15min":1.075},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":1.038,"15min":1.038},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71799999999999997,"15min":0.70999999999999996},"min":{"1min":0.59299999999999997,"5min":0.504,"15min":0.504},"max":{"1min":0.97399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.875,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56299999999999994}]},{"osd":9,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76300000000000001,"15min":0.75800000000000001},"min":{"1min":0.55700000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.75,"15min":0.73699999999999999},"min":{"1min":0.71099999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.032,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.94899999999999995,"5min":1.216,"15min":1.216},"last":0.62},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.61599999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.23,"15min":1.23},"last":0.61599999999999999}]}]},{"osd":10,"up_from":65,"seq":279172874276,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37060,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708092,"statfs":{"total":95995035648,"available":95957086208,"internally_reserved":0,"allocated":1773568,"data_stored":1255360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.374}]},{"osd":1,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.34599999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.66800000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.503,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.61499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.43099999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.626,"15min":0.63},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.871,"15min":0.871},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63300000000000001,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45800000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.62,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.746,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72999999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71099999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.495,"15min":0.495},"max":{"1min":0.77200000000000002,"5min":1.169,"15min":1.169},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.81599999999999995,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.5},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.84299999999999997,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":8,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.83699999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.75800000000000001,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.77700000000000002,"5min":1.016,"15min":1.016},"last":0.60899999999999999}]},{"osd":9,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79800000000000004,"15min":0.80800000000000005},"min":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86199999999999999,"5min":1.3,"15min":1.3},"last":0.62},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.76200000000000001,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.83699999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56000000000000005}]},{"osd":11,"last update":"Wed May 8 23:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.57999999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.87,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95299999999999996,"5min":1.083,"15min":1.083},"last":0.63700000000000001}]}]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28348,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716804,"statfs":{"total":95995035648,"available":95966007296,"internally_reserved":0,"allocated":1363968,"data_stored":870460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.29999999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.49099999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42299999999999999}]},{"osd":1,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.66500000000000004},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.1040000000000001,"5min":4.1040000000000001,"15min":4.1040000000000001},"last":0.54300000000000004}]},{"osd":2,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65600000000000003,"15min":0.66000000000000003},"min":{"1min":0.40899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0090000000000003,"5min":4.0090000000000003,"15min":4.0090000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.504},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69599999999999995}]},{"osd":5,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 8 23:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.58299999999999996}]},{"osd":8,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.81799999999999995,"15min":0.80500000000000005},"min":{"1min":0.58199999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.038,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60499999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0660000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.58299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70599999999999996}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.66400000000000003}]}]},{"osd":8,"up_from":53,"seq":227633266731,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28344,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716808,"statfs":{"total":95995035648,"available":95966011392,"internally_reserved":0,"allocated":1359872,"data_stored":866284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78000000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.40000000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83199999999999996,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.48799999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.51600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48399999999999999}]},{"osd":2,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.317},{"interface":"front","average":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.35499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":1.137,"15min":1.137},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84399999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64500000000000002,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65700000000000003,"15min":0.65200000000000002},"min":{"1min":0.503,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.5}]},{"osd":5,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78400000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85599999999999998,"5min":0.876,"15min":0.876},"last":0.46999999999999997}]},{"osd":6,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.871,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.65500000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95499999999999996,"5min":1.042,"15min":1.042},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.71199999999999997},"min":{"1min":0.57999999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94699999999999995,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.623},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94599999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.61399999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97099999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.64300000000000002,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.93600000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63300000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68300000000000005}]}]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1556480,"data_stored":1070723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.441,"15min":0.443},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.51400000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.38},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.439,"15min":0.433},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":0.63500000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76400000000000001,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54800000000000004}]},{"osd":3,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81399999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.629,"15min":0.63600000000000001},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.746,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51500000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.74399999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.36599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.66700000000000004,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.502},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.49299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":0.77800000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.627}]},{"osd":6,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68700000000000006,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.80000000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69399999999999995,"15min":0.70899999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79800000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56599999999999995}]},{"osd":8,"last update":"Wed May 8 23:19:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.59899999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.82599999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.622,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.81599999999999995,"5min":0.875,"15min":0.875},"last":0.63800000000000001}]},{"osd":9,"last update":"Wed May 8 23:19:38 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.878,"5min":0.997,"15min":0.997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.73299999999999998},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.752}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.86799999999999999,"5min":0.999,"15min":0.999},"last":0.59199999999999997}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68300000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36136,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709016,"statfs":{"total":95995035648,"available":95958032384,"internally_reserved":0,"allocated":946176,"data_stored":468324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.45600000000000002,"15min":0.44},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.39200000000000002}]},{"osd":1,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.45600000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.47799999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.41199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39900000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.40200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79700000000000004}]},{"osd":5,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.623}]},{"osd":7,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.72199999999999998},"min":{"1min":0.501,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0309999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.72199999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:23 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1699999999999999,"5min":1.177,"15min":1.177},"last":0.53800000000000003}]},{"osd":9,"last update":"Wed May 8 23:19:37 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.433}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.79700000000000004,"15min":0.79000000000000004},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81000000000000005}]},{"osd":11,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":1015808,"data_stored":530911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52100000000000002,"15min":0.495},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82499999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55100000000000005,"15min":0.51500000000000001},"min":{"1min":0.50900000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.495,"15min":0.46500000000000002},"min":{"1min":0.433,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.52700000000000002,"15min":0.48199999999999998},"min":{"1min":0.52600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.40999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.57499999999999996,"15min":0.53900000000000003},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.44700000000000001,"15min":0.40400000000000003},"min":{"1min":0.376,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.45600000000000002,"15min":0.41199999999999998},"min":{"1min":0.371,"5min":0.248,"15min":0.248},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.436}]},{"osd":5,"last update":"Wed May 8 23:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.627,"15min":0.56699999999999995},"min":{"1min":0.53300000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.63300000000000001,"15min":0.57199999999999995},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.58299999999999996,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.621,"15min":0.59299999999999997},"min":{"1min":0.54700000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.504},"min":{"1min":0.44700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77200000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.374},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.35899999999999999}]},{"osd":8,"last update":"Wed May 8 23:19:24 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.67700000000000005,"15min":0.64800000000000002},"min":{"1min":0.48799999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.57799999999999996,"5min":0.496,"15min":0.496},"max":{"1min":1.014,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.39600000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.624,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53700000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.59399999999999997}]},{"osd":11,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":970752,"data_stored":469712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 23:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58699999999999997,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.89000000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 8 23:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85399999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Wed May 8 23:19:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59699999999999998,"15min":0.55300000000000005},"min":{"1min":0.48299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.49299999999999999}]},{"osd":5,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.66000000000000003,"15min":0.624},"min":{"1min":0.56899999999999995,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.63},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66100000000000003,"15min":0.629},"min":{"1min":0.502,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.40799999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70399999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.54800000000000004,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66000000000000003}]},{"osd":8,"last update":"Wed May 8 23:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72399999999999998,"15min":0.71099999999999997},"min":{"1min":0.58399999999999996,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.996,"5min":1.014,"15min":1.014},"last":0.64700000000000002}]},{"osd":9,"last update":"Wed May 8 23:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96399999999999997,"5min":1.016,"15min":1.016},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.755}]},{"osd":10,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.93999999999999995,"5min":0.997,"15min":0.997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0740000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68500000000000005}]},{"osd":11,"last update":"Wed May 8 23:19:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0820000000000001,"5min":1.246,"15min":1.246},"last":0.78000000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":531923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.57599999999999996,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.173,"5min":6.173,"15min":6.173},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.52400000000000002,"15min":0.501},"min":{"1min":0.438,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.41699999999999998}]},{"osd":1,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.54300000000000004,"15min":0.51700000000000002},"min":{"1min":0.48599999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.55100000000000005,"15min":0.501},"min":{"1min":0.46600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.873,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.60399999999999998,"15min":0.54000000000000004},"min":{"1min":0.54500000000000004,"5min":0.375,"15min":0.375},"max":{"1min":0.78500000000000003,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed May 8 23:19:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.46999999999999997,"15min":0.41999999999999998},"min":{"1min":0.33400000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83099999999999996,"5min":0.872,"15min":0.872},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.44500000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.754,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93000000000000005,"5min":3.0659999999999998,"15min":3.0659999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.58699999999999997},"min":{"1min":0.52900000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.90100000000000002,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59199999999999997}]},{"osd":8,"last update":"Wed May 8 23:19:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.433,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.61299999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.50800000000000001}]},{"osd":9,"last update":"Wed May 8 23:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.05,"5min":0.82799999999999996,"15min":0.79100000000000004},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":6.8899999999999997,"5min":6.8899999999999997,"15min":6.8899999999999997},"last":0.65400000000000003}]},{"osd":10,"last update":"Wed May 8 23:18:53 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.55400000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.625},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.67700000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63600000000000001}]},{"osd":11,"last update":"Wed May 8 23:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64500000000000002}]}]},{"osd":3,"up_from":23,"seq":98784247868,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36660,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708492,"statfs":{"total":95995035648,"available":95957495808,"internally_reserved":0,"allocated":1486848,"data_stored":993334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.436,"15min":0.36799999999999999},"min":{"1min":0.32600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.83199999999999996,"5min":6.3040000000000003,"15min":6.3040000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.441,"15min":0.38600000000000001},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":5.5049999999999999,"15min":5.5049999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.59399999999999997},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.98499999999999999,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed May 8 23:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.57499999999999996},"min":{"1min":0.51400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.54500000000000004},"min":{"1min":0.44500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.105,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed May 8 23:19:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.627,"15min":0.56999999999999995},"min":{"1min":0.46100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0469999999999999,"5min":5.9630000000000001,"15min":5.9630000000000001},"last":0.497},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.621},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":5.5350000000000001,"15min":5.5350000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed May 8 23:19:33 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64100000000000001,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96099999999999997,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.63},"min":{"1min":0.53400000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.997,"5min":5.3769999999999998,"15min":5.3769999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Wed May 8 23:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.624,"15min":0.58799999999999997},"min":{"1min":0.39800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91900000000000004,"5min":6.0869999999999997,"15min":6.0869999999999997},"last":0.622},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.36799999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.81100000000000005,"5min":5.9189999999999996,"15min":5.9189999999999996},"last":0.44700000000000001}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.72299999999999998,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.94499999999999995,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.58199999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1879999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.73099999999999998}]},{"osd":8,"last update":"Wed May 8 23:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.70099999999999996}]},{"osd":9,"last update":"Wed May 8 23:19:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.79700000000000004,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95999999999999996,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71899999999999997,"15min":0.71199999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0920000000000001,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Wed May 8 23:18:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.92200000000000004,"15min":0.94799999999999995},"min":{"1min":0.65300000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.135,"5min":6.0300000000000002,"15min":6.0300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.97899999999999998,"15min":1.0049999999999999},"min":{"1min":0.67800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.171,"5min":5.5659999999999998,"15min":5.5659999999999998},"last":0.74299999999999999}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65400000000000003}]}]},{"osd":4,"up_from":30,"seq":128849018936,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":473422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.33600000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.502,"15min":0.49199999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61599999999999999}]},{"osd":1,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.371,"15min":0.34100000000000003},"min":{"1min":0.32900000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.55500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.39500000000000002,"15min":0.36699999999999999},"min":{"1min":0.33600000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.58299999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56000000000000005,"15min":0.52500000000000002},"min":{"1min":0.48799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.44500000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67200000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed May 8 23:19:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.53300000000000003},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84399999999999997,"5min":0.874,"15min":0.874},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.44700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38}]},{"osd":5,"last update":"Wed May 8 23:19:35 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72599999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79900000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.42599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.50700000000000001}]},{"osd":7,"last update":"Wed May 8 23:19:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52000000000000002,"15min":0.51000000000000001},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72299999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.28999999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69599999999999995,"15min":0.68600000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65000000000000002,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80300000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.498}]},{"osd":9,"last update":"Wed May 8 23:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.55300000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.86299999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.755,"15min":0.754},"min":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60099999999999998}]},{"osd":10,"last update":"Wed May 8 23:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89700000000000002,"5min":1.002,"15min":1.002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66800000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81100000000000005,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41099999999999998}]},{"osd":11,"last update":"Wed May 8 23:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55800000000000005}]}]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36872,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708280,"statfs":{"total":95995035648,"available":95957278720,"internally_reserved":0,"allocated":1703936,"data_stored":1197008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54100000000000004,"15min":0.55700000000000005},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.748,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.372},{"interface":"front","average":{"1min":0.496,"5min":0.53300000000000003,"15min":0.56499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39800000000000002}]},{"osd":1,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.31,"15min":0.31},"max":{"1min":0.69299999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41999999999999998}]},{"osd":2,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.29999999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.434},"min":{"1min":0.27900000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.27500000000000002}]},{"osd":3,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54800000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71999999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38600000000000001}]},{"osd":4,"last update":"Wed May 8 23:19:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.60699999999999998},"min":{"1min":0.42799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76000000000000001,"5min":0.871,"15min":0.871},"last":0.35599999999999998}]},{"osd":6,"last update":"Wed May 8 23:18:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.443},{"interface":"front","average":{"1min":0.622,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.40600000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.433}]},{"osd":7,"last update":"Wed May 8 23:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65500000000000003,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79500000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.41399999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.40999999999999998}]},{"osd":8,"last update":"Wed May 8 23:19:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60699999999999998,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.374,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47899999999999998}]},{"osd":9,"last update":"Wed May 8 23:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.55800000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.91200000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55400000000000005}]},{"osd":10,"last update":"Wed May 8 23:18:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71599999999999997},"min":{"1min":0.61699999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.56200000000000006,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":197089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T23:19:45.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 -- 172.21.15.16:0/84369975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb40fff80 msgr2=0x7f1bb4100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 --2- 172.21.15.16:0/84369975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb40fff80 0x7f1bb4100360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009990 tx=0x7f1bb0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 -- 172.21.15.16:0/84369975 shutdown_connections 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 --2- 172.21.15.16:0/84369975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb41050a0 0x7f1bb410cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 --2- 172.21.15.16:0/84369975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb40fff80 0x7f1bb4100360 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 --2- 172.21.15.16:0/84369975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bb4100930 0x7f1bb4104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 -- 172.21.15.16:0/84369975 >> 172.21.15.16:0/84369975 conn(0x7f1bb40fb840 msgr2=0x7f1bb40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 -- 172.21.15.16:0/84369975 shutdown_connections 2024-05-08T23:19:45.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.922+0000 7f1bbb325700 1 -- 172.21.15.16:0/84369975 wait complete. 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 Processor -- start 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 -- start start 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb40fff80 0x7f1bb4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bb41050a0 0x7f1bb41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 0x7f1bb41a3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a3820 con 0x7f1bb4198b10 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a3990 con 0x7f1bb40fff80 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bbb325700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a3b50 con 0x7f1bb41050a0 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bb98c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 0x7f1bb41a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bb98c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 0x7f1bb41a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60430/0 (socket says 172.21.15.16:60430) 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bb98c2700 1 -- 172.21.15.16:0/3360784826 learned_addr learned my addr 172.21.15.16:0/3360784826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bb90c1700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb40fff80 0x7f1bb4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.923+0000 7f1bb88c0700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bb41050a0 0x7f1bb41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bb98c2700 1 -- 172.21.15.16:0/3360784826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bb41050a0 msgr2=0x7f1bb41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bb98c2700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bb41050a0 0x7f1bb41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bb98c2700 1 -- 172.21.15.16:0/3360784826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb40fff80 msgr2=0x7f1bb4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bb98c2700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb40fff80 0x7f1bb4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bb98c2700 1 -- 172.21.15.16:0/3360784826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00095f0 con 0x7f1bb4198b10 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bb98c2700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 0x7f1bb41a3110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1bac007ee0 tx=0x7f1bac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac017070 con 0x7f1bb4198b10 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bac00f040 con 0x7f1bb4198b10 2024-05-08T23:19:45.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb41a3e30 con 0x7f1bb4198b10 2024-05-08T23:19:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac005710 con 0x7f1bb4198b10 2024-05-08T23:19:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.924+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb41a4310 con 0x7f1bb4198b10 2024-05-08T23:19:45.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.925+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb4108310 con 0x7f1bb4198b10 2024-05-08T23:19:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.926+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f1bac005870 con 0x7f1bb4198b10 2024-05-08T23:19:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.927+0000 7f1baa7fc700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ba00a13a0 0x7f1ba00a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.927+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f1bac0ccea0 con 0x7f1bb4198b10 2024-05-08T23:19:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.927+0000 7f1bb90c1700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ba00a13a0 0x7f1ba00a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.927+0000 7f1bb90c1700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ba00a13a0 0x7f1ba00a3860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006010 tx=0x7f1bb000b280 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:45.930+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1bac095e70 con 0x7f1bb4198b10 2024-05-08T23:19:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:45 smithi016 ceph-mon[42541]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:46.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.158+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bb4069780 con 0x7f1bb4198b10 2024-05-08T23:19:46.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.159+0000 7f1baa7fc700 1 -- 172.21.15.16:0/3360784826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24906 (secure 0 0 0) 0x7f1bac099490 con 0x7f1bb4198b10 2024-05-08T23:19:46.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:19:37.917761+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ba00a13a0 msgr2=0x7f1ba00a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ba00a13a0 0x7f1ba00a3860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006010 tx=0x7f1bb000b280 comp rx=0 tx=0).stop 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 msgr2=0x7f1bb41a3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 0x7f1bb41a3110 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1bac007ee0 tx=0x7f1bac00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 shutdown_connections 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ba00a13a0 0x7f1ba00a3860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4198b10 0x7f1bb41a3110 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1bb40fff80 0x7f1bb4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 --2- 172.21.15.16:0/3360784826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bb41050a0 0x7f1bb41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 >> 172.21.15.16:0/3360784826 conn(0x7f1bb40fb840 msgr2=0x7f1bb40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 shutdown_connections 2024-05-08T23:19:46.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.162+0000 7f1bbb325700 1 -- 172.21.15.16:0/3360784826 wait complete. 2024-05-08T23:19:46.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:19:46.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:19:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:45 smithi155 ceph-mon[47105]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 -- 172.21.15.16:0/823855308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8069dd0 msgr2=0x7f8bc8105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 --2- 172.21.15.16:0/823855308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8069dd0 0x7f8bc8105b70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009990 tx=0x7f8bb0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 -- 172.21.15.16:0/823855308 shutdown_connections 2024-05-08T23:19:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 --2- 172.21.15.16:0/823855308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc81062a0 0x7f8bc810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 --2- 172.21.15.16:0/823855308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc8069420 0x7f8bc8069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 --2- 172.21.15.16:0/823855308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8069dd0 0x7f8bc8105b70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 -- 172.21.15.16:0/823855308 >> 172.21.15.16:0/823855308 conn(0x7f8bc8076ba0 msgr2=0x7f8bc8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 -- 172.21.15.16:0/823855308 shutdown_connections 2024-05-08T23:19:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.317+0000 7f8bce01c700 1 -- 172.21.15.16:0/823855308 wait complete. 2024-05-08T23:19:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 Processor -- start 2024-05-08T23:19:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 -- start start 2024-05-08T23:19:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc8069420 0x7f8bc81001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc81062a0 0x7f8bc8100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 0x7f8bc81a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a5ab0 con 0x7f8bc8100c40 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a5c20 con 0x7f8bc81062a0 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bce01c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a5de0 con 0x7f8bc8069420 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bc7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 0x7f8bc81a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bc7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 0x7f8bc81a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60446/0 (socket says 172.21.15.16:60446) 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bc7fff700 1 -- 172.21.15.16:0/1953772898 learned_addr learned my addr 172.21.15.16:0/1953772898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bc6ffd700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc81062a0 0x7f8bc8100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.318+0000 7f8bc77fe700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc8069420 0x7f8bc81001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc7fff700 1 -- 172.21.15.16:0/1953772898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc8069420 msgr2=0x7f8bc81001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc7fff700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc8069420 0x7f8bc81001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc7fff700 1 -- 172.21.15.16:0/1953772898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc81062a0 msgr2=0x7f8bc8100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc7fff700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc81062a0 0x7f8bc8100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc7fff700 1 -- 172.21.15.16:0/1953772898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00095f0 con 0x7f8bc8100c40 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc7fff700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 0x7f8bc81a54e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000ef30 tx=0x7f8bc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0018070 con 0x7f8bc8100c40 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc81a6060 con 0x7f8bc8100c40 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bc000f040 con 0x7f8bc8100c40 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000cbc0 con 0x7f8bc8100c40 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.319+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81a64f0 con 0x7f8bc8100c40 2024-05-08T23:19:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.320+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc8109570 con 0x7f8bc8100c40 2024-05-08T23:19:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.322+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f8bc000cd80 con 0x7f8bc8100c40 2024-05-08T23:19:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.322+0000 7f8bc4ff9700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8bb40a1320 0x7f8bb40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.323+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f8bc00cd220 con 0x7f8bc8100c40 2024-05-08T23:19:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.323+0000 7f8bc77fe700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8bb40a1320 0x7f8bb40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.323+0000 7f8bc77fe700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8bb40a1320 0x7f8bb40a37e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.326+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8bc0096280 con 0x7f8bc8100c40 2024-05-08T23:19:46.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:45 smithi189 ceph-mon[46919]: from='client.15606 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.544+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bc8067490 con 0x7f8bc8100c40 2024-05-08T23:19:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.545+0000 7f8bc4ff9700 1 -- 172.21.15.16:0/1953772898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bc00998a0 con 0x7f8bc8100c40 2024-05-08T23:19:46.546 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:19:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8bb40a1320 msgr2=0x7f8bb40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8bb40a1320 0x7f8bb40a37e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb800b410 comp rx=0 tx=0).stop 2024-05-08T23:19:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 msgr2=0x7f8bc81a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 0x7f8bc81a54e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000ef30 tx=0x7f8bc000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 shutdown_connections 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8bb40a1320 0x7f8bb40a37e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc8100c40 0x7f8bc81a54e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc8069420 0x7f8bc81001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 --2- 172.21.15.16:0/1953772898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8bc81062a0 0x7f8bc8100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 >> 172.21.15.16:0/1953772898 conn(0x7f8bc8076ba0 msgr2=0x7f8bc8107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 shutdown_connections 2024-05-08T23:19:46.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.547+0000 7f8bce01c700 1 -- 172.21.15.16:0/1953772898 wait complete. 2024-05-08T23:19:46.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:19:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 -- 172.21.15.16:0/4107299092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 msgr2=0x7ff79010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 --2- 172.21.15.16:0/4107299092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff79010cce0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b0a0 tx=0x7ff78800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 -- 172.21.15.16:0/4107299092 shutdown_connections 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 --2- 172.21.15.16:0/4107299092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff79010cce0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 --2- 172.21.15.16:0/4107299092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff790100060 0x7ff790100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 --2- 172.21.15.16:0/4107299092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff790100a10 0x7ff790104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 -- 172.21.15.16:0/4107299092 >> 172.21.15.16:0/4107299092 conn(0x7ff790076f40 msgr2=0x7ff790077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 -- 172.21.15.16:0/4107299092 shutdown_connections 2024-05-08T23:19:46.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.703+0000 7ff7950d3700 1 -- 172.21.15.16:0/4107299092 wait complete. 2024-05-08T23:19:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 Processor -- start 2024-05-08T23:19:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 -- start start 2024-05-08T23:19:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff790100060 0x7ff7901982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff790198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff790198d40 0x7ff7901a3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901a3a20 con 0x7ff790105160 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901a3b90 con 0x7ff790100060 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff7950d3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901a3d50 con 0x7ff790198d40 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff78e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff790198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff78e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff790198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44738/0 (socket says 172.21.15.16:44738) 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff78e59c700 1 -- 172.21.15.16:0/2733894604 learned_addr learned my addr 172.21.15.16:0/2733894604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff78f59e700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff790198d40 0x7ff7901a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.704+0000 7ff78ed9d700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff790100060 0x7ff7901982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff78e59c700 1 -- 172.21.15.16:0/2733894604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff790198d40 msgr2=0x7ff7901a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff78e59c700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff790198d40 0x7ff7901a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff78e59c700 1 -- 172.21.15.16:0/2733894604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff790100060 msgr2=0x7ff7901982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff78e59c700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff790100060 0x7ff7901982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff78e59c700 1 -- 172.21.15.16:0/2733894604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788009d00 con 0x7ff790105160 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff78e59c700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff790198800 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff78400d980 tx=0x7ff78400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400f890 con 0x7ff790105160 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff78400fe80 con 0x7ff790105160 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7901a3fd0 con 0x7ff790105160 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400e3b0 con 0x7ff790105160 2024-05-08T23:19:46.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.705+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7901a44b0 con 0x7ff790105160 2024-05-08T23:19:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.706+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790108430 con 0x7ff790105160 2024-05-08T23:19:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.708+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7ff78400f9f0 con 0x7ff790105160 2024-05-08T23:19:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.708+0000 7ff777fff700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff7780a1320 0x7ff7780a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.708+0000 7ff78ed9d700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff7780a1320 0x7ff7780a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.708+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7ff7840ccb70 con 0x7ff790105160 2024-05-08T23:19:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.709+0000 7ff78ed9d700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff7780a1320 0x7ff7780a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff780007be0 tx=0x7ff7800073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:46.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.713+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff784095b40 con 0x7ff790105160 2024-05-08T23:19:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.930+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7900685b0 con 0x7ff790105160 2024-05-08T23:19:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.930+0000 7ff777fff700 1 -- 172.21.15.16:0/2733894604 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff784099160 con 0x7ff790105160 2024-05-08T23:19:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff7780a1320 msgr2=0x7ff7780a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff7780a1320 0x7ff7780a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff780007be0 tx=0x7ff7800073d0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 msgr2=0x7ff790198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff790198800 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff78400d980 tx=0x7ff78400dc90 comp rx=0 tx=0).stop 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 shutdown_connections 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff7780a1320 0x7ff7780a37e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff790198d40 0x7ff7901a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff790100060 0x7ff7901982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 --2- 172.21.15.16:0/2733894604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff790105160 0x7ff790198800 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:46.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 >> 172.21.15.16:0/2733894604 conn(0x7ff790076f40 msgr2=0x7ff7900fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:46.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 shutdown_connections 2024-05-08T23:19:46.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:46.933+0000 7ff7950d3700 1 -- 172.21.15.16:0/2733894604 wait complete. 2024-05-08T23:19:46.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 -- 172.21.15.16:0/135646387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df41062a0 msgr2=0x7f6df410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 --2- 172.21.15.16:0/135646387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df41062a0 0x7f6df410de20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00b0a0 tx=0x7f6dec00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 -- 172.21.15.16:0/135646387 shutdown_connections 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 --2- 172.21.15.16:0/135646387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df41062a0 0x7f6df410de20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 --2- 172.21.15.16:0/135646387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6df4101180 0x7f6df4101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 --2- 172.21.15.16:0/135646387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4101b30 0x7f6df4105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 -- 172.21.15.16:0/135646387 >> 172.21.15.16:0/135646387 conn(0x7f6df40fc9e0 msgr2=0x7f6df40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 -- 172.21.15.16:0/135646387 shutdown_connections 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.088+0000 7f6dfb260700 1 -- 172.21.15.16:0/135646387 wait complete. 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 Processor -- start 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 -- start start 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4101180 0x7f6df40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6df41062a0 0x7f6df40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 0x7f6df41a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df41a5d10 con 0x7f6df4073d20 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df41a5e80 con 0x7f6df41062a0 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6dfb260700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df41a6040 con 0x7f6df4101180 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6df97fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 0x7f6df41a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6df97fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 0x7f6df41a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44762/0 (socket says 172.21.15.16:44762) 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6df97fd700 1 -- 172.21.15.16:0/3868471698 learned_addr learned my addr 172.21.15.16:0/3868471698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6df8ffc700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4101180 0x7f6df40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.089+0000 7f6df3fff700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6df41062a0 0x7f6df40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df97fd700 1 -- 172.21.15.16:0/3868471698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4101180 msgr2=0x7f6df40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df97fd700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4101180 0x7f6df40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df97fd700 1 -- 172.21.15.16:0/3868471698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6df41062a0 msgr2=0x7f6df40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df97fd700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6df41062a0 0x7f6df40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df97fd700 1 -- 172.21.15.16:0/3868471698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec009d00 con 0x7f6df4073d20 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df97fd700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 0x7f6df41a56d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6dec004de0 tx=0x7f6dec004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00e040 con 0x7f6df4073d20 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6dec01b450 con 0x7f6df4073d20 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df41a62c0 con 0x7f6df4073d20 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0052a0 con 0x7f6df4073d20 2024-05-08T23:19:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.090+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df41a6790 con 0x7f6df4073d20 2024-05-08T23:19:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.092+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df4109570 con 0x7f6df4073d20 2024-05-08T23:19:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.093+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f6dec01b5c0 con 0x7f6df4073d20 2024-05-08T23:19:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.093+0000 7f6df1ffb700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6de00a13a0 0x7f6de00a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.093+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f6dec0ce380 con 0x7f6df4073d20 2024-05-08T23:19:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.093+0000 7f6df8ffc700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6de00a13a0 0x7f6de00a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.094+0000 7f6df8ffc700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6de00a13a0 0x7f6de00a3860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009c60 tx=0x7f6de4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.098+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6dec0973e0 con 0x7f6df4073d20 2024-05-08T23:19:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:46 smithi155 ceph-mon[47105]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:46 smithi155 ceph-mon[47105]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:46 smithi155 ceph-mon[47105]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 KiB/s wr, 22 op/s 2024-05-08T23:19:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3360784826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1953772898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:47.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2733894604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.312+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6df4074780 con 0x7f6df4073d20 2024-05-08T23:19:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.313+0000 7f6df1ffb700 1 -- 172.21.15.16:0/3868471698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dec09aa00 con 0x7f6df4073d20 2024-05-08T23:19:47.313 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:19:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6de00a13a0 msgr2=0x7f6de00a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6de00a13a0 0x7f6de00a3860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009c60 tx=0x7f6de4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 msgr2=0x7f6df41a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 0x7f6df41a56d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6dec004de0 tx=0x7f6dec004e10 comp rx=0 tx=0).stop 2024-05-08T23:19:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 shutdown_connections 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6de00a13a0 0x7f6de00a3860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df4073d20 0x7f6df41a56d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df4101180 0x7f6df40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 --2- 172.21.15.16:0/3868471698 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6df41062a0 0x7f6df40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 >> 172.21.15.16:0/3868471698 conn(0x7f6df40fc9e0 msgr2=0x7f6df40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 shutdown_connections 2024-05-08T23:19:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.315+0000 7f6dfb260700 1 -- 172.21.15.16:0/3868471698 wait complete. 2024-05-08T23:19:47.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:46 smithi016 ceph-mon[42541]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:47.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:46 smithi016 ceph-mon[42541]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:47.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:46 smithi016 ceph-mon[42541]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 KiB/s wr, 22 op/s 2024-05-08T23:19:47.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3360784826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:47.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1953772898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:47.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2733894604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:47.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:19:47.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:46 smithi189 ceph-mon[46919]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:46 smithi189 ceph-mon[46919]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T23:19:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:46 smithi189 ceph-mon[46919]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 KiB/s wr, 22 op/s 2024-05-08T23:19:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3360784826' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1953772898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:19:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2733894604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:19:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 -- 172.21.15.16:0/2893623386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4584105170 msgr2=0x7f458410ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 --2- 172.21.15.16:0/2893623386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4584105170 0x7f458410ccf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b0a0 tx=0x7f457c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 -- 172.21.15.16:0/2893623386 shutdown_connections 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 --2- 172.21.15.16:0/2893623386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4584105170 0x7f458410ccf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 --2- 172.21.15.16:0/2893623386 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584100050 0x7f4584100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 --2- 172.21.15.16:0/2893623386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4584100a00 0x7f4584104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 -- 172.21.15.16:0/2893623386 >> 172.21.15.16:0/2893623386 conn(0x7f4584076fa0 msgr2=0x7f45840773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 -- 172.21.15.16:0/2893623386 shutdown_connections 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.470+0000 7f458ab61700 1 -- 172.21.15.16:0/2893623386 wait complete. 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 Processor -- start 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 -- start start 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4584100050 0x7f4584199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584105170 0x7f4584199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 0x7f458419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458419e980 con 0x7f458419df00 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458419eaf0 con 0x7f4584105170 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f458ab61700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458419ecb0 con 0x7f4584100050 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f45890fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 0x7f458419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f45890fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 0x7f458419e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44778/0 (socket says 172.21.15.16:44778) 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f45890fe700 1 -- 172.21.15.16:0/2863797825 learned_addr learned my addr 172.21.15.16:0/2863797825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f4583fff700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584105170 0x7f4584199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.471+0000 7f45888fd700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4584100050 0x7f4584199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f45890fe700 1 -- 172.21.15.16:0/2863797825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4584100050 msgr2=0x7f4584199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f45890fe700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4584100050 0x7f4584199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f45890fe700 1 -- 172.21.15.16:0/2863797825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584105170 msgr2=0x7f4584199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f45890fe700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584105170 0x7f4584199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f45890fe700 1 -- 172.21.15.16:0/2863797825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c009d00 con 0x7f458419df00 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f4583fff700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584105170 0x7f4584199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f45890fe700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 0x7f458419e380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f457c006130 tx=0x7f457c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c00e040 con 0x7f458419df00 2024-05-08T23:19:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f457c01b450 con 0x7f458419df00 2024-05-08T23:19:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45841a61b0 con 0x7f458419df00 2024-05-08T23:19:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0052a0 con 0x7f458419df00 2024-05-08T23:19:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.472+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45841a6680 con 0x7f458419df00 2024-05-08T23:19:47.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.473+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4584101520 con 0x7f458419df00 2024-05-08T23:19:47.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.475+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f457c01b5c0 con 0x7f458419df00 2024-05-08T23:19:47.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.476+0000 7f4581ffb700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f456c0a13a0 0x7f456c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.476+0000 7f45888fd700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f456c0a13a0 0x7f456c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.476+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f457c0ceb50 con 0x7f458419df00 2024-05-08T23:19:47.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.477+0000 7f45888fd700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f456c0a13a0 0x7f456c0a3860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4574009c60 tx=0x7f4574009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:47.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.480+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f457c097b50 con 0x7f458419df00 2024-05-08T23:19:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.698+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f458419a9c0 con 0x7f458419df00 2024-05-08T23:19:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.699+0000 7f4581ffb700 1 -- 172.21.15.16:0/2863797825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7f457c09b170 con 0x7f458419df00 2024-05-08T23:19:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f456c0a13a0 msgr2=0x7f456c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f456c0a13a0 0x7f456c0a3860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4574009c60 tx=0x7f4574009260 comp rx=0 tx=0).stop 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 msgr2=0x7f458419e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 0x7f458419e380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f457c006130 tx=0x7f457c009500 comp rx=0 tx=0).stop 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 shutdown_connections 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f456c0a13a0 0x7f456c0a3860 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f458419df00 0x7f458419e380 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4584100050 0x7f4584199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 --2- 172.21.15.16:0/2863797825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4584105170 0x7f4584199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 >> 172.21.15.16:0/2863797825 conn(0x7f4584076fa0 msgr2=0x7f45840fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 shutdown_connections 2024-05-08T23:19:47.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.701+0000 7f458ab61700 1 -- 172.21.15.16:0/2863797825 wait complete. 2024-05-08T23:19:47.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 -- 172.21.15.16:0/298612607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0101060 msgr2=0x7f98e0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 --2- 172.21.15.16:0/298612607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0101060 0x7f98e0101440 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009990 tx=0x7f98d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 -- 172.21.15.16:0/298612607 shutdown_connections 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 --2- 172.21.15.16:0/298612607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0106180 0x7f98e010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 --2- 172.21.15.16:0/298612607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0101060 0x7f98e0101440 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 --2- 172.21.15.16:0/298612607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98e0101a10 0x7f98e0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 -- 172.21.15.16:0/298612607 >> 172.21.15.16:0/298612607 conn(0x7f98e0078360 msgr2=0x7f98e0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:47.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 -- 172.21.15.16:0/298612607 shutdown_connections 2024-05-08T23:19:47.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.852+0000 7f98e4a64700 1 -- 172.21.15.16:0/298612607 wait complete. 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 Processor -- start 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 -- start start 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0101060 0x7f98e01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 0x7f98e01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98e019ba80 0x7f98e019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e019c500 con 0x7f98e0106180 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e019c670 con 0x7f98e0101060 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98e4a64700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e019c830 con 0x7f98e019ba80 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98ddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 0x7f98e01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98ddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 0x7f98e01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44792/0 (socket says 172.21.15.16:44792) 2024-05-08T23:19:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98ddd9b700 1 -- 172.21.15.16:0/3891402057 learned_addr learned my addr 172.21.15.16:0/3891402057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98de59c700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0101060 0x7f98e01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.853+0000 7f98ded9d700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98e019ba80 0x7f98e019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98ddd9b700 1 -- 172.21.15.16:0/3891402057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98e019ba80 msgr2=0x7f98e019bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98ddd9b700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98e019ba80 0x7f98e019bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98ddd9b700 1 -- 172.21.15.16:0/3891402057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0101060 msgr2=0x7f98e01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98ddd9b700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0101060 0x7f98e01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98ddd9b700 1 -- 172.21.15.16:0/3891402057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00095f0 con 0x7f98e0106180 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98de59c700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0101060 0x7f98e01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98ddd9b700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 0x7f98e01976e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98d400d950 tx=0x7f98d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4004220 con 0x7f98e0106180 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e01a3d90 con 0x7f98e0106180 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98d4007620 con 0x7f98e0106180 2024-05-08T23:19:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4005530 con 0x7f98e0106180 2024-05-08T23:19:47.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.854+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e01a4210 con 0x7f98e0106180 2024-05-08T23:19:47.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.855+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e01093f0 con 0x7f98e0106180 2024-05-08T23:19:47.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.856+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f98d4004750 con 0x7f98e0106180 2024-05-08T23:19:47.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.857+0000 7f98cf7fe700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f98c80a1320 0x7f98c80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:47.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.857+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f98d40cd460 con 0x7f98e0106180 2024-05-08T23:19:47.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.857+0000 7f98de59c700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f98c80a1320 0x7f98c80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:47.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.858+0000 7f98de59c700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f98c80a1320 0x7f98c80a37e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98d0006010 tx=0x7f98d000b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:47.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:47.861+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f98d4096430 con 0x7f98e0106180 2024-05-08T23:19:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3868471698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2863797825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:48.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.138+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e0102530 con 0x7f98e0106180 2024-05-08T23:19:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.139+0000 7f98cf7fe700 1 -- 172.21.15.16:0/3891402057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f98d4016020 con 0x7f98e0106180 2024-05-08T23:19:48.139 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:48.139 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f98c80a1320 msgr2=0x7f98c80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f98c80a1320 0x7f98c80a37e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f98d0006010 tx=0x7f98d000b540 comp rx=0 tx=0).stop 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 msgr2=0x7f98e01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 0x7f98e01976e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f98d400d950 tx=0x7f98d400dc60 comp rx=0 tx=0).stop 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 shutdown_connections 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f98c80a1320 0x7f98c80a37e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98e019ba80 0x7f98e019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98e0101060 0x7f98e01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 --2- 172.21.15.16:0/3891402057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e0106180 0x7f98e01976e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 >> 172.21.15.16:0/3891402057 conn(0x7f98e0078360 msgr2=0x7f98e00ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 shutdown_connections 2024-05-08T23:19:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.141+0000 7f98e4a64700 1 -- 172.21.15.16:0/3891402057 wait complete. 2024-05-08T23:19:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:19:48.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:19:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:47 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3868471698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:47 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2863797825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 -- 172.21.15.16:0/3805413152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf41008f0 msgr2=0x7f7bf4104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/3805413152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf41008f0 0x7f7bf4104930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009990 tx=0x7f7be4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 -- 172.21.15.16:0/3805413152 shutdown_connections 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/3805413152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf4105060 0x7f7bf410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/3805413152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf40fff40 0x7f7bf4100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/3805413152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf41008f0 0x7f7bf4104930 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 -- 172.21.15.16:0/3805413152 >> 172.21.15.16:0/3805413152 conn(0x7f7bf40fb7e0 msgr2=0x7f7bf40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 -- 172.21.15.16:0/3805413152 shutdown_connections 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.292+0000 7f7bfc55a700 1 -- 172.21.15.16:0/3805413152 wait complete. 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 Processor -- start 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 -- start start 2024-05-08T23:19:48.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf40fff40 0x7f7bf41019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 0x7f7bf4101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf4102450 0x7f7bf41a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41a5970 con 0x7f7bf4105060 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41a5ae0 con 0x7f7bf40fff40 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfc55a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41a5ca0 con 0x7f7bf4102450 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bf9af5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 0x7f7bf4101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bf9af5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 0x7f7bf4101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44810/0 (socket says 172.21.15.16:44810) 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bf9af5700 1 -- 172.21.15.16:0/1674512550 learned_addr learned my addr 172.21.15.16:0/1674512550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfaaf7700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf4102450 0x7f7bf41a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.293+0000 7f7bfa2f6700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf40fff40 0x7f7bf41019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bf9af5700 1 -- 172.21.15.16:0/1674512550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf4102450 msgr2=0x7f7bf41a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bf9af5700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf4102450 0x7f7bf41a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bf9af5700 1 -- 172.21.15.16:0/1674512550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf40fff40 msgr2=0x7f7bf41019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bf9af5700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf40fff40 0x7f7bf41019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bf9af5700 1 -- 172.21.15.16:0/1674512550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40095f0 con 0x7f7bf4105060 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bf9af5700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 0x7f7bf4101f10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7be40036c0 tx=0x7f7be4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be401d070 con 0x7f7bf4105060 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7be40059c0 con 0x7f7bf4105060 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf41a5f20 con 0x7f7bf4105060 2024-05-08T23:19:48.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4022a50 con 0x7f7bf4105060 2024-05-08T23:19:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.294+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf41a6340 con 0x7f7bf4105060 2024-05-08T23:19:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.295+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf4109340 con 0x7f7bf4105060 2024-05-08T23:19:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.296+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f7be400f670 con 0x7f7bf4105060 2024-05-08T23:19:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.297+0000 7f7beb7fe700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7be00a13a0 0x7f7be00a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.297+0000 7f7bfa2f6700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7be00a13a0 0x7f7be00a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.297+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f7be40ce300 con 0x7f7bf4105060 2024-05-08T23:19:48.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.298+0000 7f7bfa2f6700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7be00a13a0 0x7f7be00a3860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005950 tx=0x7f7bf000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.302+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7be4097300 con 0x7f7bf4105060 2024-05-08T23:19:48.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:47 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3868471698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:19:48.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:47 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2863797825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:19:48.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.524+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf4101410 con 0x7f7bf4105060 2024-05-08T23:19:48.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.525+0000 7f7beb7fe700 1 -- 172.21.15.16:0/1674512550 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+24906 (secure 0 0 0) 0x7f7be409a920 con 0x7f7bf4105060 2024-05-08T23:19:48.527 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:19:48.527 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:19:37.917761+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:19:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7be00a13a0 msgr2=0x7f7be00a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7be00a13a0 0x7f7be00a3860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005950 tx=0x7f7bf000b410 comp rx=0 tx=0).stop 2024-05-08T23:19:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 msgr2=0x7f7bf4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 0x7f7bf4101f10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7be40036c0 tx=0x7f7be4009580 comp rx=0 tx=0).stop 2024-05-08T23:19:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 shutdown_connections 2024-05-08T23:19:48.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7be00a13a0 0x7f7be00a3860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf4102450 0x7f7bf41a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bf40fff40 0x7f7bf41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 --2- 172.21.15.16:0/1674512550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bf4105060 0x7f7bf4101f10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 >> 172.21.15.16:0/1674512550 conn(0x7f7bf40fb7e0 msgr2=0x7f7bf4068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 shutdown_connections 2024-05-08T23:19:48.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.528+0000 7f7bfc55a700 1 -- 172.21.15.16:0/1674512550 wait complete. 2024-05-08T23:19:48.544 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T23:19:48.544 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T23:19:48.555 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:19:48.555 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:19:48.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-08T23:19:48.555 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T23:19:48.555 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:19:48.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:19:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 -- 172.21.15.16:0/1574958475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114069440 msgr2=0x7fe114069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 --2- 172.21.15.16:0/1574958475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114069440 0x7fe114069820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe1100099c0 tx=0x7fe110009cd0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 -- 172.21.15.16:0/1720483040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f4100a10 msgr2=0x7f72f4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 --2- 172.21.15.16:0/1720483040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f4100a10 0x7f72f4104a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72e40098f0 tx=0x7f72e4009c00 comp rx=0 tx=0).stop 2024-05-08T23:19:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 -- 172.21.15.16:0/1574958475 shutdown_connections 2024-05-08T23:19:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 --2- 172.21.15.16:0/1574958475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114106250 0x7fe11410ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 --2- 172.21.15.16:0/1574958475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114069440 0x7fe114069820 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 --2- 172.21.15.16:0/1574958475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe114069df0 0x7fe114105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 -- 172.21.15.16:0/1574958475 >> 172.21.15.16:0/1574958475 conn(0x7fe114076f40 msgr2=0x7fe114077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 -- 172.21.15.16:0/1720483040 shutdown_connections 2024-05-08T23:19:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 --2- 172.21.15.16:0/1720483040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4105160 0x7f72f410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 -- 172.21.15.16:0/1574958475 shutdown_connections 2024-05-08T23:19:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 --2- 172.21.15.16:0/1720483040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4100060 0x7f72f4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 --2- 172.21.15.16:0/1720483040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f4100a10 0x7f72f4104a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 -- 172.21.15.16:0/1720483040 >> 172.21.15.16:0/1720483040 conn(0x7f72f4076f40 msgr2=0x7f72f4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7fe11b662700 1 -- 172.21.15.16:0/1574958475 wait complete. 2024-05-08T23:19:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 -- 172.21.15.16:0/1720483040 shutdown_connections 2024-05-08T23:19:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.696+0000 7f72fc146700 1 -- 172.21.15.16:0/1720483040 wait complete. 2024-05-08T23:19:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 Processor -- start 2024-05-08T23:19:48.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 -- start start 2024-05-08T23:19:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 Processor -- start 2024-05-08T23:19:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 -- start start 2024-05-08T23:19:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114069440 0x7fe1141994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 0x7fe1141999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe11419dd90 0x7fe11419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11419e810 con 0x7fe114106250 2024-05-08T23:19:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11419e980 con 0x7fe11419dd90 2024-05-08T23:19:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe11b662700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11419eb40 con 0x7fe114069440 2024-05-08T23:19:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4100060 0x7f72f4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4105160 0x7f72f41978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f419bc50 0x7f72f419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f419c6d0 con 0x7f72f4105160 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f419c840 con 0x7f72f4100060 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7f72fc146700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f419ca00 con 0x7f72f419bc50 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe118bfd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 0x7fe1141999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe118bfd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 0x7fe1141999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44834/0 (socket says 172.21.15.16:44834) 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe118bfd700 1 -- 172.21.15.16:0/1314460467 learned_addr learned my addr 172.21.15.16:0/1314460467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe1193fe700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114069440 0x7fe1141994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.697+0000 7fe119bff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe11419dd90 0x7fe11419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72fa6e3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f419bc50 0x7f72f419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4100060 0x7f72f4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72fa6e3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f419bc50 0x7f72f419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:59764/0 (socket says 172.21.15.16:59764) 2024-05-08T23:19:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72fa6e3700 1 -- 172.21.15.16:0/3132387716 learned_addr learned my addr 172.21.15.16:0/3132387716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4105160 0x7f72f41978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe118bfd700 1 -- 172.21.15.16:0/1314460467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114069440 msgr2=0x7fe1141994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe118bfd700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114069440 0x7fe1141994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe118bfd700 1 -- 172.21.15.16:0/1314460467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe11419dd90 msgr2=0x7fe11419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe118bfd700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe11419dd90 0x7fe11419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe118bfd700 1 -- 172.21.15.16:0/1314460467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009c40 con 0x7fe114106250 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe1193fe700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114069440 0x7fe1141994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe119bff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe11419dd90 0x7fe11419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 -- 172.21.15.16:0/3132387716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f419bc50 msgr2=0x7f72f419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f419bc50 0x7f72f419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 -- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4105160 msgr2=0x7f72f41978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4105160 0x7f72f41978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 -- 172.21.15.16:0/3132387716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f00095f0 con 0x7f72f4100060 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4105160 0x7f72f41978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe118bfd700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 0x7fe1141999f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e970 tx=0x7fe10400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4100060 0x7f72f4197370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72f0010ea0 tx=0x7f72f000b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104009790 con 0x7fe114106250 2024-05-08T23:19:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe10400eec0 con 0x7fe114106250 2024-05-08T23:19:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe11b662700 1 -- 172.21.15.16:0/1314460467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1100095f0 con 0x7fe114106250 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1040054b0 con 0x7fe114106250 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7fe11b662700 1 -- 172.21.15.16:0/1314460467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141a63f0 con 0x7fe114106250 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.698+0000 7f72eaffd700 1 -- 172.21.15.16:0/3132387716 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0015070 con 0x7f72f4100060 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.699+0000 7f72eaffd700 1 -- 172.21.15.16:0/3132387716 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72f00044b0 con 0x7f72f4100060 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.699+0000 7f72eaffd700 1 -- 172.21.15.16:0/3132387716 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0003dd0 con 0x7f72f4100060 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.699+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e4009520 con 0x7f72f4100060 2024-05-08T23:19:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.699+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f410d8b0 con 0x7f72f4100060 2024-05-08T23:19:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.700+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe114102600 con 0x7fe114106250 2024-05-08T23:19:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.701+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fe104004750 con 0x7fe114106250 2024-05-08T23:19:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.701+0000 7f72eaffd700 1 -- 172.21.15.16:0/3132387716 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f72f0003f30 con 0x7f72f4100060 2024-05-08T23:19:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7fe10a7fc700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1000a1320 0x7fe1000a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7fe1193fe700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1000a1320 0x7fe1000a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fe104014070 con 0x7fe114106250 2024-05-08T23:19:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7f72eaffd700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72e00a1320 0x7f72e00a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72e00a1320 0x7f72e00a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7fe1193fe700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1000a1320 0x7fe1000a37e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110005c60 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.702+0000 7f72eaffd700 1 -- 172.21.15.16:0/3132387716 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f72f00d0b50 con 0x7f72f4100060 2024-05-08T23:19:48.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.703+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72e00a1320 0x7f72e00a37e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005e90 tx=0x7f72e4019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.703+0000 7f7308300b80 1 Processor -- start 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 -- start start 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f871260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f86b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f7c21d0 con 0x558f9f870e80 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f7bfb90 con 0x558f9f86b910 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f7308300b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f7bb480 con 0x558f9f8717a0 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f871260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f871260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44852/0 (socket says 172.21.15.16:44852) 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 -- 172.21.15.16:0/1414564994 learned_addr learned my addr 172.21.15.16:0/1414564994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72fa6e3700 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f96e1700 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f86b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 -- 172.21.15.16:0/1414564994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 msgr2=0x558f9f86b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f86b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 -- 172.21.15.16:0/1414564994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 msgr2=0x558f9f86bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.704+0000 7f72f9ee2700 1 -- 172.21.15.16:0/1414564994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9f63c430 con 0x558f9f870e80 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.705+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f871260 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b8f0 tx=0x7f72f0004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7486b4e89daf962 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f72dffff700 1 -- 172.21.15.16:0/1414564994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f009de80 con 0x558f9f870e80 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f72dffff700 1 -- 172.21.15.16:0/1414564994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72f009d780 con 0x558f9f870e80 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f72dffff700 1 -- 172.21.15.16:0/1414564994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f00bd0d0 con 0x558f9f870e80 2024-05-08T23:19:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe1040963d0 con 0x7fe114106250 2024-05-08T23:19:48.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 -- 172.21.15.16:0/1414564994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 msgr2=0x558f9f871260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f871260 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f72f000b8f0 tx=0x7f72f0004620 comp rx=0 tx=0).stop 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 -- 172.21.15.16:0/1414564994 shutdown_connections 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f871260 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 --2- 172.21.15.16:0/1414564994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f86b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.707+0000 7f7308300b80 1 -- 172.21.15.16:0/1414564994 >> 172.21.15.16:0/1414564994 conn(0x558f9f7c9290 msgr2=0x558f9f865e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.708+0000 7f7308300b80 1 -- 172.21.15.16:0/1414564994 shutdown_connections 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.708+0000 7f7308300b80 1 -- 172.21.15.16:0/1414564994 wait complete. 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.708+0000 7f7308300b80 1 -- start start 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.708+0000 7f7308300b80 1 Processor -- start 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f874ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f875010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f875550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f7575e0 con 0x558f9f870e80 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f757280 con 0x558f9f86b910 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f63e720 con 0x558f9f8717a0 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f7308300b80 10 client.0 _ll_register_callbacks cb 0x558f9f760bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f875010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f875010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44866/0 (socket says 172.21.15.16:44866) 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 -- 172.21.15.16:0/3271805826 learned_addr learned my addr 172.21.15.16:0/3271805826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f874ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72fa6e3700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f875550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 -- 172.21.15.16:0/3271805826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 msgr2=0x558f9f875550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f875550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 -- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 msgr2=0x558f9f874ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f874ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 -- 172.21.15.16:0/3271805826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f410d8b0 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f874ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f875010 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f72e4016000 tx=0x7f72e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92971135ad7bbbdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.709+0000 7f72dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3271805826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4005040 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f9f7142f0 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3271805826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f72e40051a0 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3271805826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e4005360 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 10 client.15678 fetch_fsmap learned FSMap version 25 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558f9f5e8be0 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3271805826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558f9f7142f0 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3271805826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72e4023420 con 0x558f9f870e80 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:48.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 20 client.15678 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f885240 0x558f9f887660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 4 client.15678 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.710+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f9f83a890 con 0x558f9f885240 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.711+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f885240 0x558f9f887660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.711+0000 7f72dd7fa700 10 client.15678 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.711+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f885240 0x558f9f887660 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f5386f5ec72e2f server_cookie=b607ea8b2b51f6bc in_seq=0 out_seq=0 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3271805826 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f9f83a890 con 0x558f9f885240 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f72dd7fa700 10 client.15678 handle_command_reply: tid=0 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 client.15678 shutdown 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 msgr2=0x558f9f875010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f875010 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f72e4016000 tx=0x7f72e4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 shutdown_connections 2024-05-08T23:19:48.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f875550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f885240 0x558f9f887660 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f874ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 --2- 172.21.15.16:0/3271805826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f875010 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f72dd7fa700 0 client.15678 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 >> 172.21.15.16:0/3271805826 conn(0x558f9f7c9290 msgr2=0x558f9f865620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 shutdown_connections 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 1 -- 172.21.15.16:0/3271805826 wait complete. 2024-05-08T23:19:48.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.712+0000 7f7308300b80 20 client.15678 trim_cache size 0 max 16384 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 Processor -- start 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 -- start start 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f873c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f865da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f86f610 con 0x558f9f870e80 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f862e40 con 0x558f9f86b910 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f7308300b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f866960 con 0x558f9f8717a0 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f72f96e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f873c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f72f96e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f873c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44880/0 (socket says 172.21.15.16:44880) 2024-05-08T23:19:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f72f96e1700 1 -- 172.21.15.16:0/279649786 learned_addr learned my addr 172.21.15.16:0/279649786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.731+0000 7f72fa6e3700 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f865da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72f96e1700 1 -- 172.21.15.16:0/279649786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 msgr2=0x558f9f865da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72f96e1700 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f865da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72f96e1700 1 -- 172.21.15.16:0/279649786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 msgr2=0x558f9f86b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72f96e1700 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72f96e1700 1 -- 172.21.15.16:0/279649786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f9f86bfc0 con 0x558f9f870e80 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72f96e1700 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f873c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000c00 tx=0x7f72e400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1ee3e8a391a118 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72e8ff9700 1 -- 172.21.15.16:0/279649786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4005b40 con 0x558f9f870e80 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72e8ff9700 1 -- 172.21.15.16:0/279649786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72e4029040 con 0x558f9f870e80 2024-05-08T23:19:48.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.732+0000 7f72e8ff9700 1 -- 172.21.15.16:0/279649786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e400ec40 con 0x558f9f870e80 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 -- 172.21.15.16:0/279649786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 msgr2=0x558f9f873c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f873c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f72e4000c00 tx=0x7f72e400b660 comp rx=0 tx=0).stop 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 -- 172.21.15.16:0/279649786 shutdown_connections 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f865da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f86b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 --2- 172.21.15.16:0/279649786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f873c30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 -- 172.21.15.16:0/279649786 >> 172.21.15.16:0/279649786 conn(0x558f9f7c9290 msgr2=0x558f9f870220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 -- 172.21.15.16:0/279649786 shutdown_connections 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 -- 172.21.15.16:0/279649786 wait complete. 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 -- start start 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.733+0000 7f7308300b80 1 Processor -- start 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f878f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f879440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f879980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f8db8e0 con 0x558f9f870e80 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f8dba50 con 0x558f9f86b910 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f9f8dbc10 con 0x558f9f8717a0 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f7308300b80 10 client.0 _ll_register_callbacks cb 0x558f9f760bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f879440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f879440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44888/0 (socket says 172.21.15.16:44888) 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 -- 172.21.15.16:0/3976826606 learned_addr learned my addr 172.21.15.16:0/3976826606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72fa6e3700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f879980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f878f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 -- 172.21.15.16:0/3976826606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 msgr2=0x558f9f879980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f879980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 -- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 msgr2=0x558f9f878f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f878f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f96e1700 1 -- 172.21.15.16:0/3976826606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f9f5e8be0 con 0x558f9f870e80 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.734+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f878f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72f96e1700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f879440 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009c40 tx=0x7f72e4023650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f2403c96e73be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3976826606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40050e0 con 0x558f9f870e80 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3976826606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f72e4005240 con 0x558f9f870e80 2024-05-08T23:19:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72f009d780 con 0x558f9f870e80 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3976826606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e4005400 con 0x558f9f870e80 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f7308300b80 10 client.15684 fetch_fsmap learned FSMap version 25 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558f9f7815e0 con 0x558f9f870e80 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3976826606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72f009d780 con 0x558f9f870e80 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.735+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3976826606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72e4020070 con 0x558f9f870e80 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f7308300b80 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f7308300b80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f7308300b80 20 client.15684 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f7308300b80 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f8dd010 0x558f9f8df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f7308300b80 4 client.15684 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558f9f7b3f40 con 0x558f9f8dd010 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f8dd010 0x558f9f8df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f72dd7fa700 10 client.15684 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.736+0000 7f72f9ee2700 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f8dd010 0x558f9f8df430 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96789b57365d13af server_cookie=246de2afc2c3fde5 in_seq=0 out_seq=0 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f72dd7fa700 1 -- 172.21.15.16:0/3976826606 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558f9f7b3f40 con 0x558f9f8dd010 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f72dd7fa700 10 client.15684 handle_command_reply: tid=0 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 client.15684 shutdown 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 msgr2=0x558f9f879440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f879440 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009c40 tx=0x7f72e4023650 comp rx=0 tx=0).stop 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 shutdown_connections 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f9f8717a0 0x558f9f879980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558f9f8dd010 0x558f9f8df430 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558f9f86b910 0x558f9f878f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 --2- 172.21.15.16:0/3976826606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f9f870e80 0x558f9f879440 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f72dd7fa700 0 client.15684 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 >> 172.21.15.16:0/3976826606 conn(0x558f9f7c9290 msgr2=0x558f9f872410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 shutdown_connections 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.738+0000 7f7308300b80 1 -- 172.21.15.16:0/3976826606 wait complete. 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f7308300b80 20 client.15684 trim_cache size 0 max 16384 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "663684ba-82be-4cd6-b41b-1cec4e58da90", 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:19:48.744 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72e00a1320 msgr2=0x7f72e00a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72e00a1320 0x7f72e00a37e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005e90 tx=0x7f72e4019040 comp rx=0 tx=0).stop 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4100060 msgr2=0x7f72f4197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4100060 0x7f72f4197370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72f0010ea0 tx=0x7f72f000b5a0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 shutdown_connections 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72e00a1320 0x7f72e00a37e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72f419bc50 0x7f72f419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72f4100060 0x7f72f4197370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 --2- 172.21.15.16:0/3132387716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f4105160 0x7f72f41978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 >> 172.21.15.16:0/3132387716 conn(0x7f72f4076f40 msgr2=0x7f72f40fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 shutdown_connections 2024-05-08T23:19:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.739+0000 7f72fc146700 1 -- 172.21.15.16:0/3132387716 wait complete. 2024-05-08T23:19:48.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:19:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 -- 172.21.15.16:0/1041563490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 msgr2=0x7f9cb80ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 --2- 172.21.15.16:0/1041563490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb80ffeb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800aaa0 tx=0x7f9ca800adb0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 -- 172.21.15.16:0/1041563490 shutdown_connections 2024-05-08T23:19:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 --2- 172.21.15.16:0/1041563490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb8104900 0x7f9cb810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 --2- 172.21.15.16:0/1041563490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb80ffeb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 --2- 172.21.15.16:0/1041563490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8100480 0x7f9cb8104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 -- 172.21.15.16:0/1041563490 >> 172.21.15.16:0/1041563490 conn(0x7f9cb80750c0 msgr2=0x7f9cb80754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 -- 172.21.15.16:0/1041563490 shutdown_connections 2024-05-08T23:19:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.895+0000 7f9cbd682700 1 -- 172.21.15.16:0/1041563490 wait complete. 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.896+0000 7f9cbd682700 1 Processor -- start 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.896+0000 7f9cbd682700 1 -- start start 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cbd682700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb81950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cbd682700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb8104900 0x7f9cb81955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cbd682700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8199980 0x7f9cb8199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cbd682700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819a400 con 0x7f9cb80ffad0 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cbd682700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819a570 con 0x7f9cb8104900 2024-05-08T23:19:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cbd682700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819a730 con 0x7f9cb8199980 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb81950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb81950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44912/0 (socket says 172.21.15.16:44912) 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/495431904 learned_addr learned my addr 172.21.15.16:0/495431904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb8104900 0x7f9cb81955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8199980 0x7f9cb8199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/495431904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8199980 msgr2=0x7f9cb8199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8199980 0x7f9cb8199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/495431904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb8104900 msgr2=0x7f9cb81955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb8104900 0x7f9cb81955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/495431904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca800a700 con 0x7f9cb80ffad0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.897+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb81950a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006010 tx=0x7f9ca800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.898+0000 7f9cb4ff9700 1 -- 172.21.15.16:0/495431904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800eba0 con 0x7f9cb80ffad0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.898+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb81a1c30 con 0x7f9cb80ffad0 2024-05-08T23:19:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.898+0000 7f9cb4ff9700 1 -- 172.21.15.16:0/495431904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca800ed00 con 0x7f9cb80ffad0 2024-05-08T23:19:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.898+0000 7f9cb4ff9700 1 -- 172.21.15.16:0/495431904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8020670 con 0x7f9cb80ffad0 2024-05-08T23:19:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.898+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb81a20b0 con 0x7f9cb80ffad0 2024-05-08T23:19:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.900+0000 7f9cb4ff9700 1 -- 172.21.15.16:0/495431904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f9ca8016420 con 0x7f9cb80ffad0 2024-05-08T23:19:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.901+0000 7f9cb4ff9700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9ca40a1320 0x7f9ca40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.901+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9ca40a1320 0x7f9ca40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.901+0000 7f9cb4ff9700 1 -- 172.21.15.16:0/495431904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f9ca8012070 con 0x7f9cb80ffad0 2024-05-08T23:19:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.901+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9ca40a1320 0x7f9ca40a37e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cb81965f0 tx=0x7f9ca0009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.902+0000 7f9cc983cb80 1 Processor -- start 2024-05-08T23:19:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.902+0000 7f9cc983cb80 1 -- start start 2024-05-08T23:19:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.902+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d85510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.902+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d7f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.902+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631cd4150 con 0x55f631d7ff20 2024-05-08T23:19:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631cd1b10 con 0x55f631d7f560 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631ccd400 con 0x55f631d85130 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44914/0 (socket says 172.21.15.16:44914) 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 -- 172.21.15.16:0/366347474 learned_addr learned my addr 172.21.15.16:0/366347474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d85510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d7f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 -- 172.21.15.16:0/366347474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 msgr2=0x55f631d85510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d85510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 -- 172.21.15.16:0/366347474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 msgr2=0x55f631d7f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d7f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 -- 172.21.15.16:0/366347474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f631b4e3b0 con 0x55f631d7ff20 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d85510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8bea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00098f0 tx=0x7f9cb0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e8df81dfb33b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.903+0000 7f9cad7fa700 1 -- 172.21.15.16:0/366347474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0019070 con 0x55f631d7ff20 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cad7fa700 1 -- 172.21.15.16:0/366347474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb000f040 con 0x55f631d7ff20 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cad7fa700 1 -- 172.21.15.16:0/366347474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0007a50 con 0x55f631d7ff20 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 -- 172.21.15.16:0/366347474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 msgr2=0x55f631d8bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8bea0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00098f0 tx=0x7f9cb0009c00 comp rx=0 tx=0).stop 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 -- 172.21.15.16:0/366347474 shutdown_connections 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8bea0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d85510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/366347474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d7f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 -- 172.21.15.16:0/366347474 >> 172.21.15.16:0/366347474 conn(0x55f631cdb210 msgr2=0x55f631d77df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.904+0000 7f9cc983cb80 1 -- 172.21.15.16:0/366347474 shutdown_connections 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 -- 172.21.15.16:0/366347474 wait complete. 2024-05-08T23:19:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 -- start start 2024-05-08T23:19:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 Processor -- start 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d8edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d89160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631c69560 con 0x55f631d7ff20 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631c69200 con 0x55f631d7f560 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.905+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631b506a0 con 0x55f631d85130 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cc983cb80 10 client.0 _ll_register_callbacks cb 0x55f631c72b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d8edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb77fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d89160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d8edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:52642/0 (socket says 172.21.15.16:52642) 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/2945276479 learned_addr learned my addr 172.21.15.16:0/2945276479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/2945276479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 msgr2=0x55f631d89160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d89160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/2945276479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 msgr2=0x55f631d8f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.906+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/2945276479 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb81a20b0 con 0x55f631d7f560 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d8edf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c6d0 tx=0x7f9ca80cd6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f555bdb7ccc637 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9c9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/2945276479 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca809cad0 con 0x55f631d7f560 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f631c26270 con 0x55f631d7f560 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/2945276479 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f9ca80a0900 con 0x55f631d7f560 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/2945276479 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca80a4ec0 con 0x55f631d7f560 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cc983cb80 10 client.25532 fetch_fsmap learned FSMap version 25 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9cb0003c50 con 0x55f631d7f560 2024-05-08T23:19:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/2945276479 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f631c26270 con 0x55f631d7f560 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/2945276479 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9ca80a4a30 con 0x55f631d7f560 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cc983cb80 10 client.25532 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cc983cb80 10 client.25532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.907+0000 7f9cc983cb80 20 client.25532 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.908+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631d98ff0 0x55f631d9b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.908+0000 7f9cc983cb80 4 client.25532 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.908+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f631d4c810 con 0x55f631d98ff0 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.908+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631d98ff0 0x55f631d9b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.908+0000 7f9c9e7fc700 10 client.25532 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.908+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631d98ff0 0x55f631d9b410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6ed245aba3b0aa server_cookie=c29e677850bfa8ce in_seq=0 out_seq=0 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/2945276479 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f631d4c810 con 0x55f631d98ff0 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9c9e7fc700 10 client.25532 handle_command_reply: tid=0 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 client.25532 shutdown 2024-05-08T23:19:48.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 msgr2=0x55f631d8edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d8edf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c6d0 tx=0x7f9ca80cd6c0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 shutdown_connections 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631d89160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631d98ff0 0x55f631d9b410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7f560 0x55f631d8edf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/2945276479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d8f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 >> 172.21.15.16:0/2945276479 conn(0x55f631cdb210 msgr2=0x55f631d775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9c9e7fc700 0 client.25532 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 shutdown_connections 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 1 -- 172.21.15.16:0/2945276479 wait complete. 2024-05-08T23:19:48.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.909+0000 7f9cc983cb80 20 client.25532 trim_cache size 0 max 16384 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 Processor -- start 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 -- start start 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d7f560 0x55f631d77d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d85130 0x55f631d88f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631d858c0 con 0x55f631d7ff20 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631d78b30 con 0x55f631d85130 2024-05-08T23:19:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631d8a710 con 0x55f631d7f560 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d889f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44942/0 (socket says 172.21.15.16:44942) 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.928+0000 7f9cae7fc700 1 -- 172.21.15.16:0/3293314618 learned_addr learned my addr 172.21.15.16:0/3293314618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d7f560 0x55f631d77d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d85130 0x55f631d88f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cae7fc700 1 -- 172.21.15.16:0/3293314618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d7f560 msgr2=0x55f631d77d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d7f560 0x55f631d77d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cae7fc700 1 -- 172.21.15.16:0/3293314618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d85130 msgr2=0x55f631d88f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d85130 0x55f631d88f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cae7fc700 1 -- 172.21.15.16:0/3293314618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f631afab60 con 0x55f631d7ff20 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d889f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0013010 tx=0x7f9ca0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b3e30c5f317aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cadffb700 1 -- 172.21.15.16:0/3293314618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0022030 con 0x55f631d7ff20 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cadffb700 1 -- 172.21.15.16:0/3293314618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca0023040 con 0x55f631d7ff20 2024-05-08T23:19:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.929+0000 7f9cadffb700 1 -- 172.21.15.16:0/3293314618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca002b5e0 con 0x55f631d7ff20 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 -- 172.21.15.16:0/3293314618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 msgr2=0x55f631d889f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d889f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0013010 tx=0x7f9ca0014000 comp rx=0 tx=0).stop 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 -- 172.21.15.16:0/3293314618 shutdown_connections 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d85130 0x55f631d88f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d7f560 0x55f631d77d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/3293314618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7ff20 0x55f631d889f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 -- 172.21.15.16:0/3293314618 >> 172.21.15.16:0/3293314618 conn(0x55f631cdb210 msgr2=0x55f631d83170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 -- 172.21.15.16:0/3293314618 shutdown_connections 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 -- 172.21.15.16:0/3293314618 wait complete. 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.930+0000 7f9cc983cb80 1 -- start start 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 Processor -- start 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 0x55f631d93820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7ff20 0x55f631d93d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631de8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631de8a70 con 0x55f631d7f560 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631de8be0 con 0x55f631d7ff20 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f631de8da0 con 0x55f631d85130 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cc983cb80 10 client.0 _ll_register_callbacks cb 0x55f631c72b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 0x55f631d93820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 0x55f631d93820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44956/0 (socket says 172.21.15.16:44956) 2024-05-08T23:19:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb6ffd700 1 -- 172.21.15.16:0/4249140431 learned_addr learned my addr 172.21.15.16:0/4249140431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631de8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cae7fc700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7ff20 0x55f631d93d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb77fe700 1 -- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7ff20 msgr2=0x55f631d93d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7ff20 0x55f631d93d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb77fe700 1 -- 172.21.15.16:0/4249140431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 msgr2=0x55f631d93820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 0x55f631d93820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.931+0000 7f9cb77fe700 1 -- 172.21.15.16:0/4249140431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca0017070 con 0x55f631d85130 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 0x55f631d93820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9cb77fe700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631de8530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0000f80 tx=0x7f9cb0008e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c62fbaadb67ac63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9c9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/4249140431 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0019040 con 0x55f631d85130 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/4249140431 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f9cb00046d0 con 0x55f631d85130 2024-05-08T23:19:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cb000f040 con 0x55f631d85130 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/4249140431 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb00048d0 con 0x55f631d85130 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9cc983cb80 10 client.25558 fetch_fsmap learned FSMap version 25 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f631c93560 con 0x55f631d85130 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.932+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/4249140431 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cb000f040 con 0x55f631d85130 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/4249140431 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cb0015be0 con 0x55f631d85130 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cc983cb80 10 client.25558 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cc983cb80 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cc983cb80 20 client.25558 populate_metadata read hostname 'smithi016' 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631df04c0 0x55f631df28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cc983cb80 4 client.25558 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f631cc5ec0 con 0x55f631df04c0 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631df04c0 0x55f631df28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9cb6ffd700 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631df04c0 0x55f631df28e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522a0688417f17d6 server_cookie=1336c7da161aa860 in_seq=0 out_seq=0 2024-05-08T23:19:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.933+0000 7f9c9e7fc700 10 client.25558 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9c9e7fc700 1 -- 172.21.15.16:0/4249140431 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f631cc5ec0 con 0x55f631df04c0 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9c9e7fc700 10 client.25558 handle_command_reply: tid=0 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 client.25558 shutdown 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 msgr2=0x55f631de8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631de8530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0000f80 tx=0x7f9cb0008e20 comp rx=0 tx=0).stop 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 shutdown_connections 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f631d85130 0x55f631de8530 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f631d7f560 0x55f631d93820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f631df04c0 0x55f631df28e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 --2- 172.21.15.16:0/4249140431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f631d7ff20 0x55f631d93d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9c9e7fc700 0 client.25558 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 >> 172.21.15.16:0/4249140431 conn(0x55f631cdb210 msgr2=0x55f631d889f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 shutdown_connections 2024-05-08T23:19:48.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 1 -- 172.21.15.16:0/4249140431 wait complete. 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cc983cb80 20 client.25558 trim_cache size 0 max 16384 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "663684ba-82be-4cd6-b41b-1cec4e58da90": { 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "663684ba-82be-4cd6-b41b-1cec4e58da90", 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:19:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:19:48.942 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:19:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9ca40a1320 msgr2=0x7f9ca40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cbd682700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9ca40a1320 0x7f9ca40a37e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cb81965f0 tx=0x7f9ca0009260 comp rx=0 tx=0).stop 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 msgr2=0x7f9cb81950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.934+0000 7f9cbd682700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb81950a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006010 tx=0x7f9ca800e770 comp rx=0 tx=0).stop 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 shutdown_connections 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9ca40a1320 0x7f9ca40a37e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8199980 0x7f9cb8199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cb80ffad0 0x7f9cb81950a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 --2- 172.21.15.16:0/495431904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb8104900 0x7f9cb81955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 >> 172.21.15.16:0/495431904 conn(0x7f9cb80750c0 msgr2=0x7f9cb80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 shutdown_connections 2024-05-08T23:19:48.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.935+0000 7f9cbd682700 1 -- 172.21.15.16:0/495431904 wait complete. 2024-05-08T23:19:48.950 INFO:tasks.cephfs.filesystem:scrub status for tag:663684ba-82be-4cd6-b41b-1cec4e58da90 - {'path': '/', 'tag': '663684ba-82be-4cd6-b41b-1cec4e58da90', 'options': 'recursive,force'} 2024-05-08T23:19:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:48.974+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1140675a0 con 0x7fe1000a1320 2024-05-08T23:19:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.050+0000 7fe10a7fc700 1 -- 172.21.15.16:0/1314460467 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe1140675a0 con 0x7fe1000a1320 2024-05-08T23:19:49.051 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1000a1320 msgr2=0x7fe1000a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1000a1320 0x7fe1000a37e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110005c60 comp rx=0 tx=0).stop 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 msgr2=0x7fe1141999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 0x7fe1141999f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e970 tx=0x7fe10400ec80 comp rx=0 tx=0).stop 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 shutdown_connections 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1000a1320 0x7fe1000a37e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe11419dd90 0x7fe11419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe114069440 0x7fe1141994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 --2- 172.21.15.16:0/1314460467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe114106250 0x7fe1141999f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 >> 172.21.15.16:0/1314460467 conn(0x7fe114076f40 msgr2=0x7fe11410c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 shutdown_connections 2024-05-08T23:19:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.053+0000 7fe0fffff700 1 -- 172.21.15.16:0/1314460467 wait complete. 2024-05-08T23:19:49.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-08T23:19:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:48 smithi155 ceph-mon[47105]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 911 B/s wr, 17 op/s 2024-05-08T23:19:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:48 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3891402057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:48 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1674512550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 -- 172.21.15.16:0/4146419522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 msgr2=0x7fcba810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 --2- 172.21.15.16:0/4146419522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba810ccb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcba000b0a0 tx=0x7fcba000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 -- 172.21.15.16:0/4146419522 shutdown_connections 2024-05-08T23:19:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 --2- 172.21.15.16:0/4146419522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba810ccb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 --2- 172.21.15.16:0/4146419522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcba8100010 0x7fcba81003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 --2- 172.21.15.16:0/4146419522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba81009c0 0x7fcba8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 -- 172.21.15.16:0/4146419522 >> 172.21.15.16:0/4146419522 conn(0x7fcba80fb870 msgr2=0x7fcba80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:49.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 -- 172.21.15.16:0/4146419522 shutdown_connections 2024-05-08T23:19:49.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.254+0000 7fcbb0784700 1 -- 172.21.15.16:0/4146419522 wait complete. 2024-05-08T23:19:49.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 Processor -- start 2024-05-08T23:19:49.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 -- start start 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba8100010 0x7fcba8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcba819bd00 0x7fcba819c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba819c780 con 0x7fcba8105130 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba819c8f0 con 0x7fcba819bd00 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbb0784700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba819cab0 con 0x7fcba8100010 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba8197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44978/0 (socket says 172.21.15.16:44978) 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 -- 172.21.15.16:0/1689855987 learned_addr learned my addr 172.21.15.16:0/1689855987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbaed21700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcba819bd00 0x7fcba819c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbae520700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba8100010 0x7fcba8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 -- 172.21.15.16:0/1689855987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba8100010 msgr2=0x7fcba8197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba8100010 0x7fcba8197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 -- 172.21.15.16:0/1689855987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcba819bd00 msgr2=0x7fcba819c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.255+0000 7fcbadd1f700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcba819bd00 0x7fcba819c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcbadd1f700 1 -- 172.21.15.16:0/1689855987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0009d00 con 0x7fcba8105130 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcbadd1f700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba8197960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800d980 tx=0x7fcb9800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98009b40 con 0x7fcba8105130 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb9800de60 con 0x7fcba8105130 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba81a3fb0 con 0x7fcba8105130 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800f620 con 0x7fcba8105130 2024-05-08T23:19:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.256+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba81a44e0 con 0x7fcba8105130 2024-05-08T23:19:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.257+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba8109410 con 0x7fcba8105130 2024-05-08T23:19:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.258+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7fcb98010460 con 0x7fcba8105130 2024-05-08T23:19:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.259+0000 7fcb9f7fe700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb940a13a0 0x7fcb940a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.259+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fcb980ccc20 con 0x7fcba8105130 2024-05-08T23:19:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.259+0000 7fcbae520700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb940a13a0 0x7fcb940a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.260+0000 7fcbae520700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb940a13a0 0x7fcb940a3860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009bf0 tx=0x7fcba4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.263+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb98095c70 con 0x7fcba8105130 2024-05-08T23:19:49.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:48 smithi016 ceph-mon[42541]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 911 B/s wr, 17 op/s 2024-05-08T23:19:49.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:48 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3891402057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:49.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:48 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1674512550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:48 smithi189 ceph-mon[46919]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 911 B/s wr, 17 op/s 2024-05-08T23:19:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:48 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3891402057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:19:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:48 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1674512550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:19:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.510+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba8198790 con 0x7fcb940a13a0 2024-05-08T23:19:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.580+0000 7fcb9f7fe700 1 -- 172.21.15.16:0/1689855987 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fcba8198790 con 0x7fcb940a13a0 2024-05-08T23:19:49.581 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.582+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb940a13a0 msgr2=0x7fcb940a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.582+0000 7fcbb0784700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb940a13a0 0x7fcb940a3860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009bf0 tx=0x7fcba4009260 comp rx=0 tx=0).stop 2024-05-08T23:19:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.582+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 msgr2=0x7fcba8197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.582+0000 7fcbb0784700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba8197960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800d980 tx=0x7fcb9800dd40 comp rx=0 tx=0).stop 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 shutdown_connections 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb940a13a0 0x7fcb940a3860 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcba819bd00 0x7fcba819c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba8100010 0x7fcba8197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 --2- 172.21.15.16:0/1689855987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8105130 0x7fcba8197960 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 >> 172.21.15.16:0/1689855987 conn(0x7fcba80fb870 msgr2=0x7fcba8069550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 shutdown_connections 2024-05-08T23:19:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.583+0000 7fcbb0784700 1 -- 172.21.15.16:0/1689855987 wait complete. 2024-05-08T23:19:49.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T23:19:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 -- 172.21.15.16:0/3826122318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 msgr2=0x7f226810d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3826122318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f226810d3b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2250009990 tx=0x7f2250009ca0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 -- 172.21.15.16:0/3826122318 shutdown_connections 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3826122318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f226810d8f0 0x7f226810fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3826122318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2268100880 0x7f2268100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3826122318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f226810d3b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 -- 172.21.15.16:0/3826122318 >> 172.21.15.16:0/3826122318 conn(0x7f2268076f10 msgr2=0x7f2268077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 -- 172.21.15.16:0/3826122318 shutdown_connections 2024-05-08T23:19:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.739+0000 7f226d1d2700 1 -- 172.21.15.16:0/3826122318 wait complete. 2024-05-08T23:19:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.740+0000 7f226d1d2700 1 Processor -- start 2024-05-08T23:19:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.740+0000 7f226d1d2700 1 -- start start 2024-05-08T23:19:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226d1d2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2268100880 0x7f2268199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226d1d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f22681999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226d1d2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226810d8f0 0x7f226819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226d1d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226819e380 con 0x7f22681011a0 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226d1d2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226819e4f0 con 0x7f226810d8f0 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226d1d2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226819e6b0 con 0x7f2268100880 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f22681999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f22681999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45004/0 (socket says 172.21.15.16:45004) 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 -- 172.21.15.16:0/3351706850 learned_addr learned my addr 172.21.15.16:0/3351706850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f2266d9d700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2268100880 0x7f2268199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f226759e700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226810d8f0 0x7f226819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 -- 172.21.15.16:0/3351706850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2268100880 msgr2=0x7f2268199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2268100880 0x7f2268199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 -- 172.21.15.16:0/3351706850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226810d8f0 msgr2=0x7f226819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226810d8f0 0x7f226819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 -- 172.21.15.16:0/3351706850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500095f0 con 0x7f22681011a0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.741+0000 7f225e59c700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f22681999a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2250004730 tx=0x7f2250009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.742+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225001d070 con 0x7f22681011a0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.742+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226819e930 con 0x7f22681011a0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.742+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2250004ca0 con 0x7f22681011a0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.742+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225000f750 con 0x7f22681011a0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.742+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226819eba0 con 0x7f22681011a0 2024-05-08T23:19:49.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.743+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2268108b10 con 0x7f22681011a0 2024-05-08T23:19:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.744+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f2250004810 con 0x7f22681011a0 2024-05-08T23:19:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.744+0000 7f2264d99700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22540a1320 0x7f22540a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.745+0000 7f2266d9d700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22540a1320 0x7f22540a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.745+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f22500ce030 con 0x7f22681011a0 2024-05-08T23:19:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.745+0000 7f2266d9d700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22540a1320 0x7f22540a37e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f225800ac50 tx=0x7f225800a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.749+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f225005bd60 con 0x7f22681011a0 2024-05-08T23:19:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:49.997+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2268112f00 con 0x7f22540a1320 2024-05-08T23:19:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.006+0000 7f2264d99700 1 -- 172.21.15.16:0/3351706850 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f2268112f00 con 0x7f22540a1320 2024-05-08T23:19:50.006 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T23:19:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22540a1320 msgr2=0x7f22540a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22540a1320 0x7f22540a37e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f225800ac50 tx=0x7f225800a380 comp rx=0 tx=0).stop 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 msgr2=0x7f22681999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f22681999a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2250004730 tx=0x7f2250009580 comp rx=0 tx=0).stop 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 shutdown_connections 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22540a1320 0x7f22540a37e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226810d8f0 0x7f226819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2268100880 0x7f2268199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 --2- 172.21.15.16:0/3351706850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22681011a0 0x7f22681999a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 >> 172.21.15.16:0/3351706850 conn(0x7f2268076f10 msgr2=0x7f226810f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 shutdown_connections 2024-05-08T23:19:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.008+0000 7f226d1d2700 1 -- 172.21.15.16:0/3351706850 wait complete. 2024-05-08T23:19:50.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T23:19:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 -- 172.21.15.16:0/3042296720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 msgr2=0x7f292810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 --2- 172.21.15.16:0/3042296720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f292810cc30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f292000b0a0 tx=0x7f292000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 -- 172.21.15.16:0/3042296720 shutdown_connections 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 --2- 172.21.15.16:0/3042296720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f292810cc30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 --2- 172.21.15.16:0/3042296720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928105f20 0x7f2928106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 --2- 172.21.15.16:0/3042296720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2928100280 0x7f2928100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 -- 172.21.15.16:0/3042296720 >> 172.21.15.16:0/3042296720 conn(0x7f2928076bc0 msgr2=0x7f2928076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 -- 172.21.15.16:0/3042296720 shutdown_connections 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.162+0000 7f292daf0700 1 -- 172.21.15.16:0/3042296720 wait complete. 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 Processor -- start 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 -- start start 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928100280 0x7f29281972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f2928197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2928105f20 0x7f292819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292819c1f0 con 0x7f2928100c40 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292819c360 con 0x7f2928105f20 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f292daf0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292819c520 con 0x7f2928100280 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f2926ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f2928197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f2926ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f2928197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45030/0 (socket says 172.21.15.16:45030) 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.163+0000 7f2926ffd700 1 -- 172.21.15.16:0/688745810 learned_addr learned my addr 172.21.15.16:0/688745810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f29277fe700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928100280 0x7f29281972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2927fff700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2928105f20 0x7f292819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2926ffd700 1 -- 172.21.15.16:0/688745810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928100280 msgr2=0x7f29281972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2926ffd700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928100280 0x7f29281972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2926ffd700 1 -- 172.21.15.16:0/688745810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2928105f20 msgr2=0x7f292819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2926ffd700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2928105f20 0x7f292819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2926ffd700 1 -- 172.21.15.16:0/688745810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009d00 con 0x7f2928100c40 2024-05-08T23:19:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f29277fe700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928100280 0x7f29281972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:19:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2926ffd700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f2928197810 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f291800e9a0 tx=0x7f291800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918009790 con 0x7f2928100c40 2024-05-08T23:19:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f291800cbd0 con 0x7f2928100c40 2024-05-08T23:19:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292819c800 con 0x7f2928100c40 2024-05-08T23:19:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918018680 con 0x7f2928100c40 2024-05-08T23:19:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.164+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29281a4040 con 0x7f2928100c40 2024-05-08T23:19:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.165+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2928101530 con 0x7f2928100c40 2024-05-08T23:19:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.167+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) v1 ==== 136049+0+0 (secure 0 0 0) 0x7f291800cd40 con 0x7f2928100c40 2024-05-08T23:19:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.167+0000 7f2924ff9700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29140a13a0 0x7f29140a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:19:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.167+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f2918014070 con 0x7f2928100c40 2024-05-08T23:19:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.167+0000 7f29277fe700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29140a13a0 0x7f29140a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:19:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.168+0000 7f29277fe700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29140a13a0 0x7f29140a3860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2910009cb0 tx=0x7f2910009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:19:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.171+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2918096600 con 0x7f2928100c40 2024-05-08T23:19:50.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:49 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:19:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:49 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:49 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:19:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:49 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:49 smithi016 ceph-mon[42541]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.416+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f292810fbd0 con 0x7f29140a13a0 2024-05-08T23:19:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:49 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:19:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:49 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:49 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:19:50.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:49 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:50.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:49 smithi189 ceph-mon[46919]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.423+0000 7f2924ff9700 1 -- 172.21.15.16:0/688745810 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f292810fbd0 con 0x7f29140a13a0 2024-05-08T23:19:50.425 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-05-08T23:19:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29140a13a0 msgr2=0x7f29140a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29140a13a0 0x7f29140a3860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2910009cb0 tx=0x7f2910009260 comp rx=0 tx=0).stop 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 msgr2=0x7f2928197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f2928197810 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f291800e9a0 tx=0x7f291800ed60 comp rx=0 tx=0).stop 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 shutdown_connections 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29140a13a0 0x7f29140a3860 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2928105f20 0x7f292819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2928100280 0x7f29281972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 --2- 172.21.15.16:0/688745810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2928100c40 0x7f2928197810 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:19:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 >> 172.21.15.16:0/688745810 conn(0x7f2928076bc0 msgr2=0x7f29280692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:19:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 shutdown_connections 2024-05-08T23:19:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:19:50.426+0000 7f292daf0700 1 -- 172.21.15.16:0/688745810 wait complete. 2024-05-08T23:19:50.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-08T23:19:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:49 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:19:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:49 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:49 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:19:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:49 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:19:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:49 smithi155 ceph-mon[47105]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:50.478 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-08T23:19:50.535 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T23:19:50.597 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:19:50.597 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T23:19:50.598 INFO:tasks.workunit:timeout=3h 2024-05-08T23:19:50.598 INFO:tasks.workunit:cleanup=True 2024-05-08T23:19:50.598 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 7ch/124d Inode: 1 Links: 2 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-05-08 23:18:13.574649337 +0000 2024-05-08T23:19:50.624 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-05-08 23:18:13.574649337 +0000 2024-05-08T23:19:50.625 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-05-08 23:19:38.886722768 +0000 2024-05-08T23:19:50.625 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2024-05-08T23:19:50.625 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T23:19:50.625 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T23:19:50.709 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T23:19:50.772 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T23:19:51.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:50 smithi016 ceph-mon[42541]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:50 smithi016 ceph-mon[42541]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 745 B/s wr, 17 op/s 2024-05-08T23:19:51.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:50 smithi189 ceph-mon[46919]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:50 smithi189 ceph-mon[46919]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 745 B/s wr, 17 op/s 2024-05-08T23:19:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:50 smithi155 ceph-mon[47105]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:50 smithi155 ceph-mon[47105]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 745 B/s wr, 17 op/s 2024-05-08T23:19:52.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:51 smithi016 ceph-mon[42541]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:51 smithi016 ceph-mon[42541]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:52.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:51 smithi189 ceph-mon[46919]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:51 smithi189 ceph-mon[46919]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:52.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:51 smithi155 ceph-mon[47105]: from='client.15714 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:51 smithi155 ceph-mon[47105]: from='client.15720 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:19:53.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:52 smithi016 ceph-mon[42541]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 15 op/s 2024-05-08T23:19:53.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:52 smithi189 ceph-mon[46919]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 15 op/s 2024-05-08T23:19:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:52 smithi155 ceph-mon[47105]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 15 op/s 2024-05-08T23:19:55.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:54 smithi016 ceph-mon[42541]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 15 op/s 2024-05-08T23:19:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:54 smithi189 ceph-mon[46919]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 15 op/s 2024-05-08T23:19:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:54 smithi155 ceph-mon[47105]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 682 B/s wr, 15 op/s 2024-05-08T23:19:57.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:56 smithi016 ceph-mon[42541]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T23:19:57.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:56 smithi189 ceph-mon[46919]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T23:19:57.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:56 smithi155 ceph-mon[47105]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T23:19:59.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:19:58 smithi016 ceph-mon[42541]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:19:59.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:19:58 smithi189 ceph-mon[46919]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:19:59.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:19:58 smithi155 ceph-mon[47105]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:00.864 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:20:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:20:00.591+0000 7f4156954700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:20:00.865 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:20:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:20:00.591+0000 7f4156954700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:20:00.865 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:20:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:20:00.591+0000 7f4156954700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:20:00.865 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:20:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:20:00.591+0000 7f4156954700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:20:00.865 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:20:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:20:00.591+0000 7f4156954700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T23:20:01.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:00 smithi016 ceph-mon[42541]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T23:20:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:00 smithi016 ceph-mon[42541]: overall HEALTH_OK 2024-05-08T23:20:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:00 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:20:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:00 smithi189 ceph-mon[46919]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T23:20:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:00 smithi189 ceph-mon[46919]: overall HEALTH_OK 2024-05-08T23:20:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:00 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:00 smithi155 ceph-mon[47105]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:00 smithi155 ceph-mon[47105]: overall HEALTH_OK 2024-05-08T23:20:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:00 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:20:03.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:03 smithi016 ceph-mon[42541]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:03.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:03 smithi189 ceph-mon[46919]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:03.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:03 smithi155 ceph-mon[47105]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:04.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:04 smithi016 ceph-mon[42541]: mgrmap e24: x(active, since 25s), standbys: y, z 2024-05-08T23:20:04.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:04 smithi016 ceph-mon[42541]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:04.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:04 smithi189 ceph-mon[46919]: mgrmap e24: x(active, since 25s), standbys: y, z 2024-05-08T23:20:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:04 smithi189 ceph-mon[46919]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:04 smithi155 ceph-mon[47105]: mgrmap e24: x(active, since 25s), standbys: y, z 2024-05-08T23:20:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:04 smithi155 ceph-mon[47105]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 36 op/s 2024-05-08T23:20:06.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:06 smithi016 ceph-mon[42541]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-05-08T23:20:06.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:06 smithi189 ceph-mon[46919]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-05-08T23:20:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:06 smithi155 ceph-mon[47105]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-05-08T23:20:08.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:08 smithi016 ceph-mon[42541]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:20:08.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:08 smithi189 ceph-mon[46919]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:20:08.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:08 smithi155 ceph-mon[47105]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:20:10.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:10 smithi016 ceph-mon[42541]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:20:10.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:10 smithi189 ceph-mon[46919]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:20:10.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:10 smithi155 ceph-mon[47105]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:20:12.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:12 smithi016 ceph-mon[42541]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:20:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:12 smithi189 ceph-mon[46919]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:20:12.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:12 smithi155 ceph-mon[47105]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:20:14.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:14 smithi016 ceph-mon[42541]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-08T23:20:14.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:14 smithi189 ceph-mon[46919]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-08T23:20:14.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:14 smithi155 ceph-mon[47105]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-08T23:20:16.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:16 smithi016 ceph-mon[42541]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:20:16.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:16 smithi189 ceph-mon[46919]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:20:16.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:16 smithi155 ceph-mon[47105]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:20:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:18 smithi016 ceph-mon[42541]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:20:18.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:18 smithi189 ceph-mon[46919]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:20:18.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:18 smithi155 ceph-mon[47105]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-08T23:20:18.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 -- 172.21.15.16:0/2221693928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 msgr2=0x7f96d410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/2221693928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d410d400 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009990 tx=0x7f96c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 -- 172.21.15.16:0/2221693928 shutdown_connections 2024-05-08T23:20:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/2221693928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 0x7f96d410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/2221693928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d4069250 0x7f96d4069630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/2221693928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d410d400 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 -- 172.21.15.16:0/2221693928 >> 172.21.15.16:0/2221693928 conn(0x7f96d4076f30 msgr2=0x7f96d4077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 -- 172.21.15.16:0/2221693928 shutdown_connections 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.106+0000 7f96dbf2e700 1 -- 172.21.15.16:0/2221693928 wait complete. 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 Processor -- start 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 -- start start 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d4069250 0x7f96d4199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 0x7f96d419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d419e450 con 0x7f96d4069b70 2024-05-08T23:20:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d419e5c0 con 0x7f96d410d940 2024-05-08T23:20:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96dbf2e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d419e780 con 0x7f96d4069250 2024-05-08T23:20:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d4199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55986/0 (socket says 172.21.15.16:55986) 2024-05-08T23:20:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 -- 172.21.15.16:0/3665141360 learned_addr learned my addr 172.21.15.16:0/3665141360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d9cca700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d4069250 0x7f96d4199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96da4cb700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 0x7f96d419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 -- 172.21.15.16:0/3665141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d4069250 msgr2=0x7f96d4199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d4069250 0x7f96d4199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 -- 172.21.15.16:0/3665141360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 msgr2=0x7f96d419de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 0x7f96d419de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96d94c9700 1 -- 172.21.15.16:0/3665141360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c40095f0 con 0x7f96d4069b70 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.107+0000 7f96da4cb700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 0x7f96d419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.108+0000 7f96d94c9700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d4199a70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96c4004730 tx=0x7f96c4004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.108+0000 7f96caffd700 1 -- 172.21.15.16:0/3665141360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c401d070 con 0x7f96d4069b70 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.108+0000 7f96caffd700 1 -- 172.21.15.16:0/3665141360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96c4022470 con 0x7f96d4069b70 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.108+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d419ea00 con 0x7f96d4069b70 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.108+0000 7f96caffd700 1 -- 172.21.15.16:0/3665141360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4017650 con 0x7f96d4069b70 2024-05-08T23:20:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.108+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d41a62a0 con 0x7f96d4069b70 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.110+0000 7f96caffd700 1 -- 172.21.15.16:0/3665141360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f96c40225e0 con 0x7f96d4069b70 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.111+0000 7f96caffd700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f96c00a13b0 0x7f96c00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.111+0000 7f96caffd700 1 -- 172.21.15.16:0/3665141360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f96c40ce610 con 0x7f96d4069b70 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.111+0000 7f96d9cca700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f96c00a13b0 0x7f96c00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.111+0000 7f96d9cca700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f96c00a13b0 0x7f96c00a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009c60 tx=0x7f96d0009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.111+0000 7f96e80e8b80 1 Processor -- start 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 -- start start 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b94ee00 0x55d55b94f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b949350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 0x55d55b949cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b8a0150 con 0x55d55b949890 2024-05-08T23:20:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b89db10 con 0x55d55b94f720 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b899400 con 0x55d55b94ee00 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 0x55d55b949cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 0x55d55b949cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56002/0 (socket says 172.21.15.16:56002) 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 -- 172.21.15.16:0/2120394418 learned_addr learned my addr 172.21.15.16:0/2120394418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96d9cca700 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b94ee00 0x55d55b94f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96d94c9700 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b949350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 -- 172.21.15.16:0/2120394418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b94ee00 msgr2=0x55d55b94f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b94ee00 0x55d55b94f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 -- 172.21.15.16:0/2120394418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 msgr2=0x55d55b949350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.112+0000 7f96da4cb700 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b949350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.113+0000 7f96da4cb700 1 -- 172.21.15.16:0/2120394418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d55b71a3b0 con 0x55d55b949890 2024-05-08T23:20:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.113+0000 7f96da4cb700 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 0x55d55b949cd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00e190 tx=0x7f96cc00e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2522e894191ba02c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.113+0000 7f96bffff700 1 -- 172.21.15.16:0/2120394418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc018070 con 0x55d55b949890 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.113+0000 7f96bffff700 1 -- 172.21.15.16:0/2120394418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96cc007c30 con 0x55d55b949890 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.113+0000 7f96bffff700 1 -- 172.21.15.16:0/2120394418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc014c30 con 0x55d55b949890 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 -- 172.21.15.16:0/2120394418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 msgr2=0x55d55b949cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 0x55d55b949cd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00e190 tx=0x7f96cc00e4a0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 -- 172.21.15.16:0/2120394418 shutdown_connections 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b949890 0x55d55b949cd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b94ee00 0x55d55b94f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/2120394418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b949350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 -- 172.21.15.16:0/2120394418 >> 172.21.15.16:0/2120394418 conn(0x55d55b8a7210 msgr2=0x55d55b943df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 -- 172.21.15.16:0/2120394418 shutdown_connections 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 -- 172.21.15.16:0/2120394418 wait complete. 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 -- start start 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.114+0000 7f96e80e8b80 1 Processor -- start 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b952ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b953010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b953550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b835560 con 0x55d55b94ee00 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b835200 con 0x55d55b94f720 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b71c6a0 con 0x55d55b949890 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96e80e8b80 10 client.0 _ll_register_callbacks cb 0x55d55b83eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b953010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b953010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56014/0 (socket says 172.21.15.16:56014) 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 -- 172.21.15.16:0/223773615 learned_addr learned my addr 172.21.15.16:0/223773615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d9cca700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b952ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96da4cb700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b953550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 -- 172.21.15.16:0/223773615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 msgr2=0x55d55b952ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b952ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 -- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 msgr2=0x55d55b953550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b953550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 -- 172.21.15.16:0/223773615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc00f040 con 0x55d55b94ee00 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96d94c9700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b953010 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f96c4097d70 tx=0x7f96c409b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb63063e59bb3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.115+0000 7f96bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96bd7fa700 1 -- 172.21.15.16:0/223773615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40ce440 con 0x55d55b94ee00 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96bd7fa700 1 -- 172.21.15.16:0/223773615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f96c409ba00 con 0x55d55b94ee00 2024-05-08T23:20:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d55b918810 con 0x55d55b94ee00 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96bd7fa700 1 -- 172.21.15.16:0/223773615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96c409b300 con 0x55d55b94ee00 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 10 client.15738 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d55b6c6b60 con 0x55d55b94ee00 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96bd7fa700 1 -- 172.21.15.16:0/223773615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d55b918810 con 0x55d55b94ee00 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96bd7fa700 1 -- 172.21.15.16:0/223773615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96c40cbb50 con 0x55d55b94ee00 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 20 client.15738 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b962df0 0x55d55b965210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 4 client.15738 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.116+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d55b891ec0 con 0x55d55b962df0 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.117+0000 7f96d9cca700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b962df0 0x55d55b965210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.117+0000 7f96d9cca700 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b962df0 0x55d55b965210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26e47ca42a0c860 server_cookie=97a935df6d7d5837 in_seq=0 out_seq=0 2024-05-08T23:20:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.117+0000 7f96bd7fa700 10 client.15738 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96bd7fa700 1 -- 172.21.15.16:0/223773615 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d55b891ec0 con 0x55d55b962df0 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96bd7fa700 10 client.15738 handle_command_reply: tid=0 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 client.15738 shutdown 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 msgr2=0x55d55b953010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b953010 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f96c4097d70 tx=0x7f96c409b840 comp rx=0 tx=0).stop 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 shutdown_connections 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b962df0 0x55d55b965210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b953550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b952ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/223773615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b953010 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96bd7fa700 0 client.15738 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 >> 172.21.15.16:0/223773615 conn(0x55d55b8a7210 msgr2=0x55d55b9435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 shutdown_connections 2024-05-08T23:20:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 1 -- 172.21.15.16:0/223773615 wait complete. 2024-05-08T23:20:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.118+0000 7f96e80e8b80 20 client.15738 trim_cache size 0 max 16384 2024-05-08T23:20:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.136+0000 7f96e80e8b80 1 Processor -- start 2024-05-08T23:20:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.136+0000 7f96e80e8b80 1 -- start start 2024-05-08T23:20:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.136+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b949350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.136+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b94e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.136+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b943d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b96a420 con 0x55d55b94ee00 2024-05-08T23:20:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b955aa0 con 0x55d55b94f720 2024-05-08T23:20:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b95fbe0 con 0x55d55b949890 2024-05-08T23:20:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b94e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b94e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56030/0 (socket says 172.21.15.16:56030) 2024-05-08T23:20:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 -- 172.21.15.16:0/1991126209 learned_addr learned my addr 172.21.15.16:0/1991126209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96da4cb700 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b943d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d9cca700 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b949350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 -- 172.21.15.16:0/1991126209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 msgr2=0x55d55b949350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b949350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 -- 172.21.15.16:0/1991126209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 msgr2=0x55d55b943d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b943d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 -- 172.21.15.16:0/1991126209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d55b85f560 con 0x55d55b94ee00 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96d94c9700 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b94e400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96c4097d20 tx=0x7f96c4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687bda6e949562dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96c8ff9700 1 -- 172.21.15.16:0/1991126209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c409ba00 con 0x55d55b94ee00 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96c8ff9700 1 -- 172.21.15.16:0/1991126209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96c40b0070 con 0x55d55b94ee00 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.137+0000 7f96c8ff9700 1 -- 172.21.15.16:0/1991126209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c409f1c0 con 0x55d55b94ee00 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 -- 172.21.15.16:0/1991126209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 msgr2=0x55d55b94e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b94e400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f96c4097d20 tx=0x7f96c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 -- 172.21.15.16:0/1991126209 shutdown_connections 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94f720 0x55d55b943d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b949350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/1991126209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94ee00 0x55d55b94e400 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 -- 172.21.15.16:0/1991126209 >> 172.21.15.16:0/1991126209 conn(0x55d55b8a7210 msgr2=0x55d55b9513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 -- 172.21.15.16:0/1991126209 shutdown_connections 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 -- 172.21.15.16:0/1991126209 wait complete. 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 -- start start 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.138+0000 7f96e80e8b80 1 Processor -- start 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b95a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94ee00 0x55d55b956760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 0x55d55b956ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b9571e0 con 0x55d55b94f720 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b957350 con 0x55d55b94ee00 2024-05-08T23:20:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d55b957510 con 0x55d55b949890 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96e80e8b80 10 client.0 _ll_register_callbacks cb 0x55d55b83eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 0x55d55b956ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 0x55d55b956ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56044/0 (socket says 172.21.15.16:56044) 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 -- 172.21.15.16:0/3685912836 learned_addr learned my addr 172.21.15.16:0/3685912836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96d94c9700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94ee00 0x55d55b956760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96d9cca700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b95a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 -- 172.21.15.16:0/3685912836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 msgr2=0x55d55b95a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b95a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 -- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94ee00 msgr2=0x55d55b956760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94ee00 0x55d55b956760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96da4cb700 1 -- 172.21.15.16:0/3685912836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d55b6c6b60 con 0x55d55b94f720 2024-05-08T23:20:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.139+0000 7f96d94c9700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94ee00 0x55d55b956760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96da4cb700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 0x55d55b956ca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0078a0 tx=0x7f96cc014880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17eb8ebdf422d71c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96bd7fa700 1 -- 172.21.15.16:0/3685912836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc018040 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96bd7fa700 1 -- 172.21.15.16:0/3685912836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f96cc014df0 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d55b7f2270 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96bd7fa700 1 -- 172.21.15.16:0/3685912836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96cc003a00 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96e80e8b80 10 client.15744 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d55b71ecd0 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96bd7fa700 1 -- 172.21.15.16:0/3685912836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d55b7f2270 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96bd7fa700 1 -- 172.21.15.16:0/3685912836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96cc027440 con 0x55d55b94f720 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96e80e8b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96e80e8b80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.140+0000 7f96e80e8b80 20 client.15744 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b9ba660 0x55d55b9bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96e80e8b80 4 client.15744 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d55b6e7cf0 con 0x55d55b9ba660 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96d9cca700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b9ba660 0x55d55b9bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96bd7fa700 10 client.15744 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96d9cca700 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b9ba660 0x55d55b9bca80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0f748c451f6acd server_cookie=d680ba86e0ac6c8a in_seq=0 out_seq=0 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96bd7fa700 1 -- 172.21.15.16:0/3685912836 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d55b6e7cf0 con 0x55d55b9ba660 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.141+0000 7f96bd7fa700 10 client.15744 handle_command_reply: tid=0 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 client.15744 shutdown 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 msgr2=0x55d55b956ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 0x55d55b956ca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0078a0 tx=0x7f96cc014880 comp rx=0 tx=0).stop 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 shutdown_connections 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d55b94f720 0x55d55b956ca0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d55b949890 0x55d55b95a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d55b9ba660 0x55d55b9bca80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 --2- 172.21.15.16:0/3685912836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d55b94ee00 0x55d55b956760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 >> 172.21.15.16:0/3685912836 conn(0x55d55b8a7210 msgr2=0x55d55b950390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96bd7fa700 0 client.15744 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 shutdown_connections 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 1 -- 172.21.15.16:0/3685912836 wait complete. 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96e80e8b80 20 client.15744 trim_cache size 0 max 16384 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:20:19.153 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:20:19.154 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:20:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f96c00a13b0 msgr2=0x7f96c00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f96c00a13b0 0x7f96c00a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009c60 tx=0x7f96d0009260 comp rx=0 tx=0).stop 2024-05-08T23:20:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 msgr2=0x7f96d4199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d4199a70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f96c4004730 tx=0x7f96c4004760 comp rx=0 tx=0).stop 2024-05-08T23:20:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 shutdown_connections 2024-05-08T23:20:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f96c00a13b0 0x7f96c00a3870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d410d940 0x7f96d419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d4069250 0x7f96d4199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.142+0000 7f96dbf2e700 1 --2- 172.21.15.16:0/3665141360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d4069b70 0x7f96d4199a70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.143+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 >> 172.21.15.16:0/3665141360 conn(0x7f96d4076f30 msgr2=0x7f96d410ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.143+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 shutdown_connections 2024-05-08T23:20:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.143+0000 7f96dbf2e700 1 -- 172.21.15.16:0/3665141360 wait complete. 2024-05-08T23:20:19.156 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:20:19.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:20:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 -- 172.21.15.16:0/4188754916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce010d690 msgr2=0x7efce010fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 --2- 172.21.15.16:0/4188754916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce010d690 0x7efce010fac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b0a0 tx=0x7efcd800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 -- 172.21.15.16:0/4188754916 shutdown_connections 2024-05-08T23:20:19.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 --2- 172.21.15.16:0/4188754916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce010d690 0x7efce010fac0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 --2- 172.21.15.16:0/4188754916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce0073370 0x7efce0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 --2- 172.21.15.16:0/4188754916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce0073c90 0x7efce010d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 -- 172.21.15.16:0/4188754916 >> 172.21.15.16:0/4188754916 conn(0x7efce0078360 msgr2=0x7efce0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.297+0000 7efce4f96700 1 -- 172.21.15.16:0/4188754916 shutdown_connections 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.298+0000 7efce4f96700 1 -- 172.21.15.16:0/4188754916 wait complete. 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.298+0000 7efce4f96700 1 Processor -- start 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.298+0000 7efce4f96700 1 -- start start 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efce4f96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 0x7efce0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efce4f96700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce0073c90 0x7efce01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efce4f96700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce010d690 0x7efce019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efce4f96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019e3a0 con 0x7efce0073370 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efce4f96700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019e510 con 0x7efce010d690 2024-05-08T23:20:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efce4f96700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce019e6d0 con 0x7efce0073c90 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 0x7efce0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 0x7efce0199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56062/0 (socket says 172.21.15.16:56062) 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 -- 172.21.15.16:0/3870741317 learned_addr learned my addr 172.21.15.16:0/3870741317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcddd9b700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce0073c90 0x7efce01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcded9d700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce010d690 0x7efce019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 -- 172.21.15.16:0/3870741317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce0073c90 msgr2=0x7efce01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce0073c90 0x7efce01999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 -- 172.21.15.16:0/3870741317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce010d690 msgr2=0x7efce019dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce010d690 0x7efce019dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 -- 172.21.15.16:0/3870741317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8009d00 con 0x7efce0073370 2024-05-08T23:20:19.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcde59c700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 0x7efce0199480 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c9b0 tx=0x7efcd000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.299+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0007b00 con 0x7efce0073370 2024-05-08T23:20:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.300+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcd0007c60 con 0x7efce0073370 2024-05-08T23:20:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.300+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0018660 con 0x7efce0073370 2024-05-08T23:20:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.300+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce019e9b0 con 0x7efce0073370 2024-05-08T23:20:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.300+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce01a61f0 con 0x7efce0073370 2024-05-08T23:20:19.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.301+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce01088a0 con 0x7efce0073370 2024-05-08T23:20:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.302+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7efcd000f450 con 0x7efce0073370 2024-05-08T23:20:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.302+0000 7efcd77fe700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efccc0a13b0 0x7efccc0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.303+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7efcd00cd580 con 0x7efce0073370 2024-05-08T23:20:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.303+0000 7efcddd9b700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efccc0a13b0 0x7efccc0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.303+0000 7efcddd9b700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efccc0a13b0 0x7efccc0a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efce019a9d0 tx=0x7efcc800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.307+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efcd0096500 con 0x7efce0073370 2024-05-08T23:20:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.575+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efce019a780 con 0x7efce0073370 2024-05-08T23:20:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.575+0000 7efcd77fe700 1 -- 172.21.15.16:0/3870741317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7efcd000fe60 con 0x7efce0073370 2024-05-08T23:20:19.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:20:19.575 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:20:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efccc0a13b0 msgr2=0x7efccc0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efccc0a13b0 0x7efccc0a3870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efce019a9d0 tx=0x7efcc800b410 comp rx=0 tx=0).stop 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 msgr2=0x7efce0199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 0x7efce0199480 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c9b0 tx=0x7efcd000ccc0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 shutdown_connections 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efccc0a13b0 0x7efccc0a3870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efce010d690 0x7efce019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efce0073370 0x7efce0199480 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 --2- 172.21.15.16:0/3870741317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce0073c90 0x7efce01999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 >> 172.21.15.16:0/3870741317 conn(0x7efce0078360 msgr2=0x7efce00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 shutdown_connections 2024-05-08T23:20:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.578+0000 7efce4f96700 1 -- 172.21.15.16:0/3870741317 wait complete. 2024-05-08T23:20:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:20:19.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:20:19.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 -- 172.21.15.16:0/1874096127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de01011e0 msgr2=0x7f4de01015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 --2- 172.21.15.16:0/1874096127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de01011e0 0x7f4de01015c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009990 tx=0x7f4dd0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 -- 172.21.15.16:0/1874096127 shutdown_connections 2024-05-08T23:20:19.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 --2- 172.21.15.16:0/1874096127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de0106300 0x7f4de010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 --2- 172.21.15.16:0/1874096127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de01011e0 0x7f4de01015c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 --2- 172.21.15.16:0/1874096127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4de0101b90 0x7f4de0105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 -- 172.21.15.16:0/1874096127 >> 172.21.15.16:0/1874096127 conn(0x7f4de00fca80 msgr2=0x7f4de00feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 -- 172.21.15.16:0/1874096127 shutdown_connections 2024-05-08T23:20:19.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.733+0000 7f4de7e50700 1 -- 172.21.15.16:0/1874096127 wait complete. 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 Processor -- start 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 -- start start 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 0x7f4de01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4de0106300 0x7f4de0197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 0x7f4de019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019c630 con 0x7f4de019bbb0 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019c7a0 con 0x7f4de0106300 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de7e50700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019c960 con 0x7f4de01011e0 2024-05-08T23:20:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 0x7f4de019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.734+0000 7f4de5bec700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 0x7f4de01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 0x7f4de01972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:39186/0 (socket says 172.21.15.16:39186) 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 0x7f4de019c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56072/0 (socket says 172.21.15.16:56072) 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 -- 172.21.15.16:0/3434863478 learned_addr learned my addr 172.21.15.16:0/3434863478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de53eb700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4de0106300 0x7f4de0197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 -- 172.21.15.16:0/3434863478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4de0106300 msgr2=0x7f4de0197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4de0106300 0x7f4de0197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 -- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 msgr2=0x7f4de019c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 0x7f4de019c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 -- 172.21.15.16:0/3434863478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00095f0 con 0x7f4de01011e0 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de63ed700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 0x7f4de019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de5bec700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 0x7f4de01972d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00059d0 tx=0x7f4dd0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4dd6ffd700 1 -- 172.21.15.16:0/3434863478 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd001d070 con 0x7f4de01011e0 2024-05-08T23:20:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4dd6ffd700 1 -- 172.21.15.16:0/3434863478 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd000fb00 con 0x7f4de01011e0 2024-05-08T23:20:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de01a3e60 con 0x7f4de01011e0 2024-05-08T23:20:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4dd6ffd700 1 -- 172.21.15.16:0/3434863478 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0022a40 con 0x7f4de01011e0 2024-05-08T23:20:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.735+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de01a4280 con 0x7f4de01011e0 2024-05-08T23:20:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.738+0000 7f4dd6ffd700 1 -- 172.21.15.16:0/3434863478 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f4dd000fc70 con 0x7f4de01011e0 2024-05-08T23:20:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.738+0000 7f4dd6ffd700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4dcc0a1430 0x7f4dcc0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.738+0000 7f4de53eb700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4dcc0a1430 0x7f4dcc0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.738+0000 7f4dd6ffd700 1 -- 172.21.15.16:0/3434863478 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f4dd00ce430 con 0x7f4de01011e0 2024-05-08T23:20:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.739+0000 7f4de53eb700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4dcc0a1430 0x7f4dcc0a38f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4de0198820 tx=0x7f4ddc00a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.739+0000 7f4df400ab80 1 Processor -- start 2024-05-08T23:20:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.739+0000 7f4df400ab80 1 -- start start 2024-05-08T23:20:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.739+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4e9e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9e720 0x564df4e98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4df400ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4def150 con 0x564df4e98890 2024-05-08T23:20:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4df400ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4decb10 con 0x564df4e9de00 2024-05-08T23:20:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4df400ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4de8400 con 0x564df4e9e720 2024-05-08T23:20:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56076/0 (socket says 172.21.15.16:56076) 2024-05-08T23:20:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 -- 172.21.15.16:0/640175247 learned_addr learned my addr 172.21.15.16:0/640175247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de5bec700 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4e9e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de53eb700 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9e720 0x564df4e98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 -- 172.21.15.16:0/640175247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9e720 msgr2=0x564df4e98350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9e720 0x564df4e98350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 -- 172.21.15.16:0/640175247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 msgr2=0x564df4e9e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4e9e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 -- 172.21.15.16:0/640175247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564df4c693b0 con 0x564df4e98890 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de53eb700 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9e720 0x564df4e98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4de63ed700 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98cd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8007740 tx=0x7f4dd8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94009f0799913206 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.740+0000 7f4dcbfff700 1 -- 172.21.15.16:0/640175247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8015850 con 0x564df4e98890 2024-05-08T23:20:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4dcbfff700 1 -- 172.21.15.16:0/640175247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd800f040 con 0x564df4e98890 2024-05-08T23:20:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4dcbfff700 1 -- 172.21.15.16:0/640175247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8014cc0 con 0x564df4e98890 2024-05-08T23:20:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 -- 172.21.15.16:0/640175247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 msgr2=0x564df4e98cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98cd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8007740 tx=0x7f4dd8009f90 comp rx=0 tx=0).stop 2024-05-08T23:20:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 -- 172.21.15.16:0/640175247 shutdown_connections 2024-05-08T23:20:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98cd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4e9e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 --2- 172.21.15.16:0/640175247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9e720 0x564df4e98350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 -- 172.21.15.16:0/640175247 >> 172.21.15.16:0/640175247 conn(0x564df4df6210 msgr2=0x564df4e92df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 -- 172.21.15.16:0/640175247 shutdown_connections 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.741+0000 7f4df400ab80 1 -- 172.21.15.16:0/640175247 wait complete. 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 -- start start 2024-05-08T23:20:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 Processor -- start 2024-05-08T23:20:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e98890 0x564df4ea1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4ea2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4d84560 con 0x564df4e9e720 2024-05-08T23:20:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4d84200 con 0x564df4e98890 2024-05-08T23:20:19.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4c6b6a0 con 0x564df4e9de00 2024-05-08T23:20:19.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4df400ab80 10 client.0 _ll_register_callbacks cb 0x564df4d8db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:19.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.742+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56092/0 (socket says 172.21.15.16:56092) 2024-05-08T23:20:19.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 -- 172.21.15.16:0/2560327655 learned_addr learned my addr 172.21.15.16:0/2560327655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de5bec700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e98890 0x564df4ea1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de53eb700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4ea2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 -- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 msgr2=0x564df4ea2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4ea2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 -- 172.21.15.16:0/2560327655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e98890 msgr2=0x564df4ea1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e98890 0x564df4ea1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 -- 172.21.15.16:0/2560327655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd8009e00 con 0x564df4e9e720 2024-05-08T23:20:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de53eb700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4ea2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de5bec700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e98890 0x564df4ea1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4de63ed700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea2580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8004fd0 tx=0x7f4dd8014af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3695ae60df80c654 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4dc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4dc97fa700 1 -- 172.21.15.16:0/2560327655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8015850 con 0x564df4e9e720 2024-05-08T23:20:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4dc97fa700 1 -- 172.21.15.16:0/2560327655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f4dd80143a0 con 0x564df4e9e720 2024-05-08T23:20:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564df4d41270 con 0x564df4e9e720 2024-05-08T23:20:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4dc97fa700 1 -- 172.21.15.16:0/2560327655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd80146a0 con 0x564df4e9e720 2024-05-08T23:20:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4df400ab80 10 client.15762 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.743+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564df4c15b60 con 0x564df4e9e720 2024-05-08T23:20:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4dc97fa700 1 -- 172.21.15.16:0/2560327655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564df4d41270 con 0x564df4e9e720 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4dc97fa700 1 -- 172.21.15.16:0/2560327655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4dd801ea00 con 0x564df4e9e720 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4df400ab80 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4df400ab80 10 client.15762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4df400ab80 20 client.15762 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4df400ab80 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4eb20a0 0x564df4eb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4df400ab80 4 client.15762 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564df4e67810 con 0x564df4eb20a0 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.744+0000 7f4de5bec700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4eb20a0 0x564df4eb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.745+0000 7f4dc97fa700 10 client.15762 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.745+0000 7f4de5bec700 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4eb20a0 0x564df4eb44c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b39b53ac66dacc3 server_cookie=eec88e0f26ae8467 in_seq=0 out_seq=0 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.745+0000 7f4dc97fa700 1 -- 172.21.15.16:0/2560327655 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564df4e67810 con 0x564df4eb20a0 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.745+0000 7f4dc97fa700 10 client.15762 handle_command_reply: tid=0 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.745+0000 7f4df400ab80 1 client.15762 shutdown 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 msgr2=0x564df4ea2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea2580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8004fd0 tx=0x7f4dd8014af0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 shutdown_connections 2024-05-08T23:20:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea2580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4eb20a0 0x564df4eb44c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e98890 0x564df4ea1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 --2- 172.21.15.16:0/2560327655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4ea2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4dc97fa700 0 client.15762 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 >> 172.21.15.16:0/2560327655 conn(0x564df4df6210 msgr2=0x564df4e925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 shutdown_connections 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 1 -- 172.21.15.16:0/2560327655 wait complete. 2024-05-08T23:20:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.746+0000 7f4df400ab80 20 client.15762 trim_cache size 0 max 16384 2024-05-08T23:20:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 Processor -- start 2024-05-08T23:20:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 -- start start 2024-05-08T23:20:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4eb4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9e720 0x564df4e92d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4ea3100 con 0x564df4e98890 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4e94e50 con 0x564df4e9e720 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.760+0000 7f4df400ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4e9e560 con 0x564df4e9de00 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56100/0 (socket says 172.21.15.16:56100) 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 -- 172.21.15.16:0/268597929 learned_addr learned my addr 172.21.15.16:0/268597929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de53eb700 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4eb4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de63ed700 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9e720 0x564df4e92d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 -- 172.21.15.16:0/268597929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 msgr2=0x564df4eb4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4eb4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 -- 172.21.15.16:0/268597929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9e720 msgr2=0x564df4e92d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9e720 0x564df4e92d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 -- 172.21.15.16:0/268597929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564df4dae560 con 0x564df4e98890 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4de5bec700 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98350 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4dd009b1f0 tx=0x7f4dd00a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5c78bbc2863de3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4dd4ff9700 1 -- 172.21.15.16:0/268597929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd009b960 con 0x564df4e98890 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4dd4ff9700 1 -- 172.21.15.16:0/268597929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd009b260 con 0x564df4e98890 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.761+0000 7f4dd4ff9700 1 -- 172.21.15.16:0/268597929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0099720 con 0x564df4e98890 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 -- 172.21.15.16:0/268597929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 msgr2=0x564df4e98350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98350 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4dd009b1f0 tx=0x7f4dd00a1040 comp rx=0 tx=0).stop 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 -- 172.21.15.16:0/268597929 shutdown_connections 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9e720 0x564df4e92d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e98890 0x564df4e98350 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 --2- 172.21.15.16:0/268597929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e9de00 0x564df4eb4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 -- 172.21.15.16:0/268597929 >> 172.21.15.16:0/268597929 conn(0x564df4df6210 msgr2=0x564df4e98c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 -- 172.21.15.16:0/268597929 shutdown_connections 2024-05-08T23:20:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 -- 172.21.15.16:0/268597929 wait complete. 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 -- start start 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.762+0000 7f4df400ab80 1 Processor -- start 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e98890 0x564df4ea91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4ea96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4ea7750 con 0x564df4e9e720 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4ea78c0 con 0x564df4e9de00 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564df4ea7a80 con 0x564df4e98890 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4df400ab80 10 client.0 _ll_register_callbacks cb 0x564df4d8db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56114/0 (socket says 172.21.15.16:56114) 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 -- 172.21.15.16:0/4042671819 learned_addr learned my addr 172.21.15.16:0/4042671819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de53eb700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4ea96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de5bec700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e98890 0x564df4ea91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 -- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e98890 msgr2=0x564df4ea91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e98890 0x564df4ea91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 -- 172.21.15.16:0/4042671819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 msgr2=0x564df4ea96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4ea96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.763+0000 7f4de63ed700 1 -- 172.21.15.16:0/4042671819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd0095e70 con 0x564df4e9e720 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4de63ed700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea7180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4dd801ede0 tx=0x7f4dd801ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b13e6a942ca2965 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4dc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4dc97fa700 1 -- 172.21.15.16:0/4042671819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8005090 con 0x564df4e9e720 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4dc97fa700 1 -- 172.21.15.16:0/4042671819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f4dd80143a0 con 0x564df4e9e720 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dd800f040 con 0x564df4e9e720 2024-05-08T23:20:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4dc97fa700 1 -- 172.21.15.16:0/4042671819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd8014c10 con 0x564df4e9e720 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 10 client.15768 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564df4c6dcd0 con 0x564df4e9e720 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4dc97fa700 1 -- 172.21.15.16:0/4042671819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd800f040 con 0x564df4e9e720 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4dc97fa700 1 -- 172.21.15.16:0/4042671819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4dd801e880 con 0x564df4e9e720 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 20 client.15768 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4f082d0 0x564df4f0a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 4 client.15768 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.764+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564df4de0ec0 con 0x564df4f082d0 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4de5bec700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4f082d0 0x564df4f0a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4dc97fa700 10 client.15768 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4de5bec700 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4f082d0 0x564df4f0a6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69570199f8f9f8d0 server_cookie=dc8ac517c2fbb3a6 in_seq=0 out_seq=0 2024-05-08T23:20:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4dc97fa700 1 -- 172.21.15.16:0/4042671819 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564df4de0ec0 con 0x564df4f082d0 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4dc97fa700 10 client.15768 handle_command_reply: tid=0 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 client.15768 shutdown 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 msgr2=0x564df4ea7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea7180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4dd801ede0 tx=0x7f4dd801ef00 comp rx=0 tx=0).stop 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 shutdown_connections 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564df4e9e720 0x564df4ea7180 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564df4e98890 0x564df4ea91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x564df4f082d0 0x564df4f0a6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 --2- 172.21.15.16:0/4042671819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564df4e9de00 0x564df4ea96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4dc97fa700 0 client.15768 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.765+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 >> 172.21.15.16:0/4042671819 conn(0x564df4df6210 msgr2=0x564df4ea0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 shutdown_connections 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4df400ab80 1 -- 172.21.15.16:0/4042671819 wait complete. 2024-05-08T23:20:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4df400ab80 20 client.15768 trim_cache size 0 max 16384 2024-05-08T23:20:19.774 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4dcc0a1430 msgr2=0x7f4dcc0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4dcc0a1430 0x7f4dcc0a38f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4de0198820 tx=0x7f4ddc00a380 comp rx=0 tx=0).stop 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 msgr2=0x7f4de01972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 0x7f4de01972d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00059d0 tx=0x7f4dd0004730 comp rx=0 tx=0).stop 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 shutdown_connections 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4dcc0a1430 0x7f4dcc0a38f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4de019bbb0 0x7f4de019c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4de01011e0 0x7f4de01972d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 --2- 172.21.15.16:0/3434863478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4de0106300 0x7f4de0197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 >> 172.21.15.16:0/3434863478 conn(0x7f4de00fca80 msgr2=0x7f4de00fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 shutdown_connections 2024-05-08T23:20:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.766+0000 7f4de7e50700 1 -- 172.21.15.16:0/3434863478 wait complete. 2024-05-08T23:20:19.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:20:19.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:19 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3870741317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:19.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:19 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3870741317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/1582661582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0105170 msgr2=0x7fe4e010ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/1582661582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0105170 0x7fe4e010ccf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b0a0 tx=0x7fe4d800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/1582661582 shutdown_connections 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/1582661582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0105170 0x7fe4e010ccf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/1582661582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e0100060 0x7fe4e0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/1582661582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0100a10 0x7fe4e0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/1582661582 >> 172.21.15.16:0/1582661582 conn(0x7fe4e0076f30 msgr2=0x7fe4e0077340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/1582661582 shutdown_connections 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.919+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/1582661582 wait complete. 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.920+0000 7fe4e4ccf700 1 Processor -- start 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.920+0000 7fe4e4ccf700 1 -- start start 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.920+0000 7fe4e4ccf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 0x7fe4e0101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.920+0000 7fe4e4ccf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e0105170 0x7fe4e0102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4e4ccf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0102560 0x7fe4e01a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4e4ccf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01a5c20 con 0x7fe4e0100060 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4e4ccf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01a5d90 con 0x7fe4e0105170 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4e4ccf700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01a5f50 con 0x7fe4e0102560 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 0x7fe4e0101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 0x7fe4e0101ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56144/0 (socket says 172.21.15.16:56144) 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 -- 172.21.15.16:0/2226420388 learned_addr learned my addr 172.21.15.16:0/2226420388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0102560 0x7fe4e01a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e0105170 0x7fe4e0102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 -- 172.21.15.16:0/2226420388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0102560 msgr2=0x7fe4e01a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0102560 0x7fe4e01a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 -- 172.21.15.16:0/2226420388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e0105170 msgr2=0x7fe4e0102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e0105170 0x7fe4e0102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 -- 172.21.15.16:0/2226420388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d8009d00 con 0x7fe4e0100060 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 0x7fe4e0101ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400d950 tx=0x7fe4d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4cf7fe700 1 -- 172.21.15.16:0/2226420388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4009940 con 0x7fe4e0100060 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4cf7fe700 1 -- 172.21.15.16:0/2226420388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d4010460 con 0x7fe4e0100060 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e01a6230 con 0x7fe4e0100060 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4cf7fe700 1 -- 172.21.15.16:0/2226420388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400f620 con 0x7fe4e0100060 2024-05-08T23:20:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.921+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e01a66f0 con 0x7fe4e0100060 2024-05-08T23:20:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.924+0000 7fe4cf7fe700 1 -- 172.21.15.16:0/2226420388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fe4d40105d0 con 0x7fe4e0100060 2024-05-08T23:20:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.924+0000 7fe4cf7fe700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe4c80a13b0 0x7fe4c80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.925+0000 7fe4cf7fe700 1 -- 172.21.15.16:0/2226420388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fe4d40cd120 con 0x7fe4e0100060 2024-05-08T23:20:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.925+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe4c80a13b0 0x7fe4c80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.925+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe4c80a13b0 0x7fe4c80a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005950 tx=0x7fe4d00058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.925+0000 7fe4f0e89b80 1 Processor -- start 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 -- start start 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b75908150 con 0x555b759b6e00 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b75905b10 con 0x555b759b7720 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b75901400 con 0x555b759b1890 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56146/0 (socket says 172.21.15.16:56146) 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 -- 172.21.15.16:0/284536164 learned_addr learned my addr 172.21.15.16:0/284536164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 -- 172.21.15.16:0/284536164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 msgr2=0x555b759b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 -- 172.21.15.16:0/284536164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 msgr2=0x555b759b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4de59c700 1 -- 172.21.15.16:0/284536164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b757823b0 con 0x555b759b6e00 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.926+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.927+0000 7fe4de59c700 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759b71e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40961d0 tx=0x7fe4d4099630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d788260778d4423d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.927+0000 7fe4ccff9700 1 -- 172.21.15.16:0/284536164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d409a3d0 con 0x555b759b6e00 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.927+0000 7fe4ccff9700 1 -- 172.21.15.16:0/284536164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d4099cd0 con 0x555b759b6e00 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.927+0000 7fe4ccff9700 1 -- 172.21.15.16:0/284536164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40a1f20 con 0x555b759b6e00 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/284536164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 msgr2=0x555b759b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759b71e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40961d0 tx=0x7fe4d4099630 comp rx=0 tx=0).stop 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/284536164 shutdown_connections 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759b71e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/284536164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/284536164 >> 172.21.15.16:0/284536164 conn(0x555b7590f210 msgr2=0x555b759abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/284536164 shutdown_connections 2024-05-08T23:20:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/284536164 wait complete. 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 -- start start 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 Processor -- start 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.928+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7589d560 con 0x555b759b6e00 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b7589d200 con 0x555b759b7720 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b757846a0 con 0x555b759b1890 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4f0e89b80 10 client.0 _ll_register_callbacks cb 0x555b758a6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759baab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56154/0 (socket says 172.21.15.16:56154) 2024-05-08T23:20:19.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2260204271 learned_addr learned my addr 172.21.15.16:0/2260204271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 msgr2=0x555b759bd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759bd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2260204271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 msgr2=0x555b759bb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759bb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2260204271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e01a66f0 con 0x555b759b6e00 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759bb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759baab0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0103210 tx=0x7fe4d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76eea1b9fdbeb001 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/2260204271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001f630 con 0x555b759b6e00 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/2260204271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fe4d0011e40 con 0x555b759b6e00 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.929+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b7585a270 con 0x555b759b6e00 2024-05-08T23:20:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/2260204271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d001f790 con 0x555b759b6e00 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 10 client.15780 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555b7572eb60 con 0x555b759b6e00 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/2260204271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555b7585a270 con 0x555b759b6e00 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/2260204271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4d0025020 con 0x555b759b6e00 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 10 client.15780 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 20 client.15780 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b759cb240 0x555b759cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 4 client.15780 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b75980810 con 0x555b759cb240 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.930+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b759cb240 0x555b759cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.931+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b759cb240 0x555b759cd660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4440b1db55b7c6e9 server_cookie=5937f6716d4d025 in_seq=0 out_seq=0 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.931+0000 7fe4c5ffb700 10 client.15780 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/2260204271 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b75980810 con 0x555b759cb240 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4c5ffb700 10 client.15780 handle_command_reply: tid=0 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 client.15780 shutdown 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 msgr2=0x555b759baab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759baab0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0103210 tx=0x7fe4d0004750 comp rx=0 tx=0).stop 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 shutdown_connections 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b759cb240 0x555b759cd660 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759bb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b1890 0x555b759bd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4c5ffb700 0 client.15780 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2260204271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b6e00 0x555b759baab0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 >> 172.21.15.16:0/2260204271 conn(0x555b7590f210 msgr2=0x555b759ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 shutdown_connections 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2260204271 wait complete. 2024-05-08T23:20:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.932+0000 7fe4f0e89b80 20 client.15780 trim_cache size 0 max 16384 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 Processor -- start 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 -- start start 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 0x555b759b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b759ac8e0 con 0x555b759b1890 2024-05-08T23:20:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b759c88c0 con 0x555b759b7720 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.946+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b759bbf40 con 0x555b759b6e00 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 0x555b759b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 0x555b759b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56164/0 (socket says 172.21.15.16:56164) 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4de59c700 1 -- 172.21.15.16:0/2370578541 learned_addr learned my addr 172.21.15.16:0/2370578541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2370578541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 msgr2=0x555b759abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2370578541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 msgr2=0x555b759b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 0x555b759b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 -- 172.21.15.16:0/2370578541 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b758c7560 con 0x555b759b6e00 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4de59c700 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 0x555b759b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759bb190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0010850 tx=0x7fe4d000b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=229e721c1631f6e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4cd7fa700 1 -- 172.21.15.16:0/2370578541 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0011ab0 con 0x555b759b6e00 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4cd7fa700 1 -- 172.21.15.16:0/2370578541 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d0005c60 con 0x555b759b6e00 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.947+0000 7fe4cd7fa700 1 -- 172.21.15.16:0/2370578541 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0019e90 con 0x555b759b6e00 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2370578541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 msgr2=0x555b759bb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759bb190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0010850 tx=0x7fe4d000b3c0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2370578541 shutdown_connections 2024-05-08T23:20:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b7720 0x555b759abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b1890 0x555b759b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/2370578541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759bb190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2370578541 >> 172.21.15.16:0/2370578541 conn(0x555b7590f210 msgr2=0x555b759b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2370578541 shutdown_connections 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/2370578541 wait complete. 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.948+0000 7fe4f0e89b80 1 -- start start 2024-05-08T23:20:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 Processor -- start 2024-05-08T23:20:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 0x555b759c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 0x555b759be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b759be9f0 con 0x555b759b7720 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b759beb60 con 0x555b759b1890 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b759bed20 con 0x555b759b6e00 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4f0e89b80 10 client.0 _ll_register_callbacks cb 0x555b758a6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4ded9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 0x555b759be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 0x555b759c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4ded9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 0x555b759be2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56176/0 (socket says 172.21.15.16:56176) 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4de59c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 0x555b759c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:59098/0 (socket says 172.21.15.16:59098) 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4ded9d700 1 -- 172.21.15.16:0/1513725932 learned_addr learned my addr 172.21.15.16:0/1513725932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.949+0000 7fe4ddd9b700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4ded9d700 1 -- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 msgr2=0x555b759c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4ded9d700 1 -- 172.21.15.16:0/1513725932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 msgr2=0x555b759c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 0x555b759c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4ded9d700 1 -- 172.21.15.16:0/1513725932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b757823b0 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4de59c700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 0x555b759c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4ded9d700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 0x555b759be2e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b0a0 tx=0x7fe4d80074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dabacf96d240cb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/1513725932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800e040 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/1513725932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fe4d8003a00 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d4099cd0 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/1513725932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d8007920 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4f0e89b80 10 client.15786 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555b7572eb60 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.950+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/1513725932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4d4099cd0 con 0x555b759b7720 2024-05-08T23:20:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/1513725932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4d800b0d0 con 0x555b759b7720 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4f0e89b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4f0e89b80 10 client.15786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4f0e89b80 20 client.15786 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b75a21bb0 0x555b75a23fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4f0e89b80 4 client.15786 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555b758f9ec0 con 0x555b75a21bb0 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.951+0000 7fe4de59c700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b75a21bb0 0x555b75a23fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4de59c700 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b75a21bb0 0x555b75a23fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3eba435b72500e97 server_cookie=bad3a8ab508150e7 in_seq=0 out_seq=0 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4c5ffb700 10 client.15786 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4c5ffb700 1 -- 172.21.15.16:0/1513725932 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555b758f9ec0 con 0x555b75a21bb0 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4c5ffb700 10 client.15786 handle_command_reply: tid=0 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 client.15786 shutdown 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 msgr2=0x555b759be2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 0x555b759be2e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b0a0 tx=0x7fe4d80074e0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 shutdown_connections 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b759b7720 0x555b759be2e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555b75a21bb0 0x555b75a23fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555b759b1890 0x555b759c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 --2- 172.21.15.16:0/1513725932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b759b6e00 0x555b759c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4c5ffb700 0 client.15786 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 >> 172.21.15.16:0/1513725932 conn(0x555b7590f210 msgr2=0x555b759b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 shutdown_connections 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 1 -- 172.21.15.16:0/1513725932 wait complete. 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.952+0000 7fe4f0e89b80 20 client.15786 trim_cache size 0 max 16384 2024-05-08T23:20:19.958 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe4c80a13b0 msgr2=0x7fe4c80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe4c80a13b0 0x7fe4c80a3870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005950 tx=0x7fe4d00058e0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 msgr2=0x7fe4e0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 0x7fe4e0101ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400d950 tx=0x7fe4d400dc60 comp rx=0 tx=0).stop 2024-05-08T23:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 shutdown_connections 2024-05-08T23:20:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe4c80a13b0 0x7fe4c80a3870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4e0102560 0x7fe4e01a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0100060 0x7fe4e0101ae0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 --2- 172.21.15.16:0/2226420388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e0105170 0x7fe4e0102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 >> 172.21.15.16:0/2226420388 conn(0x7fe4e0076f30 msgr2=0x7fe4e00695d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 shutdown_connections 2024-05-08T23:20:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:19.953+0000 7fe4e4ccf700 1 -- 172.21.15.16:0/2226420388 wait complete. 2024-05-08T23:20:19.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:19 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3870741317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:19.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:20:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 -- 172.21.15.16:0/2810611634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b81011e0 msgr2=0x7fd7b81015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2810611634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b81011e0 0x7fd7b81015c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009990 tx=0x7fd7a8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 -- 172.21.15.16:0/2810611634 shutdown_connections 2024-05-08T23:20:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2810611634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b8106300 0x7fd7b810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2810611634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b81011e0 0x7fd7b81015c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2810611634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7b8101b90 0x7fd7b8105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 -- 172.21.15.16:0/2810611634 >> 172.21.15.16:0/2810611634 conn(0x7fd7b80fca80 msgr2=0x7fd7b80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 -- 172.21.15.16:0/2810611634 shutdown_connections 2024-05-08T23:20:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.106+0000 7fd7bca26700 1 -- 172.21.15.16:0/2810611634 wait complete. 2024-05-08T23:20:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.107+0000 7fd7bca26700 1 Processor -- start 2024-05-08T23:20:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.107+0000 7fd7bca26700 1 -- start start 2024-05-08T23:20:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.107+0000 7fd7bca26700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7b81011e0 0x7fd7b8197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7bca26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 0x7fd7b8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7bca26700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b819bcb0 0x7fd7b819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7bca26700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b819c730 con 0x7fd7b8106300 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7bca26700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b819c8a0 con 0x7fd7b819bcb0 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7bca26700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b819ca60 con 0x7fd7b81011e0 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 0x7fd7b8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 0x7fd7b8197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56180/0 (socket says 172.21.15.16:56180) 2024-05-08T23:20:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 -- 172.21.15.16:0/2114072133 learned_addr learned my addr 172.21.15.16:0/2114072133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b659c700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7b81011e0 0x7fd7b8197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b6d9d700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b819bcb0 0x7fd7b819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 -- 172.21.15.16:0/2114072133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7b81011e0 msgr2=0x7fd7b8197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7b81011e0 0x7fd7b8197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 -- 172.21.15.16:0/2114072133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b819bcb0 msgr2=0x7fd7b819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b819bcb0 0x7fd7b819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 -- 172.21.15.16:0/2114072133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a80095f0 con 0x7fd7b8106300 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 0x7fd7b8197960 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00e970 tx=0x7fd7ac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7a77fe700 1 -- 172.21.15.16:0/2114072133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac009790 con 0x7fd7b8106300 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b81a3fc0 con 0x7fd7b8106300 2024-05-08T23:20:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.108+0000 7fd7a77fe700 1 -- 172.21.15.16:0/2114072133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7ac00eec0 con 0x7fd7b8106300 2024-05-08T23:20:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.109+0000 7fd7a77fe700 1 -- 172.21.15.16:0/2114072133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0054b0 con 0x7fd7b8106300 2024-05-08T23:20:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.109+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b81a4440 con 0x7fd7b8106300 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.111+0000 7fd7a77fe700 1 -- 172.21.15.16:0/2114072133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fd7ac004750 con 0x7fd7b8106300 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.111+0000 7fd7a77fe700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd7a00a13b0 0x7fd7a00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.112+0000 7fd7b659c700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd7a00a13b0 0x7fd7a00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.112+0000 7fd7a77fe700 1 -- 172.21.15.16:0/2114072133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fd7ac014070 con 0x7fd7b8106300 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.112+0000 7fd7b659c700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd7a00a13b0 0x7fd7a00a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8006010 tx=0x7fd7a800b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.112+0000 7fd7c8be0b80 1 Processor -- start 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 -- start start 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 0x562041c085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c02ca0 0x562041c03100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041b55150 con 0x562041c081f0 2024-05-08T23:20:20.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041b52b10 con 0x562041c08b10 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041b4e400 con 0x562041c02ca0 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 0x562041c085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 0x562041c085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56190/0 (socket says 172.21.15.16:56190) 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 -- 172.21.15.16:0/804565610 learned_addr learned my addr 172.21.15.16:0/804565610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b6d9d700 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c02ca0 0x562041c03100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 -- 172.21.15.16:0/804565610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c02ca0 msgr2=0x562041c03100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c02ca0 0x562041c03100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 -- 172.21.15.16:0/804565610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 msgr2=0x562041c02760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c02760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.113+0000 7fd7b659c700 1 -- 172.21.15.16:0/804565610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620419cf3b0 con 0x562041c081f0 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.114+0000 7fd7b659c700 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 0x562041c085d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8005760 tx=0x7fd7a8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54419becedc2b688 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.114+0000 7fd7a4ff9700 1 -- 172.21.15.16:0/804565610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004500 con 0x562041c081f0 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.114+0000 7fd7a4ff9700 1 -- 172.21.15.16:0/804565610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a801e040 con 0x562041c081f0 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.114+0000 7fd7a4ff9700 1 -- 172.21.15.16:0/804565610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800f750 con 0x562041c081f0 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.114+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/804565610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 msgr2=0x562041c085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.114+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 0x562041c085d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8005760 tx=0x7fd7a8020000 comp rx=0 tx=0).stop 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/804565610 shutdown_connections 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c02ca0 0x562041c03100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c081f0 0x562041c085d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/804565610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/804565610 >> 172.21.15.16:0/804565610 conn(0x562041b5c210 msgr2=0x562041bf8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/804565610 shutdown_connections 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/804565610 wait complete. 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 -- start start 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 Processor -- start 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041c0c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.115+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c0c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041aea560 con 0x562041c02ca0 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041aea200 con 0x562041c08b10 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620419d16a0 con 0x562041c081f0 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7c8be0b80 10 client.0 _ll_register_callbacks cb 0x562041af3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56196/0 (socket says 172.21.15.16:56196) 2024-05-08T23:20:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 -- 172.21.15.16:0/3086925110 learned_addr learned my addr 172.21.15.16:0/3086925110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041c0c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 -- 172.21.15.16:0/3086925110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 msgr2=0x562041c0c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041c0c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 -- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 msgr2=0x562041c0c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c0c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 -- 172.21.15.16:0/3086925110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b810e6f0 con 0x562041c02ca0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041c0c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7b659c700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0be30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8000c00 tx=0x7fd7a800f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01835e48a96e77e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd79dffb700 1 -- 172.21.15.16:0/3086925110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004700 con 0x562041c02ca0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd79dffb700 1 -- 172.21.15.16:0/3086925110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fd7a8004860 con 0x562041c02ca0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562041aa7270 con 0x562041c02ca0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.116+0000 7fd79dffb700 1 -- 172.21.15.16:0/3086925110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a8004a20 con 0x562041c02ca0 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 10 client.15798 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56204197bb60 con 0x562041c02ca0 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd79dffb700 1 -- 172.21.15.16:0/3086925110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562041aa7270 con 0x562041c02ca0 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd79dffb700 1 -- 172.21.15.16:0/3086925110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7a800fd20 con 0x562041c02ca0 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 20 client.15798 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c1c070 0x562041c1e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 4 client.15798 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562041bcd810 con 0x562041c1c070 2024-05-08T23:20:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.117+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c1c070 0x562041c1e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.118+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c1c070 0x562041c1e460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1724cc49852d0323 server_cookie=cc9c8a434003a2bc in_seq=0 out_seq=0 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.118+0000 7fd79dffb700 10 client.15798 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.118+0000 7fd79dffb700 1 -- 172.21.15.16:0/3086925110 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562041bcd810 con 0x562041c1c070 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.118+0000 7fd79dffb700 10 client.15798 handle_command_reply: tid=0 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.118+0000 7fd7c8be0b80 1 client.15798 shutdown 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.118+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 msgr2=0x562041c0be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0be30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8000c00 tx=0x7fd7a800f7c0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 shutdown_connections 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c1c070 0x562041c1e460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c08b10 0x562041c0c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0be30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd79dffb700 0 client.15798 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/3086925110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041c0c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 >> 172.21.15.16:0/3086925110 conn(0x562041b5c210 msgr2=0x562041bf85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 shutdown_connections 2024-05-08T23:20:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/3086925110 wait complete. 2024-05-08T23:20:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.119+0000 7fd7c8be0b80 20 client.15798 trim_cache size 0 max 16384 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 Processor -- start 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 -- start start 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c02ca0 0x562041bfa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041bf8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 0x562041c07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041c235d0 con 0x562041c08b10 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041bf3800 con 0x562041c02ca0 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041c0a380 con 0x562041c081f0 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 0x562041c07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 0x562041c07610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56200/0 (socket says 172.21.15.16:56200) 2024-05-08T23:20:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 -- 172.21.15.16:0/4293697183 learned_addr learned my addr 172.21.15.16:0/4293697183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041bf8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b659c700 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c02ca0 0x562041bfa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 -- 172.21.15.16:0/4293697183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 msgr2=0x562041bf8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041bf8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 -- 172.21.15.16:0/4293697183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c02ca0 msgr2=0x562041bfa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c02ca0 0x562041bfa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.133+0000 7fd7b6d9d700 1 -- 172.21.15.16:0/4293697183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562041bf54e0 con 0x562041c08b10 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7b6d9d700 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 0x562041c07610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009460 tx=0x7fd7b0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4f7482cd987d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7a57fa700 1 -- 172.21.15.16:0/4293697183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000e030 con 0x562041c08b10 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7a57fa700 1 -- 172.21.15.16:0/4293697183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7b000f040 con 0x562041c08b10 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7a57fa700 1 -- 172.21.15.16:0/4293697183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00157e0 con 0x562041c08b10 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/4293697183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 msgr2=0x562041c07610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 0x562041c07610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009460 tx=0x7fd7b0009920 comp rx=0 tx=0).stop 2024-05-08T23:20:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/4293697183 shutdown_connections 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c08b10 0x562041c07610 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c02ca0 0x562041bfa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/4293697183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c081f0 0x562041bf8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/4293697183 >> 172.21.15.16:0/4293697183 conn(0x562041b5c210 msgr2=0x562041c06f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.134+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/4293697183 shutdown_connections 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/4293697183 wait complete. 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 -- start start 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 Processor -- start 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c081f0 0x562041c0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c08b10 0x562041c71490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041c719d0 con 0x562041c02ca0 2024-05-08T23:20:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041c71b10 con 0x562041c081f0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562041c71ca0 con 0x562041c08b10 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7c8be0b80 10 client.0 _ll_register_callbacks cb 0x562041af3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56202/0 (socket says 172.21.15.16:56202) 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7b659c700 1 -- 172.21.15.16:0/122327112 learned_addr learned my addr 172.21.15.16:0/122327112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c081f0 0x562041c0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.135+0000 7fd7b6d9d700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c08b10 0x562041c71490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7b659c700 1 -- 172.21.15.16:0/122327112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c08b10 msgr2=0x562041c71490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7b659c700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c08b10 0x562041c71490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7b659c700 1 -- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c081f0 msgr2=0x562041c0fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7b659c700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c081f0 0x562041c0fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7b659c700 1 -- 172.21.15.16:0/122327112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b0009110 con 0x562041c02ca0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7b659c700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0f7a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8000c00 tx=0x7fd7a8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133c19a1d07da436 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd79dffb700 1 -- 172.21.15.16:0/122327112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800ea50 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd79dffb700 1 -- 172.21.15.16:0/122327112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fd7a80386a0 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a801e040 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd79dffb700 1 -- 172.21.15.16:0/122327112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a800bb70 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7c8be0b80 10 client.15804 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562041b14560 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.136+0000 7fd79dffb700 1 -- 172.21.15.16:0/122327112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7a801e040 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd79dffb700 1 -- 172.21.15.16:0/122327112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7a800b890 con 0x562041c02ca0 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7c8be0b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7c8be0b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7c8be0b80 20 client.15804 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c72fc0 0x562041c753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7c8be0b80 4 client.15804 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562041b46ec0 con 0x562041c72fc0 2024-05-08T23:20:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.137+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c72fc0 0x562041c753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd79dffb700 10 client.15804 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7b5d9b700 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c72fc0 0x562041c753b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e619bf003031f429 server_cookie=11ac9ad96f226327 in_seq=0 out_seq=0 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd79dffb700 1 -- 172.21.15.16:0/122327112 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562041b46ec0 con 0x562041c72fc0 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd79dffb700 10 client.15804 handle_command_reply: tid=0 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 client.15804 shutdown 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 msgr2=0x562041c0f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0f7a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8000c00 tx=0x7fd7a8009290 comp rx=0 tx=0).stop 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 shutdown_connections 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562041c08b10 0x562041c71490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562041c02ca0 0x562041c0f7a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562041c72fc0 0x562041c753b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 --2- 172.21.15.16:0/122327112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562041c081f0 0x562041c0fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd79dffb700 0 client.15804 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 >> 172.21.15.16:0/122327112 conn(0x562041b5c210 msgr2=0x562041bfdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 shutdown_connections 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 1 -- 172.21.15.16:0/122327112 wait complete. 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.138+0000 7fd7c8be0b80 20 client.15804 trim_cache size 0 max 16384 2024-05-08T23:20:20.144 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd7a00a13b0 msgr2=0x7fd7a00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd7a00a13b0 0x7fd7a00a3870 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8006010 tx=0x7fd7a800b260 comp rx=0 tx=0).stop 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 msgr2=0x7fd7b8197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 0x7fd7b8197960 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00e970 tx=0x7fd7ac00ec80 comp rx=0 tx=0).stop 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 shutdown_connections 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd7a00a13b0 0x7fd7a00a3870 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7b819bcb0 0x7fd7b819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7b81011e0 0x7fd7b8197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 --2- 172.21.15.16:0/2114072133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7b8106300 0x7fd7b8197960 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 >> 172.21.15.16:0/2114072133 conn(0x7fd7b80fca80 msgr2=0x7fd7b80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 shutdown_connections 2024-05-08T23:20:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:20.139+0000 7fd7bca26700 1 -- 172.21.15.16:0/2114072133 wait complete. 2024-05-08T23:20:20.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:20 smithi189 ceph-mon[46919]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-08T23:20:20.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:20 smithi155 ceph-mon[47105]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-08T23:20:21.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:20 smithi016 ceph-mon[42541]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-05-08T23:20:21.152 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:20:21.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:20:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 -- 172.21.15.16:0/2954572192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c100ab0 msgr2=0x7fc96c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 --2- 172.21.15.16:0/2954572192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c100ab0 0x7fc96c10ccc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc96400b0d0 tx=0x7fc96400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 -- 172.21.15.16:0/2954572192 shutdown_connections 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 --2- 172.21.15.16:0/2954572192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c100ab0 0x7fc96c10ccc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 --2- 172.21.15.16:0/2954572192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 0x7fc96c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 --2- 172.21.15.16:0/2954572192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc96c1000f0 0x7fc96c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 -- 172.21.15.16:0/2954572192 >> 172.21.15.16:0/2954572192 conn(0x7fc96c076ba0 msgr2=0x7fc96c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 -- 172.21.15.16:0/2954572192 shutdown_connections 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.302+0000 7fc971b7b700 1 -- 172.21.15.16:0/2954572192 wait complete. 2024-05-08T23:20:21.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.303+0000 7fc971b7b700 1 Processor -- start 2024-05-08T23:20:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 -- start start 2024-05-08T23:20:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 0x7fc96c196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc96c100ab0 0x7fc96c196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 0x7fc96c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c19b160 con 0x7fc96c1000f0 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c19b2d0 con 0x7fc96c105f90 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc971b7b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c19b490 con 0x7fc96c100ab0 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 0x7fc96c196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 0x7fc96c196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56220/0 (socket says 172.21.15.16:56220) 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3855610006 learned_addr learned my addr 172.21.15.16:0/3855610006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96affd700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc96c100ab0 0x7fc96c196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96bfff700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 0x7fc96c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3855610006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc96c100ab0 msgr2=0x7fc96c196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc96c100ab0 0x7fc96c196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3855610006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 msgr2=0x7fc96c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 0x7fc96c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3855610006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964009d00 con 0x7fc96c1000f0 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.304+0000 7fc96bfff700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 0x7fc96c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.305+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 0x7fc96c196240 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc95c007b50 tx=0x7fc95c007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.305+0000 7fc968ff9700 1 -- 172.21.15.16:0/3855610006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c011070 con 0x7fc96c1000f0 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.305+0000 7fc968ff9700 1 -- 172.21.15.16:0/3855610006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc95c00c2e0 con 0x7fc96c1000f0 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.305+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c19b770 con 0x7fc96c1000f0 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.305+0000 7fc968ff9700 1 -- 172.21.15.16:0/3855610006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c003e80 con 0x7fc96c1000f0 2024-05-08T23:20:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.305+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c069370 con 0x7fc96c1000f0 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.307+0000 7fc968ff9700 1 -- 172.21.15.16:0/3855610006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fc95c004000 con 0x7fc96c1000f0 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.308+0000 7fc968ff9700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc9540a13b0 0x7fc9540a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.308+0000 7fc96affd700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc9540a13b0 0x7fc9540a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.308+0000 7fc968ff9700 1 -- 172.21.15.16:0/3855610006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fc95c0ce5d0 con 0x7fc96c1000f0 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.308+0000 7fc96affd700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc9540a13b0 0x7fc9540a3870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc96c197790 tx=0x7fc96000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 Processor -- start 2024-05-08T23:20:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 -- start start 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 0x55793e3b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b0910 0x55793e3b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e3071d0 con 0x55793e3b5e80 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e304b90 con 0x55793e3b67a0 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e300480 con 0x55793e3b0910 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 0x55793e3b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 0x55793e3b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56232/0 (socket says 172.21.15.16:56232) 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96b7fe700 1 -- 172.21.15.16:0/4278290786 learned_addr learned my addr 172.21.15.16:0/4278290786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96affd700 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96bfff700 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b0910 0x55793e3b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96b7fe700 1 -- 172.21.15.16:0/4278290786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b0910 msgr2=0x55793e3b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.309+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b0910 0x55793e3b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc96b7fe700 1 -- 172.21.15.16:0/4278290786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 msgr2=0x55793e3b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc96b7fe700 1 -- 172.21.15.16:0/4278290786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793e181430 con 0x55793e3b5e80 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 0x55793e3b6260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc95c097c80 tx=0x7fc95c09b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b60cec273cab00d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc951ffb700 1 -- 172.21.15.16:0/4278290786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01b2b0 con 0x55793e3b5e80 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc951ffb700 1 -- 172.21.15.16:0/4278290786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc95c0b2070 con 0x55793e3b5e80 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.310+0000 7fc951ffb700 1 -- 172.21.15.16:0/4278290786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c097920 con 0x55793e3b5e80 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 -- 172.21.15.16:0/4278290786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 msgr2=0x55793e3b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 0x55793e3b6260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc95c097c80 tx=0x7fc95c09b100 comp rx=0 tx=0).stop 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 -- 172.21.15.16:0/4278290786 shutdown_connections 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b0910 0x55793e3b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b5e80 0x55793e3b6260 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/4278290786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 -- 172.21.15.16:0/4278290786 >> 172.21.15.16:0/4278290786 conn(0x55793e30e290 msgr2=0x55793e3aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 -- 172.21.15.16:0/4278290786 shutdown_connections 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 -- 172.21.15.16:0/4278290786 wait complete. 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 -- start start 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.311+0000 7fc97dd35b80 1 Processor -- start 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3ba070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3ba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e29c5e0 con 0x55793e3b67a0 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e29c280 con 0x55793e3b0910 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e183720 con 0x55793e3b5e80 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc97dd35b80 10 client.0 _ll_register_callbacks cb 0x55793e2a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3ba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3ba5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56244/0 (socket says 172.21.15.16:56244) 2024-05-08T23:20:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96bfff700 1 -- 172.21.15.16:0/3243419067 learned_addr learned my addr 172.21.15.16:0/3243419067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96affd700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3ba070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3243419067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 msgr2=0x55793e3ba070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3ba070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3243419067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 msgr2=0x55793e3ba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3ba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 -- 172.21.15.16:0/3243419067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c069370 con 0x55793e3b0910 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96affd700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96bfff700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b9b30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc95c099560 tx=0x7fc95c097ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e51f074f1bd2aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.312+0000 7fc94affd700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc94affd700 1 -- 172.21.15.16:0/3243419067 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c09b910 con 0x55793e3b0910 2024-05-08T23:20:21.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55793e2592f0 con 0x55793e3b0910 2024-05-08T23:20:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc94affd700 1 -- 172.21.15.16:0/3243419067 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fc95c09f040 con 0x55793e3b0910 2024-05-08T23:20:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc94affd700 1 -- 172.21.15.16:0/3243419067 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95c0a3600 con 0x55793e3b0910 2024-05-08T23:20:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 10 client.25631 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55793e12dbe0 con 0x55793e3b0910 2024-05-08T23:20:21.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc94affd700 1 -- 172.21.15.16:0/3243419067 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc95c081020 con 0x55793e3b0910 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc94affd700 1 -- 172.21.15.16:0/3243419067 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc95c0b92d0 con 0x55793e3b0910 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 10 client.25631 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 20 client.25631 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e3c9db0 0x55793e3cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 4 client.25631 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.313+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55793e37f890 con 0x55793e3c9db0 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.314+0000 7fc96affd700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e3c9db0 0x55793e3cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.314+0000 7fc94affd700 10 client.25631 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.314+0000 7fc96affd700 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e3c9db0 0x55793e3cc1a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355e28711bea3db4 server_cookie=728a8b8d81777d23 in_seq=0 out_seq=0 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc94affd700 1 -- 172.21.15.16:0/3243419067 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55793e37f890 con 0x55793e3c9db0 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc94affd700 10 client.25631 handle_command_reply: tid=0 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 client.25631 shutdown 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 msgr2=0x55793e3b9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b9b30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc95c099560 tx=0x7fc95c097ba0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 shutdown_connections 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e3c9db0 0x55793e3cc1a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b9b30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc94affd700 0 client.25631 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3243419067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3ba070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 >> 172.21.15.16:0/3243419067 conn(0x55793e30e290 msgr2=0x55793e3aa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 shutdown_connections 2024-05-08T23:20:21.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.315+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3243419067 wait complete. 2024-05-08T23:20:21.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.316+0000 7fc97dd35b80 20 client.25631 trim_cache size 0 max 16384 2024-05-08T23:20:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 Processor -- start 2024-05-08T23:20:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 -- start start 2024-05-08T23:20:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e3ab960 con 0x55793e3b67a0 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e3d1340 con 0x55793e3b0910 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e3a5880 con 0x55793e3b5e80 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56248/0 (socket says 172.21.15.16:56248) 2024-05-08T23:20:21.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 -- 172.21.15.16:0/3961703189 learned_addr learned my addr 172.21.15.16:0/3961703189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96affd700 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 -- 172.21.15.16:0/3961703189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 msgr2=0x55793e3b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 -- 172.21.15.16:0/3961703189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 msgr2=0x55793e3b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96bfff700 1 -- 172.21.15.16:0/3961703189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793e3a7560 con 0x55793e3b67a0 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.334+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.335+0000 7fc96bfff700 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3aada0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc9640091b0 tx=0x7fc9640091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88850f6d8d93aff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.335+0000 7fc9527fc700 1 -- 172.21.15.16:0/3961703189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400e030 con 0x55793e3b67a0 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.335+0000 7fc9527fc700 1 -- 172.21.15.16:0/3961703189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc964004c50 con 0x55793e3b67a0 2024-05-08T23:20:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.335+0000 7fc9527fc700 1 -- 172.21.15.16:0/3961703189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964025aa0 con 0x55793e3b67a0 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.335+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3961703189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 msgr2=0x55793e3aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.335+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3aada0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc9640091b0 tx=0x7fc9640091e0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3961703189 shutdown_connections 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b67a0 0x55793e3aada0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b0910 0x55793e3b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/3961703189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3961703189 >> 172.21.15.16:0/3961703189 conn(0x55793e30e290 msgr2=0x55793e3b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3961703189 shutdown_connections 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- 172.21.15.16:0/3961703189 wait complete. 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- start start 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 Processor -- start 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 0x55793e3bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e420220 con 0x55793e3b0910 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e420360 con 0x55793e3b67a0 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.336+0000 7fc97dd35b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793e4204a0 con 0x55793e3b5e80 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc97dd35b80 10 client.0 _ll_register_callbacks cb 0x55793e2a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 0x55793e3bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 0x55793e3bfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56260/0 (socket says 172.21.15.16:56260) 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 -- 172.21.15.16:0/2788764476 learned_addr learned my addr 172.21.15.16:0/2788764476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96affd700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 -- 172.21.15.16:0/2788764476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 msgr2=0x55793e3c02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96bfff700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3c02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 -- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 msgr2=0x55793e3c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 -- 172.21.15.16:0/2788764476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c10d550 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96bfff700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc96b7fe700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 0x55793e3bfd60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc95c097710 tx=0x7fc95c09b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f50b5fe9abfb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc94affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc94affd700 1 -- 172.21.15.16:0/2788764476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c09b910 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc94affd700 1 -- 172.21.15.16:0/2788764476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7fc95c082080 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc95c0b2070 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc94affd700 1 -- 172.21.15.16:0/2788764476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95c09f040 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc97dd35b80 10 client.15822 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.337+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55793e3b0fc0 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc94affd700 1 -- 172.21.15.16:0/2788764476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc95c0a9020 con 0x55793e3b0910 2024-05-08T23:20:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc94affd700 1 -- 172.21.15.16:0/2788764476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc95c0a10b0 con 0x55793e3b0910 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc97dd35b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc97dd35b80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc97dd35b80 20 client.15822 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e4217b0 0x55793e423ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc97dd35b80 4 client.15822 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55793e2f8f40 con 0x55793e4217b0 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.338+0000 7fc96affd700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e4217b0 0x55793e423ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.339+0000 7fc94affd700 10 client.15822 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.339+0000 7fc96affd700 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e4217b0 0x55793e423ba0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ce13a6199ff3e6 server_cookie=c7b15275f677956f in_seq=0 out_seq=0 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc94affd700 1 -- 172.21.15.16:0/2788764476 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55793e2f8f40 con 0x55793e4217b0 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc94affd700 10 client.15822 handle_command_reply: tid=0 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 client.15822 shutdown 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 msgr2=0x55793e3bfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 0x55793e3bfd60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc95c097710 tx=0x7fc95c09b210 comp rx=0 tx=0).stop 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 shutdown_connections 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55793e4217b0 0x55793e423ba0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55793e3b67a0 0x55793e3c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55793e3b0910 0x55793e3bfd60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 --2- 172.21.15.16:0/2788764476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55793e3b5e80 0x55793e3c02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc94affd700 0 client.15822 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 >> 172.21.15.16:0/2788764476 conn(0x55793e30e290 msgr2=0x55793e3ba430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 shutdown_connections 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 1 -- 172.21.15.16:0/2788764476 wait complete. 2024-05-08T23:20:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.340+0000 7fc97dd35b80 20 client.15822 trim_cache size 0 max 16384 2024-05-08T23:20:21.349 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:20:21.349 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:20:21.349 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7f07bcd5-e66c-4890-8a2a-7c8d88b8885a", 2024-05-08T23:20:21.349 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:20:21.349 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc9540a13b0 msgr2=0x7fc9540a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc9540a13b0 0x7fc9540a3870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc96c197790 tx=0x7fc96000b410 comp rx=0 tx=0).stop 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 msgr2=0x7fc96c196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 0x7fc96c196240 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc95c007b50 tx=0x7fc95c007e60 comp rx=0 tx=0).stop 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 shutdown_connections 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc9540a13b0 0x7fc9540a3870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc96c105f90 0x7fc96c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c1000f0 0x7fc96c196240 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 --2- 172.21.15.16:0/3855610006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc96c100ab0 0x7fc96c196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 >> 172.21.15.16:0/3855610006 conn(0x7fc96c076ba0 msgr2=0x7fc96c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 shutdown_connections 2024-05-08T23:20:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.341+0000 7fc971b7b700 1 -- 172.21.15.16:0/3855610006 wait complete. 2024-05-08T23:20:21.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 -- 172.21.15.16:0/174108496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 msgr2=0x7f531810cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 --2- 172.21.15.16:0/174108496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f531810cce0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f531000b0a0 tx=0x7f531000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 -- 172.21.15.16:0/174108496 shutdown_connections 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 --2- 172.21.15.16:0/174108496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f531810cce0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 --2- 172.21.15.16:0/174108496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5318100060 0x7f5318100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 --2- 172.21.15.16:0/174108496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5318100a10 0x7f5318104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 -- 172.21.15.16:0/174108496 >> 172.21.15.16:0/174108496 conn(0x7f5318076f40 msgr2=0x7f5318077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 -- 172.21.15.16:0/174108496 shutdown_connections 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.493+0000 7f531dc13700 1 -- 172.21.15.16:0/174108496 wait complete. 2024-05-08T23:20:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 Processor -- start 2024-05-08T23:20:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 -- start start 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5318100060 0x7f5318199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f5318199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531819df40 0x7f531819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531819e9c0 con 0x7f5318105160 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531819eb30 con 0x7f531819df40 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f531dc13700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531819ecf0 con 0x7f5318100060 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f5318199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f5318199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56280/0 (socket says 172.21.15.16:56280) 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.494+0000 7f5316ffd700 1 -- 172.21.15.16:0/4091630908 learned_addr learned my addr 172.21.15.16:0/4091630908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f53177fe700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5318100060 0x7f5318199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5317fff700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531819df40 0x7f531819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5316ffd700 1 -- 172.21.15.16:0/4091630908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5318100060 msgr2=0x7f5318199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5316ffd700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5318100060 0x7f5318199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5316ffd700 1 -- 172.21.15.16:0/4091630908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531819df40 msgr2=0x7f531819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5316ffd700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531819df40 0x7f531819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5316ffd700 1 -- 172.21.15.16:0/4091630908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310009d00 con 0x7f5318105160 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5316ffd700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f5318199ba0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f530000d950 tx=0x7f530000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5314ff9700 1 -- 172.21.15.16:0/4091630908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300004220 con 0x7f5318105160 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53181a6250 con 0x7f5318105160 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5314ff9700 1 -- 172.21.15.16:0/4091630908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5300007620 con 0x7f5318105160 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f5314ff9700 1 -- 172.21.15.16:0/4091630908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300005530 con 0x7f5318105160 2024-05-08T23:20:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.495+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53181a6780 con 0x7f5318105160 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.498+0000 7f5314ff9700 1 -- 172.21.15.16:0/4091630908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f5300004750 con 0x7f5318105160 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.498+0000 7f5314ff9700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f53040a13b0 0x7f53040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.498+0000 7f5314ff9700 1 -- 172.21.15.16:0/4091630908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f53000cd070 con 0x7f5318105160 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.498+0000 7f53177fe700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f53040a13b0 0x7f53040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f53177fe700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f53040a13b0 0x7f53040a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5308009c60 tx=0x7f5308009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f5329dcdb80 1 Processor -- start 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f5329dcdb80 1 -- start start 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc07f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07f720 0x55fddc079350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc079890 0x55fddc079cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.499+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddbfd0150 con 0x55fddc07ee00 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddbfcdb10 con 0x55fddc07f720 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddbfc9400 con 0x55fddc079890 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc07f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc07f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56294/0 (socket says 172.21.15.16:56294) 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 -- 172.21.15.16:0/2904218958 learned_addr learned my addr 172.21.15.16:0/2904218958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f5317fff700 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc079890 0x55fddc079cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f5316ffd700 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07f720 0x55fddc079350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 -- 172.21.15.16:0/2904218958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc079890 msgr2=0x55fddc079cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc079890 0x55fddc079cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 -- 172.21.15.16:0/2904218958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07f720 msgr2=0x55fddc079350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07f720 0x55fddc079350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 -- 172.21.15.16:0/2904218958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fddbe4a3b0 con 0x55fddc07ee00 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f5316ffd700 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07f720 0x55fddc079350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f53177fe700 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc07f1e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f530800e4e0 tx=0x7f5308014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba80201f797f899 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f530dffb700 1 -- 172.21.15.16:0/2904218958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308026430 con 0x55fddc07ee00 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f530dffb700 1 -- 172.21.15.16:0/2904218958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5308021070 con 0x55fddc07ee00 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.500+0000 7f530dffb700 1 -- 172.21.15.16:0/2904218958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530801d670 con 0x55fddc07ee00 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 -- 172.21.15.16:0/2904218958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 msgr2=0x55fddc07f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc07f1e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f530800e4e0 tx=0x7f5308014000 comp rx=0 tx=0).stop 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 -- 172.21.15.16:0/2904218958 shutdown_connections 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc079890 0x55fddc079cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc07f1e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/2904218958 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07f720 0x55fddc079350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 -- 172.21.15.16:0/2904218958 >> 172.21.15.16:0/2904218958 conn(0x55fddbfd7210 msgr2=0x55fddc073df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 -- 172.21.15.16:0/2904218958 shutdown_connections 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.501+0000 7f5329dcdb80 1 -- 172.21.15.16:0/2904218958 wait complete. 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 -- start start 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 Processor -- start 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 0x55fddc082a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07ee00 0x55fddc082f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc083480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddbf65560 con 0x55fddc079890 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddbf65200 con 0x55fddc07ee00 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddbe4c6a0 con 0x55fddc07f720 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5329dcdb80 10 client.0 _ll_register_callbacks cb 0x55fddbf6eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f53177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 0x55fddc082a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f53177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 0x55fddc082a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56298/0 (socket says 172.21.15.16:56298) 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f53177fe700 1 -- 172.21.15.16:0/3385025998 learned_addr learned my addr 172.21.15.16:0/3385025998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5316ffd700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07ee00 0x55fddc082f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.502+0000 7f5317fff700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc083480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f53177fe700 1 -- 172.21.15.16:0/3385025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 msgr2=0x55fddc083480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f53177fe700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc083480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f53177fe700 1 -- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07ee00 msgr2=0x55fddc082f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f53177fe700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07ee00 0x55fddc082f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f53177fe700 1 -- 172.21.15.16:0/3385025998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5308017070 con 0x55fddc079890 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f53177fe700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 0x55fddc082a00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5308011ab0 tx=0x7f530801cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec6f30c93e4051c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f52f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f52f6ffd700 1 -- 172.21.15.16:0/3385025998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530800ee70 con 0x55fddc079890 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f52f6ffd700 1 -- 172.21.15.16:0/3385025998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f530801c620 con 0x55fddc079890 2024-05-08T23:20:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fddbf22270 con 0x55fddc079890 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f52f6ffd700 1 -- 172.21.15.16:0/3385025998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f530801c7e0 con 0x55fddc079890 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f5329dcdb80 10 client.15834 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.503+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fddbdf6b60 con 0x55fddc079890 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f52f6ffd700 1 -- 172.21.15.16:0/3385025998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fddbf22270 con 0x55fddc079890 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f52f6ffd700 1 -- 172.21.15.16:0/3385025998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53080038b0 con 0x55fddc079890 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5329dcdb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5329dcdb80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5329dcdb80 20 client.15834 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc092fe0 0x55fddc0953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5329dcdb80 4 client.15834 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fddc048810 con 0x55fddc092fe0 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.504+0000 7f5316ffd700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc092fe0 0x55fddc0953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5316ffd700 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc092fe0 0x55fddc0953d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc993354e995c88 server_cookie=d58a194664ddcd68 in_seq=0 out_seq=0 2024-05-08T23:20:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f52f6ffd700 10 client.15834 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f52f6ffd700 1 -- 172.21.15.16:0/3385025998 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fddc048810 con 0x55fddc092fe0 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f52f6ffd700 10 client.15834 handle_command_reply: tid=0 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 client.15834 shutdown 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 msgr2=0x55fddc082a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 0x55fddc082a00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5308011ab0 tx=0x7f530801cf00 comp rx=0 tx=0).stop 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 shutdown_connections 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc083480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc079890 0x55fddc082a00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc092fe0 0x55fddc0953d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/3385025998 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc07ee00 0x55fddc082f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.505+0000 7f52f6ffd700 0 client.15834 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.506+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 >> 172.21.15.16:0/3385025998 conn(0x55fddbfd7210 msgr2=0x55fddc0735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.506+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 shutdown_connections 2024-05-08T23:20:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.506+0000 7f5329dcdb80 1 -- 172.21.15.16:0/3385025998 wait complete. 2024-05-08T23:20:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.506+0000 7f5329dcdb80 20 client.15834 trim_cache size 0 max 16384 2024-05-08T23:20:21.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.524+0000 7f5329dcdb80 1 Processor -- start 2024-05-08T23:20:21.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.524+0000 7f5329dcdb80 1 -- start start 2024-05-08T23:20:21.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc079350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc073d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc0832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc06e800 con 0x55fddc07ee00 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc07f1e0 con 0x55fddc079890 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc074260 con 0x55fddc07f720 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56310/0 (socket says 172.21.15.16:56310) 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 -- 172.21.15.16:0/137631648 learned_addr learned my addr 172.21.15.16:0/137631648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5317fff700 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc0832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f53177fe700 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc079350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 -- 172.21.15.16:0/137631648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 msgr2=0x55fddc0832a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc0832a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 -- 172.21.15.16:0/137631648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 msgr2=0x55fddc079350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc079350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 -- 172.21.15.16:0/137631648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fddbf8f560 con 0x55fddc07ee00 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f53177fe700 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc079350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f5316ffd700 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc073d20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5300099b60 tx=0x7f530001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b16ea848ed4bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f530e7fc700 1 -- 172.21.15.16:0/137631648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000a20a0 con 0x55fddc07ee00 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f530e7fc700 1 -- 172.21.15.16:0/137631648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5300005690 con 0x55fddc07ee00 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.525+0000 7f530e7fc700 1 -- 172.21.15.16:0/137631648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530009a0c0 con 0x55fddc07ee00 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 -- 172.21.15.16:0/137631648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 msgr2=0x55fddc073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc073d20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5300099b60 tx=0x7f530001c040 comp rx=0 tx=0).stop 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 -- 172.21.15.16:0/137631648 shutdown_connections 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07f720 0x55fddc0832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc079350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/137631648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07ee00 0x55fddc073d20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 -- 172.21.15.16:0/137631648 >> 172.21.15.16:0/137631648 conn(0x55fddbfd7210 msgr2=0x55fddc0812d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 -- 172.21.15.16:0/137631648 shutdown_connections 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 -- 172.21.15.16:0/137631648 wait complete. 2024-05-08T23:20:21.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.526+0000 7f5329dcdb80 1 -- start start 2024-05-08T23:20:21.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 Processor -- start 2024-05-08T23:20:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc08a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07ee00 0x55fddc088770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 0x55fddc088cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc0891f0 con 0x55fddc07f720 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc089360 con 0x55fddc079890 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc089520 con 0x55fddc07ee00 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5329dcdb80 10 client.0 _ll_register_callbacks cb 0x55fddbf6eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5317fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 0x55fddc088cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5317fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 0x55fddc088cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56312/0 (socket says 172.21.15.16:56312) 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f5317fff700 1 -- 172.21.15.16:0/472581176 learned_addr learned my addr 172.21.15.16:0/472581176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.527+0000 7f53177fe700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc08a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5316ffd700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07ee00 0x55fddc088770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5317fff700 1 -- 172.21.15.16:0/472581176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07ee00 msgr2=0x55fddc088770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5317fff700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07ee00 0x55fddc088770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5317fff700 1 -- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 msgr2=0x55fddc08a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5317fff700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc08a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5317fff700 1 -- 172.21.15.16:0/472581176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53181a6780 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f53177fe700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc08a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5317fff700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 0x55fddc088cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f531000b0a0 tx=0x7f531000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b89489375d43c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f52f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f52f6ffd700 1 -- 172.21.15.16:0/472581176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000e040 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f52f6ffd700 1 -- 172.21.15.16:0/472581176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 9300+0+0 (secure 0 0 0) 0x7f5310023070 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5308021070 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f52f6ffd700 1 -- 172.21.15.16:0/472581176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53100092e0 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5329dcdb80 10 client.15840 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.528+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fddbe4ecd0 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f52f6ffd700 1 -- 172.21.15.16:0/472581176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5308021070 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f52f6ffd700 1 -- 172.21.15.16:0/472581176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f531000b0d0 con 0x55fddc07f720 2024-05-08T23:20:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f5329dcdb80 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f5329dcdb80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f5329dcdb80 20 client.15840 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc0eaa10 0x55fddc0ece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f5329dcdb80 4 client.15840 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fddbfc1ec0 con 0x55fddc0eaa10 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.529+0000 7f53177fe700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc0eaa10 0x55fddc0ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f52f6ffd700 10 client.15840 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f53177fe700 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc0eaa10 0x55fddc0ece30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9574577bbc947fe server_cookie=1160a9c47cd0f3c7 in_seq=0 out_seq=0 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f52f6ffd700 1 -- 172.21.15.16:0/472581176 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fddbfc1ec0 con 0x55fddc0eaa10 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f52f6ffd700 10 client.15840 handle_command_reply: tid=0 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 client.15840 shutdown 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 msgr2=0x55fddc088cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 0x55fddc088cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f531000b0a0 tx=0x7f531000bc80 comp rx=0 tx=0).stop 2024-05-08T23:20:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 shutdown_connections 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fddc07f720 0x55fddc088cb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55fddc0eaa10 0x55fddc0ece30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fddc079890 0x55fddc08a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 --2- 172.21.15.16:0/472581176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fddc07ee00 0x55fddc088770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 >> 172.21.15.16:0/472581176 conn(0x55fddbfd7210 msgr2=0x55fddc07fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f52f6ffd700 0 client.15840 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 shutdown_connections 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 1 -- 172.21.15.16:0/472581176 wait complete. 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.530+0000 7f5329dcdb80 20 client.15840 trim_cache size 0 max 16384 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:20:21.536 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: "7f07bcd5-e66c-4890-8a2a-7c8d88b8885a": { 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "7f07bcd5-e66c-4890-8a2a-7c8d88b8885a", 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:20:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f53040a13b0 msgr2=0x7f53040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f53040a13b0 0x7f53040a3870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5308009c60 tx=0x7f5308009260 comp rx=0 tx=0).stop 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 msgr2=0x7f5318199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f5318199ba0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f530000d950 tx=0x7f530000dc60 comp rx=0 tx=0).stop 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 shutdown_connections 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f53040a13b0 0x7f53040a3870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f531819df40 0x7f531819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5318100060 0x7f5318199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 --2- 172.21.15.16:0/4091630908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5318105160 0x7f5318199ba0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 >> 172.21.15.16:0/4091630908 conn(0x7f5318076f40 msgr2=0x7f53180ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 shutdown_connections 2024-05-08T23:20:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:21.531+0000 7f531dc13700 1 -- 172.21.15.16:0/4091630908 wait complete. 2024-05-08T23:20:21.544 INFO:tasks.cephfs.filesystem:scrub status for tag:7f07bcd5-e66c-4890-8a2a-7c8d88b8885a - {'path': '/', 'tag': '7f07bcd5-e66c-4890-8a2a-7c8d88b8885a', 'options': 'recursive,force'} 2024-05-08T23:20:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:22 smithi016 ceph-mon[42541]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T23:20:22.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:22 smithi189 ceph-mon[46919]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T23:20:22.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:22 smithi155 ceph-mon[47105]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-08T23:20:23.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:23 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:20:23.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:23 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:23.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:23 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:20:23.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:23 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:23.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:23 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:20:23.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:23 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:23.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:23 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:20:23.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:23 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:23.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:23 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:20:23.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:23 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:23.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:23 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:20:23.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:23 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:24.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:24 smithi016 ceph-mon[42541]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T23:20:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:24 smithi189 ceph-mon[46919]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T23:20:24.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:24 smithi155 ceph-mon[47105]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T23:20:25.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:25 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:20:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:25 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:20:25.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:25 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:20:26.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:26 smithi189 ceph-mon[46919]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T23:20:26.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:26 smithi155 ceph-mon[47105]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T23:20:26.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:26 smithi016 ceph-mon[42541]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-08T23:20:28.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:28 smithi016 ceph-mon[42541]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:20:28.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:28 smithi189 ceph-mon[46919]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:20:28.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:28 smithi155 ceph-mon[47105]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:20:30.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:30 smithi016 ceph-mon[42541]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:20:30.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:30 smithi189 ceph-mon[46919]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:20:30.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:30 smithi155 ceph-mon[47105]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:20:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:32 smithi016 ceph-mon[42541]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:20:32.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:32 smithi189 ceph-mon[46919]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:20:32.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:32 smithi155 ceph-mon[47105]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:20:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:34 smithi016 ceph-mon[42541]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:20:34.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:34 smithi189 ceph-mon[46919]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:20:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:34 smithi155 ceph-mon[47105]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:20:36.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:36 smithi016 ceph-mon[42541]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T23:20:36.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:36 smithi189 ceph-mon[46919]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T23:20:36.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:36 smithi155 ceph-mon[47105]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-08T23:20:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:38 smithi016 ceph-mon[42541]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:38.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-05-08T23:20:38.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:38 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-05-08T23:20:38.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:38 smithi189 ceph-mon[46919]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:38.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-05-08T23:20:38.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:38 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-05-08T23:20:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:38 smithi155 ceph-mon[47105]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-05-08T23:20:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:38 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-05-08T23:20:39.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-05-08T23:20:39.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:39 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-05-08T23:20:39.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:39 smithi016 ceph-mon[42541]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:20:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-05-08T23:20:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:39 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-05-08T23:20:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:39 smithi189 ceph-mon[46919]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:20:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-05-08T23:20:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:39 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-05-08T23:20:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:39 smithi155 ceph-mon[47105]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T23:20:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:40 smithi016 ceph-mon[42541]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:20:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:40 smithi016 ceph-mon[42541]: pgmap v36: 129 pgs: 2 peering, 127 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:40 smithi189 ceph-mon[46919]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:20:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:40 smithi189 ceph-mon[46919]: pgmap v36: 129 pgs: 2 peering, 127 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:40 smithi155 ceph-mon[47105]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T23:20:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:40 smithi155 ceph-mon[47105]: pgmap v36: 129 pgs: 2 peering, 127 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:41.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:41 smithi016 ceph-mon[42541]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-08T23:20:41.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:41 smithi189 ceph-mon[46919]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-08T23:20:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:41 smithi155 ceph-mon[47105]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-05-08T23:20:42.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:42 smithi016 ceph-mon[42541]: pgmap v37: 129 pgs: 2 peering, 127 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-08T23:20:42.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:42 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:20:42.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:42 smithi189 ceph-mon[46919]: pgmap v37: 129 pgs: 2 peering, 127 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-08T23:20:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:42 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:20:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:42 smithi155 ceph-mon[47105]: pgmap v37: 129 pgs: 2 peering, 127 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-08T23:20:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:42 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:20:43.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:43 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:20:43.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:43 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:20:43.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:43 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:20:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:43 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:20:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:43 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:20:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:43 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:20:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:43 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:20:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:43 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:20:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:43 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:20:44.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:44 smithi016 ceph-mon[42541]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:44.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:44 smithi189 ceph-mon[46919]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:44 smithi155 ceph-mon[47105]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:45.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:45 smithi016 ceph-mon[42541]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-05-08T23:20:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:45 smithi016 ceph-mon[42541]: Cluster is now healthy 2024-05-08T23:20:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:45 smithi189 ceph-mon[46919]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-05-08T23:20:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:45 smithi189 ceph-mon[46919]: Cluster is now healthy 2024-05-08T23:20:45.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:45 smithi155 ceph-mon[47105]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-05-08T23:20:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:45 smithi155 ceph-mon[47105]: Cluster is now healthy 2024-05-08T23:20:46.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:46 smithi016 ceph-mon[42541]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:46.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:46 smithi189 ceph-mon[46919]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:46.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:46 smithi155 ceph-mon[47105]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:48 smithi016 ceph-mon[42541]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-08T23:20:48.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:48 smithi189 ceph-mon[46919]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-08T23:20:48.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:48 smithi155 ceph-mon[47105]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-08T23:20:50.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:50 smithi016 ceph-mon[42541]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T23:20:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:50 smithi189 ceph-mon[46919]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T23:20:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:50 smithi155 ceph-mon[47105]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-08T23:20:51.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 -- 172.21.15.16:0/833283308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 msgr2=0x7f02c810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 --2- 172.21.15.16:0/833283308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c810de00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f02c000b0a0 tx=0x7f02c000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 -- 172.21.15.16:0/833283308 shutdown_connections 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 --2- 172.21.15.16:0/833283308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c810de00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 --2- 172.21.15.16:0/833283308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c8101160 0x7f02c8101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 --2- 172.21.15.16:0/833283308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101b10 0x7f02c8105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 -- 172.21.15.16:0/833283308 >> 172.21.15.16:0/833283308 conn(0x7f02c80fc9c0 msgr2=0x7f02c80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 -- 172.21.15.16:0/833283308 shutdown_connections 2024-05-08T23:20:51.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 -- 172.21.15.16:0/833283308 wait complete. 2024-05-08T23:20:51.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.686+0000 7f02cf502700 1 Processor -- start 2024-05-08T23:20:51.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 -- start start 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101160 0x7f02c8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c819df00 0x7f02c819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c819e980 con 0x7f02c8106280 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c819eaf0 con 0x7f02c8101160 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cf502700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c819ecb0 con 0x7f02c819df00 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c8199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57910/0 (socket says 172.21.15.16:57910) 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 -- 172.21.15.16:0/3956971838 learned_addr learned my addr 172.21.15.16:0/3956971838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:51.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cda9f700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c819df00 0x7f02c819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cd29e700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101160 0x7f02c8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 -- 172.21.15.16:0/3956971838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c819df00 msgr2=0x7f02c819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c819df00 0x7f02c819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 -- 172.21.15.16:0/3956971838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101160 msgr2=0x7f02c8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101160 0x7f02c8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cca9d700 1 -- 172.21.15.16:0/3956971838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0009d00 con 0x7f02c8106280 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cda9f700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c819df00 0x7f02c819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.687+0000 7f02cd29e700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101160 0x7f02c8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.688+0000 7f02cca9d700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c8199bb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f02b800e970 tx=0x7f02b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.688+0000 7f02be7fc700 1 -- 172.21.15.16:0/3956971838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8009790 con 0x7f02c8106280 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.688+0000 7f02be7fc700 1 -- 172.21.15.16:0/3956971838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b8004cc0 con 0x7f02c8106280 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.688+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c81a6210 con 0x7f02c8106280 2024-05-08T23:20:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.688+0000 7f02be7fc700 1 -- 172.21.15.16:0/3956971838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80055b0 con 0x7f02c8106280 2024-05-08T23:20:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.688+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c81a66f0 con 0x7f02c8106280 2024-05-08T23:20:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.690+0000 7f02be7fc700 1 -- 172.21.15.16:0/3956971838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f02b8004750 con 0x7f02c8106280 2024-05-08T23:20:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.691+0000 7f02be7fc700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f02b40a13b0 0x7f02b40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.691+0000 7f02cd29e700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f02b40a13b0 0x7f02b40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.691+0000 7f02be7fc700 1 -- 172.21.15.16:0/3956971838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f02b8014070 con 0x7f02c8106280 2024-05-08T23:20:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.692+0000 7f02cd29e700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f02b40a13b0 0x7f02b40a3870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009cb0 tx=0x7f02c4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.692+0000 7f02db6bcb80 1 Processor -- start 2024-05-08T23:20:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.692+0000 7f02db6bcb80 1 -- start start 2024-05-08T23:20:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d55371350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55371cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d552c8150 con 0x555d55376e00 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d552c5b10 con 0x555d55371890 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d552c1400 con 0x555d55377720 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57926/0 (socket says 172.21.15.16:57926) 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 -- 172.21.15.16:0/2311961571 learned_addr learned my addr 172.21.15.16:0/2311961571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cca9d700 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d55371350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 -- 172.21.15.16:0/2311961571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 msgr2=0x555d55371350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d55371350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 -- 172.21.15.16:0/2311961571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 msgr2=0x555d55371cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55371cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 -- 172.21.15.16:0/2311961571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d551423b0 con 0x555d55376e00 2024-05-08T23:20:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cca9d700 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d55371350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02cd29e700 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553771e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f02c4011fb0 tx=0x7f02c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e182e10fa903b66f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02b37fe700 1 -- 172.21.15.16:0/2311961571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4003880 con 0x555d55376e00 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.693+0000 7f02b37fe700 1 -- 172.21.15.16:0/2311961571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02c4003e70 con 0x555d55376e00 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02b37fe700 1 -- 172.21.15.16:0/2311961571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4012cc0 con 0x555d55376e00 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 -- 172.21.15.16:0/2311961571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 msgr2=0x555d553771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553771e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f02c4011fb0 tx=0x7f02c4014000 comp rx=0 tx=0).stop 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 -- 172.21.15.16:0/2311961571 shutdown_connections 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55371cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553771e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/2311961571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d55371350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 -- 172.21.15.16:0/2311961571 >> 172.21.15.16:0/2311961571 conn(0x555d552cf210 msgr2=0x555d5536bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 -- 172.21.15.16:0/2311961571 shutdown_connections 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.694+0000 7f02db6bcb80 1 -- 172.21.15.16:0/2311961571 wait complete. 2024-05-08T23:20:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 -- start start 2024-05-08T23:20:51.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 Processor -- start 2024-05-08T23:20:51.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d5537aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5537b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5525d560 con 0x555d55376e00 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5525d200 con 0x555d55377720 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d551446a0 con 0x555d55371890 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02db6bcb80 10 client.0 _ll_register_callbacks cb 0x555d55266b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.695+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57942/0 (socket says 172.21.15.16:57942) 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 -- 172.21.15.16:0/1559131983 learned_addr learned my addr 172.21.15.16:0/1559131983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cd29e700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d5537aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cda9f700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5537b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 -- 172.21.15.16:0/1559131983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 msgr2=0x555d5537aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d5537aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 -- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 msgr2=0x555d5537b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5537b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 -- 172.21.15.16:0/1559131983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c4011c60 con 0x555d55376e00 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cd29e700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d5537aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cda9f700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5537b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02cca9d700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537af80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005950 tx=0x7f02b800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce10e2cc352f589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02b0ff9700 1 -- 172.21.15.16:0/1559131983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80cd3e0 con 0x555d55376e00 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d5521a270 con 0x555d55376e00 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02b0ff9700 1 -- 172.21.15.16:0/1559131983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f02b809a0f0 con 0x555d55376e00 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02b0ff9700 1 -- 172.21.15.16:0/1559131983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b809df20 con 0x555d55376e00 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02db6bcb80 10 client.15852 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.696+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555d550eeb60 con 0x555d55376e00 2024-05-08T23:20:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02b0ff9700 1 -- 172.21.15.16:0/1559131983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555d5521a270 con 0x555d55376e00 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02b0ff9700 1 -- 172.21.15.16:0/1559131983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02b80a20e0 con 0x555d55376e00 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02db6bcb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02db6bcb80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02db6bcb80 20 client.15852 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d5538b080 0x555d5538d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02db6bcb80 4 client.15852 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d55340810 con 0x555d5538b080 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.697+0000 7f02cd29e700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d5538b080 0x555d5538d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.698+0000 7f02cd29e700 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d5538b080 0x555d5538d4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793a941c04bbadb server_cookie=486923233b6061fd in_seq=0 out_seq=0 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.698+0000 7f02b0ff9700 10 client.15852 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02b0ff9700 1 -- 172.21.15.16:0/1559131983 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d55340810 con 0x555d5538b080 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02b0ff9700 10 client.15852 handle_command_reply: tid=0 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 client.15852 shutdown 2024-05-08T23:20:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 msgr2=0x555d5537af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537af80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005950 tx=0x7f02b800ce00 comp rx=0 tx=0).stop 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 shutdown_connections 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d5538b080 0x555d5538d4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5537b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d5537aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02b0ff9700 0 client.15852 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1559131983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537af80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 >> 172.21.15.16:0/1559131983 conn(0x555d552cf210 msgr2=0x555d5536b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 shutdown_connections 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1559131983 wait complete. 2024-05-08T23:20:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.699+0000 7f02db6bcb80 20 client.15852 trim_cache size 0 max 16384 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 Processor -- start 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 -- start start 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d55371350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5536bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5536c910 con 0x555d55376e00 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d55368dc0 con 0x555d55377720 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d55377590 con 0x555d55371890 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553763c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57958/0 (socket says 172.21.15.16:57958) 2024-05-08T23:20:51.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.718+0000 7f02cca9d700 1 -- 172.21.15.16:0/1724950892 learned_addr learned my addr 172.21.15.16:0/1724950892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cd29e700 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d55371350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cda9f700 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5536bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cca9d700 1 -- 172.21.15.16:0/1724950892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 msgr2=0x555d55371350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cca9d700 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d55371350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cca9d700 1 -- 172.21.15.16:0/1724950892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 msgr2=0x555d5536bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cca9d700 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5536bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cca9d700 1 -- 172.21.15.16:0/1724950892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d55371f40 con 0x555d55376e00 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02cca9d700 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553763c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f02b8098440 tx=0x7f02b8004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92215339cb3b1e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02b3fff700 1 -- 172.21.15.16:0/1724950892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80cab30 con 0x555d55376e00 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02b3fff700 1 -- 172.21.15.16:0/1724950892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b80cd070 con 0x555d55376e00 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.719+0000 7f02b3fff700 1 -- 172.21.15.16:0/1724950892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80a20e0 con 0x555d55376e00 2024-05-08T23:20:51.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1724950892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 msgr2=0x555d553763c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553763c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f02b8098440 tx=0x7f02b8004ee0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1724950892 shutdown_connections 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55377720 0x555d5536bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55371890 0x555d55371350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/1724950892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d553763c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1724950892 >> 172.21.15.16:0/1724950892 conn(0x555d552cf210 msgr2=0x555d55379310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1724950892 shutdown_connections 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 -- 172.21.15.16:0/1724950892 wait complete. 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 -- start start 2024-05-08T23:20:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.720+0000 7f02db6bcb80 1 Processor -- start 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55382220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d5537ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5537f1c0 con 0x555d55376e00 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5537f330 con 0x555d55371890 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d5537f4f0 con 0x555d55377720 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02db6bcb80 10 client.0 _ll_register_callbacks cb 0x555d55266b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57972/0 (socket says 172.21.15.16:57972) 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 -- 172.21.15.16:0/399840269 learned_addr learned my addr 172.21.15.16:0/399840269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cda9f700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d5537ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cd29e700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55382220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 -- 172.21.15.16:0/399840269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 msgr2=0x555d5537ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d5537ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 -- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 msgr2=0x555d55382220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55382220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 -- 172.21.15.16:0/399840269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d550eeb60 con 0x555d55376e00 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02cca9d700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537e740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f02b80ca910 tx=0x7f02b8099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985b8491df4d7007 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.721+0000 7f02b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02b0ff9700 1 -- 172.21.15.16:0/399840269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b809a0f0 con 0x555d55376e00 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02b0ff9700 1 -- 172.21.15.16:0/399840269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f02b80caae0 con 0x555d55376e00 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02c4003e70 con 0x555d55376e00 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02b0ff9700 1 -- 172.21.15.16:0/399840269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b80a2570 con 0x555d55376e00 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 10 client.15858 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:51.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555d55287560 con 0x555d55376e00 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02b0ff9700 1 -- 172.21.15.16:0/399840269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02c4003e70 con 0x555d55376e00 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02b0ff9700 1 -- 172.21.15.16:0/399840269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02b80b8210 con 0x555d55376e00 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 20 client.15858 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d553e2560 0x555d553e4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 4 client.15858 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.722+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d552b9ec0 con 0x555d553e2560 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.723+0000 7f02cd29e700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d553e2560 0x555d553e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.723+0000 7f02cd29e700 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d553e2560 0x555d553e4950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea405f22a77f267 server_cookie=e22c467e08e2948e in_seq=0 out_seq=0 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.723+0000 7f02b0ff9700 10 client.15858 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.723+0000 7f02b0ff9700 1 -- 172.21.15.16:0/399840269 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555d552b9ec0 con 0x555d553e2560 2024-05-08T23:20:51.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.723+0000 7f02b0ff9700 10 client.15858 handle_command_reply: tid=0 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.723+0000 7f02db6bcb80 1 client.15858 shutdown 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 msgr2=0x555d5537e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537e740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f02b80ca910 tx=0x7f02b8099b00 comp rx=0 tx=0).stop 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 shutdown_connections 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d55377720 0x555d5537ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555d553e2560 0x555d553e4950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555d55371890 0x555d55382220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 --2- 172.21.15.16:0/399840269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d55376e00 0x555d5537e740 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02b0ff9700 0 client.15858 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 >> 172.21.15.16:0/399840269 conn(0x555d552cf210 msgr2=0x555d5537b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 shutdown_connections 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 1 -- 172.21.15.16:0/399840269 wait complete. 2024-05-08T23:20:51.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.724+0000 7f02db6bcb80 20 client.15858 trim_cache size 0 max 16384 2024-05-08T23:20:51.731 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:20:51.731 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:20:51.731 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:20:51.731 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f02b40a13b0 msgr2=0x7f02b40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f02b40a13b0 0x7f02b40a3870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009cb0 tx=0x7f02c4009260 comp rx=0 tx=0).stop 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 msgr2=0x7f02c8199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c8199bb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f02b800e970 tx=0x7f02b800ed30 comp rx=0 tx=0).stop 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 shutdown_connections 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f02b40a13b0 0x7f02b40a3870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c819df00 0x7f02c819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02c8101160 0x7f02c8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 --2- 172.21.15.16:0/3956971838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c8106280 0x7f02c8199bb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 >> 172.21.15.16:0/3956971838 conn(0x7f02c80fc9c0 msgr2=0x7f02c80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 shutdown_connections 2024-05-08T23:20:51.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.725+0000 7f02cf502700 1 -- 172.21.15.16:0/3956971838 wait complete. 2024-05-08T23:20:51.739 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:20:51.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:20:51.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 -- 172.21.15.16:0/2501960451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11410b3c0 msgr2=0x7fc11410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 --2- 172.21.15.16:0/2501960451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11410b3c0 0x7fc11410d880 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b0a0 tx=0x7fc10c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 -- 172.21.15.16:0/2501960451 shutdown_connections 2024-05-08T23:20:51.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 --2- 172.21.15.16:0/2501960451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11410b3c0 0x7fc11410d880 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 --2- 172.21.15.16:0/2501960451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc114101fa0 0x7fc114102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 --2- 172.21.15.16:0/2501960451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1141028c0 0x7fc11410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 -- 172.21.15.16:0/2501960451 >> 172.21.15.16:0/2501960451 conn(0x7fc1140fb7e0 msgr2=0x7fc1140fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 -- 172.21.15.16:0/2501960451 shutdown_connections 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.892+0000 7fc11b623700 1 -- 172.21.15.16:0/2501960451 wait complete. 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 Processor -- start 2024-05-08T23:20:51.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 -- start start 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 0x7fc1140ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1141028c0 0x7fc114100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11410b3c0 0x7fc114100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1141a56a0 con 0x7fc114101fa0 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1141a5810 con 0x7fc11410b3c0 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc11b623700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1141a59d0 con 0x7fc1141028c0 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc1193bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 0x7fc1140ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc1193bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 0x7fc1140ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57992/0 (socket says 172.21.15.16:57992) 2024-05-08T23:20:51.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc1193bf700 1 -- 172.21.15.16:0/923997108 learned_addr learned my addr 172.21.15.16:0/923997108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc119bc0700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11410b3c0 0x7fc114100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.893+0000 7fc118bbe700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1141028c0 0x7fc114100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc1193bf700 1 -- 172.21.15.16:0/923997108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1141028c0 msgr2=0x7fc114100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc1193bf700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1141028c0 0x7fc114100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc1193bf700 1 -- 172.21.15.16:0/923997108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11410b3c0 msgr2=0x7fc114100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc1193bf700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11410b3c0 0x7fc114100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc1193bf700 1 -- 172.21.15.16:0/923997108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009d00 con 0x7fc114101fa0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc119bc0700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11410b3c0 0x7fc114100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc1193bf700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 0x7fc1140ffbf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc11000e9a0 tx=0x7fc11000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110009790 con 0x7fc114101fa0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1141a5cb0 con 0x7fc114101fa0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc11000cbd0 con 0x7fc114101fa0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110018680 con 0x7fc114101fa0 2024-05-08T23:20:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.894+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1141a61c0 con 0x7fc114101fa0 2024-05-08T23:20:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.896+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1141064f0 con 0x7fc114101fa0 2024-05-08T23:20:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.897+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fc11000cd40 con 0x7fc114101fa0 2024-05-08T23:20:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.897+0000 7fc10a7fc700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc1000a13b0 0x7fc1000a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.898+0000 7fc118bbe700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc1000a13b0 0x7fc1000a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.898+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc110014070 con 0x7fc114101fa0 2024-05-08T23:20:51.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.898+0000 7fc118bbe700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc1000a13b0 0x7fc1000a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc104005e10 tx=0x7fc104005d60 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:51.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:51.902+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc110052d60 con 0x7fc114101fa0 2024-05-08T23:20:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.176+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1141010d0 con 0x7fc114101fa0 2024-05-08T23:20:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.177+0000 7fc10a7fc700 1 -- 172.21.15.16:0/923997108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fc110096410 con 0x7fc114101fa0 2024-05-08T23:20:52.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:20:52.178 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:20:52.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc1000a13b0 msgr2=0x7fc1000a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc1000a13b0 0x7fc1000a3870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc104005e10 tx=0x7fc104005d60 comp rx=0 tx=0).stop 2024-05-08T23:20:52.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 msgr2=0x7fc1140ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 0x7fc1140ffbf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc11000e9a0 tx=0x7fc11000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 shutdown_connections 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc1000a13b0 0x7fc1000a3870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc11410b3c0 0x7fc114100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc114101fa0 0x7fc1140ffbf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 --2- 172.21.15.16:0/923997108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1141028c0 0x7fc114100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.179+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 >> 172.21.15.16:0/923997108 conn(0x7fc1140fb7e0 msgr2=0x7fc1140fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.180+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 shutdown_connections 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.180+0000 7fc11b623700 1 -- 172.21.15.16:0/923997108 wait complete. 2024-05-08T23:20:52.181 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:20:52.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:20:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 -- 172.21.15.16:0/1222114416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 msgr2=0x7f1318073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 --2- 172.21.15.16:0/1222114416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f1318073750 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1308009990 tx=0x7f1308009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 -- 172.21.15.16:0/1222114416 shutdown_connections 2024-05-08T23:20:52.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 --2- 172.21.15.16:0/1222114416 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f131810d790 0x7f131810fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 --2- 172.21.15.16:0/1222114416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f1318073750 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 --2- 172.21.15.16:0/1222114416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1318073c90 0x7f131810d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 -- 172.21.15.16:0/1222114416 >> 172.21.15.16:0/1222114416 conn(0x7f13180fc9e0 msgr2=0x7f13180fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 -- 172.21.15.16:0/1222114416 shutdown_connections 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 -- 172.21.15.16:0/1222114416 wait complete. 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.340+0000 7f131d41e700 1 Processor -- start 2024-05-08T23:20:52.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 -- start start 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f13181994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318073c90 0x7f1318199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131810d790 0x7f131819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131819e3f0 con 0x7f1318073370 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131819e560 con 0x7f1318073c90 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f131d41e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131819e720 con 0x7f131810d790 2024-05-08T23:20:52.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f1316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f13181994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f1316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f13181994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58014/0 (socket says 172.21.15.16:58014) 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f1316ffd700 1 -- 172.21.15.16:0/3279729248 learned_addr learned my addr 172.21.15.16:0/3279729248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f13177fe700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131810d790 0x7f131819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.341+0000 7f13167fc700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318073c90 0x7f1318199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f1316ffd700 1 -- 172.21.15.16:0/3279729248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131810d790 msgr2=0x7f131819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f1316ffd700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131810d790 0x7f131819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f1316ffd700 1 -- 172.21.15.16:0/3279729248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318073c90 msgr2=0x7f1318199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f1316ffd700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318073c90 0x7f1318199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f1316ffd700 1 -- 172.21.15.16:0/3279729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080095f0 con 0x7f1318073370 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f1316ffd700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f13181994d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1308006010 tx=0x7f1308004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f12fffff700 1 -- 172.21.15.16:0/3279729248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130801d070 con 0x7f1318073370 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131819e9a0 con 0x7f1318073370 2024-05-08T23:20:52.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f12fffff700 1 -- 172.21.15.16:0/3279729248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f130800fb80 con 0x7f1318073370 2024-05-08T23:20:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f12fffff700 1 -- 172.21.15.16:0/3279729248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308022b90 con 0x7f1318073370 2024-05-08T23:20:52.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.342+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181a6240 con 0x7f1318073370 2024-05-08T23:20:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.344+0000 7f12fffff700 1 -- 172.21.15.16:0/3279729248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f130800f6f0 con 0x7f1318073370 2024-05-08T23:20:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.345+0000 7f12fffff700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f13000a13b0 0x7f13000a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.345+0000 7f12fffff700 1 -- 172.21.15.16:0/3279729248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f13080ce6c0 con 0x7f1318073370 2024-05-08T23:20:52.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.345+0000 7f13167fc700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f13000a13b0 0x7f13000a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13167fc700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f13000a13b0 0x7f13000a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f131819aa20 tx=0x7f130c00b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 Processor -- start 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 -- start start 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169b9d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169b97350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169b97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169aee150 con 0x565169b9d720 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169aebb10 con 0x565169b97890 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.346+0000 7f13295d8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169ae7400 con 0x565169b9ce00 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169b97350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169b97350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58028/0 (socket says 172.21.15.16:58028) 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 -- 172.21.15.16:0/3126903390 learned_addr learned my addr 172.21.15.16:0/3126903390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f1316ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169b9d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13177fe700 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169b97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 -- 172.21.15.16:0/3126903390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 msgr2=0x565169b9d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169b9d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 -- 172.21.15.16:0/3126903390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 msgr2=0x565169b97cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169b97cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 -- 172.21.15.16:0/3126903390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651699683b0 con 0x565169b9d720 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13177fe700 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169b97cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f13167fc700 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169b97350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f130c010f70 tx=0x7f130c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac97bf978aeaa685 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f12fd7fa700 1 -- 172.21.15.16:0/3126903390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c011db0 con 0x565169b9d720 2024-05-08T23:20:52.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f12fd7fa700 1 -- 172.21.15.16:0/3126903390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f130c015070 con 0x565169b9d720 2024-05-08T23:20:52.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.347+0000 7f12fd7fa700 1 -- 172.21.15.16:0/3126903390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c01c610 con 0x565169b9d720 2024-05-08T23:20:52.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 -- 172.21.15.16:0/3126903390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 msgr2=0x565169b97350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169b97350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f130c010f70 tx=0x7f130c013000 comp rx=0 tx=0).stop 2024-05-08T23:20:52.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 -- 172.21.15.16:0/3126903390 shutdown_connections 2024-05-08T23:20:52.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169b97cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169b9d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 --2- 172.21.15.16:0/3126903390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169b97350 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 -- 172.21.15.16:0/3126903390 >> 172.21.15.16:0/3126903390 conn(0x565169af5210 msgr2=0x565169b91df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 -- 172.21.15.16:0/3126903390 shutdown_connections 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.348+0000 7f13295d8b80 1 -- 172.21.15.16:0/3126903390 wait complete. 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 -- start start 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 Processor -- start 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169ba0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169a83560 con 0x565169b9d720 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169a83200 con 0x565169b97890 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56516996a6a0 con 0x565169b9ce00 2024-05-08T23:20:52.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.349+0000 7f13295d8b80 10 client.0 _ll_register_callbacks cb 0x565169a8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58038/0 (socket says 172.21.15.16:58038) 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 -- 172.21.15.16:0/1177773111 learned_addr learned my addr 172.21.15.16:0/1177773111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169ba0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13167fc700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 -- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 msgr2=0x565169ba0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 -- 172.21.15.16:0/1177773111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 msgr2=0x565169ba0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169ba0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 -- 172.21.15.16:0/1177773111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c010c20 con 0x565169b9d720 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169ba0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13167fc700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13177fe700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba1530 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1310009f60 tx=0x7f1310009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce230a1b8456485 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f12faff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f12faff5700 1 -- 172.21.15.16:0/1177773111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310007710 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f12faff5700 1 -- 172.21.15.16:0/1177773111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1310007870 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565169a40270 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.350+0000 7f12faff5700 1 -- 172.21.15.16:0/1177773111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1310004bb0 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 10 client.15876 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x565169914b60 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f12faff5700 1 -- 172.21.15.16:0/1177773111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565169a40270 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f12faff5700 1 -- 172.21.15.16:0/1177773111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1310014070 con 0x565169b9d720 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 10 client.15876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 20 client.15876 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169bb12b0 0x565169bb36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 4 client.15876 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565169b66810 con 0x565169bb12b0 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.351+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169bb12b0 0x565169bb36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.352+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169bb12b0 0x565169bb36d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a181c280960cad80 server_cookie=1c701da117929402 in_seq=0 out_seq=0 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.352+0000 7f12faff5700 10 client.15876 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.352+0000 7f12faff5700 1 -- 172.21.15.16:0/1177773111 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565169b66810 con 0x565169bb12b0 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.352+0000 7f12faff5700 10 client.15876 handle_command_reply: tid=0 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 client.15876 shutdown 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 msgr2=0x565169ba1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba1530 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1310009f60 tx=0x7f1310009f90 comp rx=0 tx=0).stop 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 shutdown_connections 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba1530 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169bb12b0 0x565169bb36d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b97890 0x565169ba0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1177773111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f12faff5700 0 client.15876 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 >> 172.21.15.16:0/1177773111 conn(0x565169af5210 msgr2=0x565169b915a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 shutdown_connections 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 1 -- 172.21.15.16:0/1177773111 wait complete. 2024-05-08T23:20:52.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.353+0000 7f13295d8b80 20 client.15876 trim_cache size 0 max 16384 2024-05-08T23:20:52.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:52 smithi016 ceph-mon[42541]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.367+0000 7f13295d8b80 1 Processor -- start 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.367+0000 7f13295d8b80 1 -- start start 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.367+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b97890 0x565169b9d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9ce00 0x565169bacc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13295d8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169ba08f0 con 0x565169b9d720 2024-05-08T23:20:52.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13295d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169bb8910 con 0x565169b9ce00 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13295d8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169b8c830 con 0x565169b97890 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58044/0 (socket says 172.21.15.16:58044) 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 -- 172.21.15.16:0/974417302 learned_addr learned my addr 172.21.15.16:0/974417302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f1316ffd700 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b97890 0x565169b9d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13167fc700 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9ce00 0x565169bacc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 -- 172.21.15.16:0/974417302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b97890 msgr2=0x565169b9d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b97890 0x565169b9d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 -- 172.21.15.16:0/974417302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9ce00 msgr2=0x565169bacc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9ce00 0x565169bacc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 -- 172.21.15.16:0/974417302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565169aad560 con 0x565169b9d720 2024-05-08T23:20:52.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.368+0000 7f13177fe700 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba03b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1310005bd0 tx=0x7f1310009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c341e82d2b6afe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f12fdffb700 1 -- 172.21.15.16:0/974417302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310005880 con 0x565169b9d720 2024-05-08T23:20:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f12fdffb700 1 -- 172.21.15.16:0/974417302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f131000f070 con 0x565169b9d720 2024-05-08T23:20:52.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f12fdffb700 1 -- 172.21.15.16:0/974417302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13100050d0 con 0x565169b9d720 2024-05-08T23:20:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 -- 172.21.15.16:0/974417302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 msgr2=0x565169ba03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba03b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1310005bd0 tx=0x7f1310009520 comp rx=0 tx=0).stop 2024-05-08T23:20:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 -- 172.21.15.16:0/974417302 shutdown_connections 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b9d720 0x565169ba03b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b97890 0x565169b9d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 --2- 172.21.15.16:0/974417302 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9ce00 0x565169bacc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 -- 172.21.15.16:0/974417302 >> 172.21.15.16:0/974417302 conn(0x565169af5210 msgr2=0x565169b97c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 -- 172.21.15.16:0/974417302 shutdown_connections 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.369+0000 7f13295d8b80 1 -- 172.21.15.16:0/974417302 wait complete. 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 -- start start 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 Processor -- start 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 0x565169ba8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9d720 0x565169b98a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169b99160 con 0x565169b97890 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169b992d0 con 0x565169b9d720 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x565169b99490 con 0x565169b9ce00 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f13295d8b80 10 client.0 _ll_register_callbacks cb 0x565169a8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f1316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 0x565169ba8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f1316ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 0x565169ba8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58052/0 (socket says 172.21.15.16:58052) 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.370+0000 7f1316ffd700 1 -- 172.21.15.16:0/1877054849 learned_addr learned my addr 172.21.15.16:0/1877054849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f13167fc700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f13177fe700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9d720 0x565169b98a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f1316ffd700 1 -- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 msgr2=0x565169ba86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f1316ffd700 1 -- 172.21.15.16:0/1877054849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9d720 msgr2=0x565169b98a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9d720 0x565169b98a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f1316ffd700 1 -- 172.21.15.16:0/1877054849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565169914b60 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f13177fe700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9d720 0x565169b98a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f1316ffd700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 0x565169ba8170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f130809b210 tx=0x7f1308009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25d59f474073463 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f12faff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f12faff5700 1 -- 172.21.15.16:0/1877054849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130809b9d0 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f12faff5700 1 -- 172.21.15.16:0/1877054849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f130809b2d0 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f130c015070 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f12faff5700 1 -- 172.21.15.16:0/1877054849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f130809f100 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f13295d8b80 10 client.15882 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.371+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56516996ccd0 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f12faff5700 1 -- 172.21.15.16:0/1877054849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f130807f080 con 0x565169b97890 2024-05-08T23:20:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f12faff5700 1 -- 172.21.15.16:0/1877054849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f130809b530 con 0x565169b97890 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13295d8b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13295d8b80 10 client.15882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13295d8b80 20 client.15882 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169c077b0 0x565169c09ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13295d8b80 4 client.15882 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565169adfec0 con 0x565169c077b0 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.372+0000 7f13167fc700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169c077b0 0x565169c09ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f12faff5700 10 client.15882 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13167fc700 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169c077b0 0x565169c09ba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c5e61f67f6be0b1 server_cookie=733cdb3973c78992 in_seq=0 out_seq=0 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f12faff5700 1 -- 172.21.15.16:0/1877054849 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565169adfec0 con 0x565169c077b0 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f12faff5700 10 client.15882 handle_command_reply: tid=0 2024-05-08T23:20:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 client.15882 shutdown 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 msgr2=0x565169ba8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 0x565169ba8170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f130809b210 tx=0x7f1308009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 shutdown_connections 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x565169c077b0 0x565169c09ba0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x565169b9d720 0x565169b98a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565169b97890 0x565169ba8170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f12faff5700 0 client.15882 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 --2- 172.21.15.16:0/1877054849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565169b9ce00 0x565169ba86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 >> 172.21.15.16:0/1877054849 conn(0x565169af5210 msgr2=0x565169b9dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 shutdown_connections 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 1 -- 172.21.15.16:0/1877054849 wait complete. 2024-05-08T23:20:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.373+0000 7f13295d8b80 20 client.15882 trim_cache size 0 max 16384 2024-05-08T23:20:52.381 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f13000a13b0 msgr2=0x7f13000a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f13000a13b0 0x7f13000a3870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f131819aa20 tx=0x7f130c00b410 comp rx=0 tx=0).stop 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 msgr2=0x7f13181994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f13181994d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1308006010 tx=0x7f1308004730 comp rx=0 tx=0).stop 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 shutdown_connections 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f13000a13b0 0x7f13000a3870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f131810d790 0x7f131819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318073370 0x7f13181994d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 --2- 172.21.15.16:0/3279729248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318073c90 0x7f1318199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 >> 172.21.15.16:0/3279729248 conn(0x7f13180fc9e0 msgr2=0x7f13180fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 shutdown_connections 2024-05-08T23:20:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.374+0000 7f131d41e700 1 -- 172.21.15.16:0/3279729248 wait complete. 2024-05-08T23:20:52.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:20:52.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:52 smithi189 ceph-mon[46919]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:52 smithi155 ceph-mon[47105]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 -- 172.21.15.16:0/2503030278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0068c90 msgr2=0x7f01f0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/2503030278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0068c90 0x7f01f0105b60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f01e40099c0 tx=0x7f01e4009cd0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 -- 172.21.15.16:0/2503030278 shutdown_connections 2024-05-08T23:20:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/2503030278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0106290 0x7f01f010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/2503030278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f00682e0 0x7f01f00686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/2503030278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0068c90 0x7f01f0105b60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 -- 172.21.15.16:0/2503030278 >> 172.21.15.16:0/2503030278 conn(0x7f01f0076f40 msgr2=0x7f01f0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 -- 172.21.15.16:0/2503030278 shutdown_connections 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.540+0000 7f01f5f9a700 1 -- 172.21.15.16:0/2503030278 wait complete. 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 Processor -- start 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 -- start start 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 0x7f01f01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0106290 0x7f01f0197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f019bbd0 0x7f01f019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f019c650 con 0x7f01f00682e0 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f019c7c0 con 0x7f01f0106290 2024-05-08T23:20:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01f5f9a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f019c980 con 0x7f01f019bbd0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 0x7f01f01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 0x7f01f01972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58064/0 (socket says 172.21.15.16:58064) 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01ef7fe700 1 -- 172.21.15.16:0/249237741 learned_addr learned my addr 172.21.15.16:0/249237741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.541+0000 7f01eeffd700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0106290 0x7f01f0197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01effff700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f019bbd0 0x7f01f019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ef7fe700 1 -- 172.21.15.16:0/249237741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f019bbd0 msgr2=0x7f01f019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f019bbd0 0x7f01f019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ef7fe700 1 -- 172.21.15.16:0/249237741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0106290 msgr2=0x7f01f0197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0106290 0x7f01f0197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ef7fe700 1 -- 172.21.15.16:0/249237741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40095f0 con 0x7f01f00682e0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01effff700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f019bbd0 0x7f01f019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 0x7f01f01972f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d950 tx=0x7f01e000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ecff9700 1 -- 172.21.15.16:0/249237741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0009b40 con 0x7f01f00682e0 2024-05-08T23:20:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f01a3ee0 con 0x7f01f00682e0 2024-05-08T23:20:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ecff9700 1 -- 172.21.15.16:0/249237741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e0009ca0 con 0x7f01f00682e0 2024-05-08T23:20:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01ecff9700 1 -- 172.21.15.16:0/249237741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000f620 con 0x7f01f00682e0 2024-05-08T23:20:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.542+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01a4310 con 0x7f01f00682e0 2024-05-08T23:20:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.546+0000 7f01ecff9700 1 -- 172.21.15.16:0/249237741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f01e0010460 con 0x7f01f00682e0 2024-05-08T23:20:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.546+0000 7f01ecff9700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01d80a13b0 0x7f01d80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.546+0000 7f01eeffd700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01d80a13b0 0x7f01d80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.546+0000 7f01ecff9700 1 -- 172.21.15.16:0/249237741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f01e00cd5a0 con 0x7f01f00682e0 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f01eeffd700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01d80a13b0 0x7f01d80a3870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f01e4000c00 tx=0x7f01e4009f90 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 Processor -- start 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 -- start start 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x5597227305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x55972272a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x55972272b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972267d150 con 0x559722730b30 2024-05-08T23:20:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972267ab10 con 0x559722730210 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.547+0000 7f0202154b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722676400 con 0x55972272aca0 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x55972272a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x55972272a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58080/0 (socket says 172.21.15.16:58080) 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 -- 172.21.15.16:0/1384153920 learned_addr learned my addr 172.21.15.16:0/1384153920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x5597227305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01effff700 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x55972272b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 -- 172.21.15.16:0/1384153920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 msgr2=0x55972272b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x55972272b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 -- 172.21.15.16:0/1384153920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 msgr2=0x5597227305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x5597227305f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 -- 172.21.15.16:0/1384153920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597224f73b0 con 0x559722730b30 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x5597227305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:52.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01eeffd700 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x55972272a760 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f01f006a6d0 tx=0x7f01e401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ff9a8732344a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01d5ffb700 1 -- 172.21.15.16:0/1384153920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e402b070 con 0x559722730b30 2024-05-08T23:20:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01d5ffb700 1 -- 172.21.15.16:0/1384153920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e401e040 con 0x559722730b30 2024-05-08T23:20:52.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.548+0000 7f01d5ffb700 1 -- 172.21.15.16:0/1384153920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40049e0 con 0x559722730b30 2024-05-08T23:20:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 -- 172.21.15.16:0/1384153920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 msgr2=0x55972272a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x55972272a760 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f01f006a6d0 tx=0x7f01e401f000 comp rx=0 tx=0).stop 2024-05-08T23:20:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 -- 172.21.15.16:0/1384153920 shutdown_connections 2024-05-08T23:20:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x55972272b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x5597227305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 --2- 172.21.15.16:0/1384153920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x55972272a760 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 -- 172.21.15.16:0/1384153920 >> 172.21.15.16:0/1384153920 conn(0x559722684210 msgr2=0x559722720df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 -- 172.21.15.16:0/1384153920 shutdown_connections 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.549+0000 7f0202154b80 1 -- 172.21.15.16:0/1384153920 wait complete. 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.550+0000 7f0202154b80 1 -- start start 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.550+0000 7f0202154b80 1 Processor -- start 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.550+0000 7f0202154b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722733e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.550+0000 7f0202154b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730210 0x559722734380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.550+0000 7f0202154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x5597227348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f0202154b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722612560 con 0x559722730210 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f0202154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722612200 con 0x559722730b30 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f0202154b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597224f96a0 con 0x55972272aca0 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f0202154b80 10 client.0 _ll_register_callbacks cb 0x55972261bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722733e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722733e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:39680/0 (socket says 172.21.15.16:39680) 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01ef7fe700 1 -- 172.21.15.16:0/4104038050 learned_addr learned my addr 172.21.15.16:0/4104038050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730210 0x559722734380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01effff700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x5597227348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 -- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 msgr2=0x559722733e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722733e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 -- 172.21.15.16:0/4104038050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 msgr2=0x5597227348c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x5597227348c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 -- 172.21.15.16:0/4104038050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01a4310 con 0x559722730210 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722733e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01eeffd700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730210 0x559722734380 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009990 tx=0x7f01e4004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45dbd5bfb018662 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.551+0000 7f01ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f01ceffd700 1 -- 172.21.15.16:0/4104038050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e402b070 con 0x559722730210 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f01ceffd700 1 -- 172.21.15.16:0/4104038050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f01e400b9a0 con 0x559722730210 2024-05-08T23:20:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597225cf270 con 0x559722730210 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f01ceffd700 1 -- 172.21.15.16:0/4104038050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e4005a00 con 0x559722730210 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 10 client.15894 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f01f010e650 con 0x559722730210 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f01ceffd700 1 -- 172.21.15.16:0/4104038050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5597225cf270 con 0x559722730210 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f01ceffd700 1 -- 172.21.15.16:0/4104038050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01e400fb60 con 0x559722730210 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 20 client.15894 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x559722743fd0 0x5597227463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 4 client.15894 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.552+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597226f5810 con 0x559722743fd0 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.553+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x559722743fd0 0x5597227463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.553+0000 7f01ceffd700 10 client.15894 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.553+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x559722743fd0 0x5597227463f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed2ac627f1c784f9 server_cookie=c55f3c2649522fed in_seq=0 out_seq=0 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f01ceffd700 1 -- 172.21.15.16:0/4104038050 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597226f5810 con 0x559722743fd0 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f01ceffd700 10 client.15894 handle_command_reply: tid=0 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 client.15894 shutdown 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730210 msgr2=0x559722734380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730210 0x559722734380 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009990 tx=0x7f01e4004940 comp rx=0 tx=0).stop 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 shutdown_connections 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x559722743fd0 0x5597227463f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x5597227348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722733e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f01ceffd700 0 client.15894 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 --2- 172.21.15.16:0/4104038050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730210 0x559722734380 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 >> 172.21.15.16:0/4104038050 conn(0x559722684210 msgr2=0x5597227205a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 shutdown_connections 2024-05-08T23:20:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 1 -- 172.21.15.16:0/4104038050 wait complete. 2024-05-08T23:20:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.554+0000 7f0202154b80 20 client.15894 trim_cache size 0 max 16384 2024-05-08T23:20:52.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.581+0000 7f0202154b80 1 Processor -- start 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.581+0000 7f0202154b80 1 -- start start 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f0202154b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 0x55972272a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f0202154b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559722730210 0x559722720d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f0202154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x55972273b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f0202154b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972271bbd0 con 0x55972272aca0 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f0202154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722735460 con 0x559722730b30 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f0202154b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55972271ddf0 con 0x559722730210 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 0x55972272a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 0x55972272a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58100/0 (socket says 172.21.15.16:58100) 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 -- 172.21.15.16:0/3454550034 learned_addr learned my addr 172.21.15.16:0/3454550034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01eeffd700 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559722730210 0x559722720d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01effff700 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x55972273b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 -- 172.21.15.16:0/3454550034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559722730210 msgr2=0x559722720d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559722730210 0x559722720d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 -- 172.21.15.16:0/3454550034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 msgr2=0x55972273b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x55972273b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.582+0000 7f01ef7fe700 1 -- 172.21.15.16:0/3454550034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597224a3b60 con 0x55972272aca0 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.583+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 0x55972272a760 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f01e0096620 tx=0x7f01e001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca1467d6dfcf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.583+0000 7f01d67fc700 1 -- 172.21.15.16:0/3454550034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e009c040 con 0x55972272aca0 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.583+0000 7f01d67fc700 1 -- 172.21.15.16:0/3454550034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e009df50 con 0x55972272aca0 2024-05-08T23:20:52.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.583+0000 7f01d67fc700 1 -- 172.21.15.16:0/3454550034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00cd440 con 0x55972272aca0 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 -- 172.21.15.16:0/3454550034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 msgr2=0x55972272a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 0x55972272a760 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f01e0096620 tx=0x7f01e001d040 comp rx=0 tx=0).stop 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 -- 172.21.15.16:0/3454550034 shutdown_connections 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730b30 0x55972273b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55972272aca0 0x55972272a760 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 --2- 172.21.15.16:0/3454550034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559722730210 0x559722720d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 -- 172.21.15.16:0/3454550034 >> 172.21.15.16:0/3454550034 conn(0x559722684210 msgr2=0x55972272b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 -- 172.21.15.16:0/3454550034 shutdown_connections 2024-05-08T23:20:52.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 -- 172.21.15.16:0/3454550034 wait complete. 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.584+0000 7f0202154b80 1 -- start start 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 Processor -- start 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722737680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x559722737bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x5597227994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722738280 con 0x559722730b30 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597227999e0 con 0x559722730210 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559722799b20 con 0x55972272aca0 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f0202154b80 10 client.0 _ll_register_callbacks cb 0x55972261bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f01effff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x5597227994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f01effff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x5597227994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58102/0 (socket says 172.21.15.16:58102) 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f01effff700 1 -- 172.21.15.16:0/109007947 learned_addr learned my addr 172.21.15.16:0/109007947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.585+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722737680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01eeffd700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x559722737bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01effff700 1 -- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 msgr2=0x559722737680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01effff700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722737680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01effff700 1 -- 172.21.15.16:0/109007947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 msgr2=0x559722737bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01effff700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x559722737bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01effff700 1 -- 172.21.15.16:0/109007947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01a3ee0 con 0x559722730b30 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722737680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01eeffd700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x559722737bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01effff700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x5597227994a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f01e8007ac0 tx=0x7f01e8007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3af820ec07fd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01ceffd700 1 -- 172.21.15.16:0/109007947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8003c10 con 0x559722730b30 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01ceffd700 1 -- 172.21.15.16:0/109007947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f01e8003d70 con 0x559722730b30 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01e401e040 con 0x559722730b30 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f01ceffd700 1 -- 172.21.15.16:0/109007947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e8004bb0 con 0x559722730b30 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f0202154b80 10 client.15900 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:52.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.586+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55972263c560 con 0x559722730b30 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f01ceffd700 1 -- 172.21.15.16:0/109007947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e401e040 con 0x559722730b30 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f01ceffd700 1 -- 172.21.15.16:0/109007947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01e801e030 con 0x559722730b30 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f0202154b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f0202154b80 10 client.15900 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f0202154b80 20 client.15900 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f0202154b80 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55972279b3f0 0x55972279d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f0202154b80 4 client.15900 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55972266eec0 con 0x55972279b3f0 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.587+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55972279b3f0 0x55972279d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f01ef7fe700 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55972279b3f0 0x55972279d810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33e00c552513c814 server_cookie=53daa712a0b41f75 in_seq=0 out_seq=0 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f01ceffd700 10 client.15900 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f01ceffd700 1 -- 172.21.15.16:0/109007947 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55972266eec0 con 0x55972279b3f0 2024-05-08T23:20:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f01ceffd700 10 client.15900 handle_command_reply: tid=0 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 client.15900 shutdown 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 msgr2=0x5597227994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x5597227994a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f01e8007ac0 tx=0x7f01e8007e80 comp rx=0 tx=0).stop 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 shutdown_connections 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559722730b30 0x5597227994a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55972272aca0 0x559722737680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55972279b3f0 0x55972279d810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 --2- 172.21.15.16:0/109007947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559722730210 0x559722737bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 >> 172.21.15.16:0/109007947 conn(0x559722684210 msgr2=0x5597227325e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f01ceffd700 0 client.15900 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 shutdown_connections 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.588+0000 7f0202154b80 1 -- 172.21.15.16:0/109007947 wait complete. 2024-05-08T23:20:52.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f0202154b80 20 client.15900 trim_cache size 0 max 16384 2024-05-08T23:20:52.595 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:20:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01d80a13b0 msgr2=0x7f01d80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01d80a13b0 0x7f01d80a3870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f01e4000c00 tx=0x7f01e4009f90 comp rx=0 tx=0).stop 2024-05-08T23:20:52.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 msgr2=0x7f01f01972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 0x7f01f01972f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d950 tx=0x7f01e000dd10 comp rx=0 tx=0).stop 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 shutdown_connections 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01d80a13b0 0x7f01d80a3870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01f019bbd0 0x7f01f019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f00682e0 0x7f01f01972f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 --2- 172.21.15.16:0/249237741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01f0106290 0x7f01f0197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 >> 172.21.15.16:0/249237741 conn(0x7f01f0076f40 msgr2=0x7f01f00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 shutdown_connections 2024-05-08T23:20:52.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.589+0000 7f01f5f9a700 1 -- 172.21.15.16:0/249237741 wait complete. 2024-05-08T23:20:52.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 -- 172.21.15.16:0/3115677543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 msgr2=0x7f885c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 --2- 172.21.15.16:0/3115677543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c069800 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f884c009990 tx=0x7f884c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 -- 172.21.15.16:0/3115677543 shutdown_connections 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 --2- 172.21.15.16:0/3115677543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c1062a0 0x7f885c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 --2- 172.21.15.16:0/3115677543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c069800 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 --2- 172.21.15.16:0/3115677543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c069dd0 0x7f885c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 -- 172.21.15.16:0/3115677543 >> 172.21.15.16:0/3115677543 conn(0x7f885c076ba0 msgr2=0x7f885c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 -- 172.21.15.16:0/3115677543 shutdown_connections 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.754+0000 7f886190c700 1 -- 172.21.15.16:0/3115677543 wait complete. 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.755+0000 7f886190c700 1 Processor -- start 2024-05-08T23:20:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.755+0000 7f886190c700 1 -- start start 2024-05-08T23:20:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.755+0000 7f886190c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.755+0000 7f886190c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c1062a0 0x7f885c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f886190c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c19de00 0x7f885c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f886190c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c19e880 con 0x7f885c069420 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f886190c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c19e9f0 con 0x7f885c19de00 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f886190c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c19ebb0 con 0x7f885c1062a0 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58116/0 (socket says 172.21.15.16:58116) 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 -- 172.21.15.16:0/117342066 learned_addr learned my addr 172.21.15.16:0/117342066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885a7fc700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c1062a0 0x7f885c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885b7fe700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c19de00 0x7f885c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 -- 172.21.15.16:0/117342066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c1062a0 msgr2=0x7f885c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c1062a0 0x7f885c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 -- 172.21.15.16:0/117342066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c19de00 msgr2=0x7f885c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c19de00 0x7f885c19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 -- 172.21.15.16:0/117342066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0095f0 con 0x7f885c069420 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885b7fe700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c19de00 0x7f885c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f885affd700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c199570 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f884c005770 tx=0x7f884c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f886090a700 1 -- 172.21.15.16:0/117342066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c01d070 con 0x7f885c069420 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.756+0000 7f886090a700 1 -- 172.21.15.16:0/117342066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884c00bea0 con 0x7f885c069420 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.757+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c1a60b0 con 0x7f885c069420 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.757+0000 7f886090a700 1 -- 172.21.15.16:0/117342066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c022e40 con 0x7f885c069420 2024-05-08T23:20:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.757+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c1a64d0 con 0x7f885c069420 2024-05-08T23:20:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.759+0000 7f886090a700 1 -- 172.21.15.16:0/117342066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f884c02c440 con 0x7f885c069420 2024-05-08T23:20:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f886090a700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88440a13b0 0x7f88440a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f885a7fc700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88440a13b0 0x7f88440a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f886090a700 1 -- 172.21.15.16:0/117342066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f884c0ce640 con 0x7f885c069420 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f886dac6b80 1 Processor -- start 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f886dac6b80 1 -- start start 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 0x564c89b73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.760+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b6b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885a7fc700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88440a13b0 0x7f88440a3870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f885c19aa70 tx=0x7f88500073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b8bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f886dac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89ac5150 con 0x564c89b72f50 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f886dac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89ac2b10 con 0x564c89b6bcf0 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f886dac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89abe400 con 0x564c89b6b330 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 0x564c89b73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 0x564c89b73330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58130/0 (socket says 172.21.15.16:58130) 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 -- 172.21.15.16:0/2210724621 learned_addr learned my addr 172.21.15.16:0/2210724621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885b7fe700 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b8bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885a7fc700 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b6b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 -- 172.21.15.16:0/2210724621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 msgr2=0x564c89b6b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b6b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 -- 172.21.15.16:0/2210724621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 msgr2=0x564c89b8bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b8bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 -- 172.21.15.16:0/2210724621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8993f3b0 con 0x564c89b72f50 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885a7fc700 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b6b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f885affd700 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 0x564c89b73330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f884c0cb9e0 tx=0x7f884c09ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21ba501c170ce87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f8841ffb700 1 -- 172.21.15.16:0/2210724621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c09b250 con 0x564c89b72f50 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f8841ffb700 1 -- 172.21.15.16:0/2210724621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884c09f080 con 0x564c89b72f50 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.761+0000 7f8841ffb700 1 -- 172.21.15.16:0/2210724621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c09b6f0 con 0x564c89b72f50 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 -- 172.21.15.16:0/2210724621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 msgr2=0x564c89b73330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 0x564c89b73330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f884c0cb9e0 tx=0x7f884c09ae10 comp rx=0 tx=0).stop 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 -- 172.21.15.16:0/2210724621 shutdown_connections 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b8bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b72f50 0x564c89b73330 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2210724621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b6b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 -- 172.21.15.16:0/2210724621 >> 172.21.15.16:0/2210724621 conn(0x564c89acc210 msgr2=0x564c89b68df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 -- 172.21.15.16:0/2210724621 shutdown_connections 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.762+0000 7f886dac6b80 1 -- 172.21.15.16:0/2210724621 wait complete. 2024-05-08T23:20:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 -- start start 2024-05-08T23:20:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 Processor -- start 2024-05-08T23:20:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 0x564c89b7d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b72f50 0x564c89b77b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89a5a560 con 0x564c89b6b330 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89a5a200 con 0x564c89b6bcf0 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.763+0000 7f886dac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c899416a0 con 0x564c89b72f50 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f886dac6b80 10 client.0 _ll_register_callbacks cb 0x564c89a63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 0x564c89b7d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 0x564c89b7d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58136/0 (socket says 172.21.15.16:58136) 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 -- 172.21.15.16:0/2193757597 learned_addr learned my addr 172.21.15.16:0/2193757597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885a7fc700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885b7fe700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b72f50 0x564c89b77b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 -- 172.21.15.16:0/2193757597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b72f50 msgr2=0x564c89b77b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b72f50 0x564c89b77b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 -- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 msgr2=0x564c89b7ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b7ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 -- 172.21.15.16:0/2193757597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c1a64d0 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f885affd700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 0x564c89b7d890 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f884c009350 tx=0x7f884c0ce510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d15e6fe9d900b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f883affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f883affd700 1 -- 172.21.15.16:0/2193757597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0b1030 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f883affd700 1 -- 172.21.15.16:0/2193757597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f884c0b2070 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.764+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c89a17270 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f883affd700 1 -- 172.21.15.16:0/2193757597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f884c0a3640 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 10 client.15912 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564c898ebb60 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f883affd700 1 -- 172.21.15.16:0/2193757597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884c080020 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f883affd700 1 -- 172.21.15.16:0/2193757597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f884c0a10c0 con 0x564c89b6b330 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 20 client.15912 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89b87960 0x564c89b89d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 4 client.15912 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c89b3d810 con 0x564c89b87960 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.765+0000 7f885a7fc700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89b87960 0x564c89b89d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.766+0000 7f883affd700 10 client.15912 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.766+0000 7f885a7fc700 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89b87960 0x564c89b89d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9baafa57fd89d0a8 server_cookie=a4add55e8e71beed in_seq=0 out_seq=0 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.766+0000 7f883affd700 1 -- 172.21.15.16:0/2193757597 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c89b3d810 con 0x564c89b87960 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.766+0000 7f883affd700 10 client.15912 handle_command_reply: tid=0 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.766+0000 7f886dac6b80 1 client.15912 shutdown 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 msgr2=0x564c89b7d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 0x564c89b7d890 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f884c009350 tx=0x7f884c0ce510 comp rx=0 tx=0).stop 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 shutdown_connections 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b72f50 0x564c89b77b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6b330 0x564c89b7d890 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89b87960 0x564c89b89d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 --2- 172.21.15.16:0/2193757597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b6bcf0 0x564c89b7ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 >> 172.21.15.16:0/2193757597 conn(0x564c89acc210 msgr2=0x564c89b685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f883affd700 0 client.15912 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 shutdown_connections 2024-05-08T23:20:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 1 -- 172.21.15.16:0/2193757597 wait complete. 2024-05-08T23:20:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.767+0000 7f886dac6b80 20 client.15912 trim_cache size 0 max 16384 2024-05-08T23:20:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.782+0000 7f886dac6b80 1 Processor -- start 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 -- start start 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b76ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b77b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89b7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89b6ae50 con 0x564c89b6bcf0 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89b78070 con 0x564c89b72f50 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f886dac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89b6aa20 con 0x564c89b6b330 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b77b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b77b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58146/0 (socket says 172.21.15.16:58146) 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 -- 172.21.15.16:0/1261277964 learned_addr learned my addr 172.21.15.16:0/1261277964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885affd700 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b76ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885b7fe700 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89b7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 -- 172.21.15.16:0/1261277964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 msgr2=0x564c89b76ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b76ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 -- 172.21.15.16:0/1261277964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 msgr2=0x564c89b7d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89b7d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.783+0000 7f885a7fc700 1 -- 172.21.15.16:0/1261277964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c89a84560 con 0x564c89b6bcf0 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.784+0000 7f885a7fc700 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b77b30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f885c067dd0 tx=0x7f8850013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79003f531a7b8b5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.784+0000 7f88427fc700 1 -- 172.21.15.16:0/1261277964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850026d00 con 0x564c89b6bcf0 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.784+0000 7f88427fc700 1 -- 172.21.15.16:0/1261277964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8850026e60 con 0x564c89b6bcf0 2024-05-08T23:20:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.784+0000 7f88427fc700 1 -- 172.21.15.16:0/1261277964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850003b40 con 0x564c89b6bcf0 2024-05-08T23:20:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.784+0000 7f886dac6b80 1 -- 172.21.15.16:0/1261277964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 msgr2=0x564c89b77b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.784+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b77b30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f885c067dd0 tx=0x7f8850013000 comp rx=0 tx=0).stop 2024-05-08T23:20:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 -- 172.21.15.16:0/1261277964 shutdown_connections 2024-05-08T23:20:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89b7d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b76ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1261277964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b77b30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 -- 172.21.15.16:0/1261277964 >> 172.21.15.16:0/1261277964 conn(0x564c89acc210 msgr2=0x564c89b77420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 -- 172.21.15.16:0/1261277964 shutdown_connections 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 -- 172.21.15.16:0/1261277964 wait complete. 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 -- start start 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.785+0000 7f886dac6b80 1 Processor -- start 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b81dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89bdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89b82c20 con 0x564c89b6bcf0 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89bdd360 con 0x564c89b72f50 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c89bdd4a0 con 0x564c89b6b330 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f886dac6b80 10 client.0 _ll_register_callbacks cb 0x564c89a63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b82310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58156/0 (socket says 172.21.15.16:58156) 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 -- 172.21.15.16:0/1068624077 learned_addr learned my addr 172.21.15.16:0/1068624077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885affd700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b81dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885b7fe700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89bdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 -- 172.21.15.16:0/1068624077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 msgr2=0x564c89b81dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b81dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 -- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 msgr2=0x564c89bdce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89bdce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.786+0000 7f885a7fc700 1 -- 172.21.15.16:0/1068624077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8850004100 con 0x564c89b6bcf0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f885a7fc700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b82310 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8850004580 tx=0x7f88500107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86fa9f6d442feb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f883affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f883affd700 1 -- 172.21.15.16:0/1068624077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850026330 con 0x564c89b6bcf0 2024-05-08T23:20:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f883affd700 1 -- 172.21.15.16:0/1068624077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f8850017040 con 0x564c89b6bcf0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f884c09f080 con 0x564c89b6bcf0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f883affd700 1 -- 172.21.15.16:0/1068624077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8850026490 con 0x564c89b6bcf0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f886dac6b80 10 client.15918 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564c89943cd0 con 0x564c89b6bcf0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f883affd700 1 -- 172.21.15.16:0/1068624077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884c09f080 con 0x564c89b6bcf0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f883affd700 1 -- 172.21.15.16:0/1068624077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f885002d070 con 0x564c89b6bcf0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f886dac6b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.787+0000 7f886dac6b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f886dac6b80 20 client.15918 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89bde9f0 0x564c89be0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f886dac6b80 4 client.15918 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c89ab6ec0 con 0x564c89bde9f0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f885affd700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89bde9f0 0x564c89be0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f883affd700 10 client.15918 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f885affd700 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89bde9f0 0x564c89be0e10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e4ca5a75af20a4 server_cookie=18903d706922aede in_seq=0 out_seq=0 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f883affd700 1 -- 172.21.15.16:0/1068624077 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c89ab6ec0 con 0x564c89bde9f0 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f883affd700 10 client.15918 handle_command_reply: tid=0 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.788+0000 7f886dac6b80 1 client.15918 shutdown 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 msgr2=0x564c89b82310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b82310 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8850004580 tx=0x7f88500107f0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 shutdown_connections 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c89bde9f0 0x564c89be0e10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c89b72f50 0x564c89bdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c89b6b330 0x564c89b81dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f883affd700 0 client.15918 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 --2- 172.21.15.16:0/1068624077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c89b6bcf0 0x564c89b82310 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 >> 172.21.15.16:0/1068624077 conn(0x564c89acc210 msgr2=0x564c89b7d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 shutdown_connections 2024-05-08T23:20:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 1 -- 172.21.15.16:0/1068624077 wait complete. 2024-05-08T23:20:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886dac6b80 20 client.15918 trim_cache size 0 max 16384 2024-05-08T23:20:52.795 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88440a13b0 msgr2=0x7f88440a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88440a13b0 0x7f88440a3870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f885c19aa70 tx=0x7f88500073d0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 msgr2=0x7f885c199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c199570 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f884c005770 tx=0x7f884c0057a0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 shutdown_connections 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88440a13b0 0x7f88440a3870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f885c19de00 0x7f885c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f885c069420 0x7f885c199570 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 --2- 172.21.15.16:0/117342066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c1062a0 0x7f885c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.789+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 >> 172.21.15.16:0/117342066 conn(0x7f885c076ba0 msgr2=0x7f885c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:52.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.790+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 shutdown_connections 2024-05-08T23:20:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:52.790+0000 7f886190c700 1 -- 172.21.15.16:0/117342066 wait complete. 2024-05-08T23:20:53.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:53 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/923997108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:53.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:53 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/923997108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:53 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/923997108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:20:53.805 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:20:53.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:20:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/3133405247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae81000f0 msgr2=0x7fcae8100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/3133405247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae81000f0 0x7fcae8100570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0099c0 tx=0x7fcadc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/3133405247 shutdown_connections 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/3133405247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8100ab0 0x7fcae810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/3133405247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae8105f90 0x7fcae8106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/3133405247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae81000f0 0x7fcae8100570 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/3133405247 >> 172.21.15.16:0/3133405247 conn(0x7fcae8076ba0 msgr2=0x7fcae8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/3133405247 shutdown_connections 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/3133405247 wait complete. 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 Processor -- start 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.958+0000 7fcaeeb9d700 1 -- start start 2024-05-08T23:20:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaeeb9d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae81000f0 0x7fcae8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaeeb9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 0x7fcae81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaeeb9d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8105f90 0x7fcae819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaeeb9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819c190 con 0x7fcae8100ab0 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaeeb9d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819c300 con 0x7fcae8105f90 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaeeb9d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819c4c0 con 0x7fcae81000f0 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 0x7fcae81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 0x7fcae81977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58172/0 (socket says 172.21.15.16:58172) 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaec939700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae81000f0 0x7fcae8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 -- 172.21.15.16:0/2062071153 learned_addr learned my addr 172.21.15.16:0/2062071153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaed13a700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8105f90 0x7fcae819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 -- 172.21.15.16:0/2062071153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae81000f0 msgr2=0x7fcae8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae81000f0 0x7fcae8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 -- 172.21.15.16:0/2062071153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8105f90 msgr2=0x7fcae819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8105f90 0x7fcae819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 -- 172.21.15.16:0/2062071153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0095f0 con 0x7fcae8100ab0 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcaec939700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae81000f0 0x7fcae8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.959+0000 7fcae7fff700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 0x7fcae81977b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00f690 tx=0x7fcadc00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.960+0000 7fcae5ffb700 1 -- 172.21.15.16:0/2062071153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc01c070 con 0x7fcae8100ab0 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.960+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae819c740 con 0x7fcae8100ab0 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.960+0000 7fcae5ffb700 1 -- 172.21.15.16:0/2062071153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcadc00fdb0 con 0x7fcae8100ab0 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.960+0000 7fcae5ffb700 1 -- 172.21.15.16:0/2062071153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc004710 con 0x7fcae8100ab0 2024-05-08T23:20:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.960+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae81a3fe0 con 0x7fcae8100ab0 2024-05-08T23:20:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.963+0000 7fcae5ffb700 1 -- 172.21.15.16:0/2062071153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7fcadc020070 con 0x7fcae8100ab0 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 Processor -- start 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 -- start start 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 0x55943bbf16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf1c00 0x55943bbeb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bb3e1d0 con 0x55943bbf12e0 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bb3bb90 con 0x55943bbf1c00 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcafad57b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bb37480 con 0x55943bbebd70 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcae5ffb700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad00a13b0 0x7fcad00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 0x55943bbf16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 0x55943bbf16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58174/0 (socket says 172.21.15.16:58174) 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcaed13a700 1 -- 172.21.15.16:0/4267922745 learned_addr learned my addr 172.21.15.16:0/4267922745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcaed13a700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad00a13b0 0x7fcad00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcaec939700 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf1c00 0x55943bbeb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcae5ffb700 1 -- 172.21.15.16:0/2062071153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcadc0cdce0 con 0x7fcae8100ab0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.964+0000 7fcae7fff700 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 -- 172.21.15.16:0/4267922745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 msgr2=0x55943bbec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 -- 172.21.15.16:0/4267922745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf1c00 msgr2=0x55943bbeb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf1c00 0x55943bbeb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 -- 172.21.15.16:0/4267922745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943b9b8430 con 0x55943bbf12e0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 0x55943bbf16c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcae0008fa0 tx=0x7fcae001a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2502426ddaa5e569 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaed13a700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad00a13b0 0x7fcad00a3870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009fd0 tx=0x7fcae001e000 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaceffd700 1 -- 172.21.15.16:0/4267922745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0027070 con 0x55943bbf12e0 2024-05-08T23:20:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaceffd700 1 -- 172.21.15.16:0/4267922745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae000f040 con 0x55943bbf12e0 2024-05-08T23:20:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.965+0000 7fcaceffd700 1 -- 172.21.15.16:0/4267922745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000e030 con 0x55943bbf12e0 2024-05-08T23:20:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 -- 172.21.15.16:0/4267922745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 msgr2=0x55943bbf16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 0x55943bbf16c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcae0008fa0 tx=0x7fcae001a6c0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 -- 172.21.15.16:0/4267922745 shutdown_connections 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf12e0 0x55943bbf16c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4267922745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf1c00 0x55943bbeb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 -- 172.21.15.16:0/4267922745 >> 172.21.15.16:0/4267922745 conn(0x55943bb45290 msgr2=0x55943bbe1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 -- 172.21.15.16:0/4267922745 shutdown_connections 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.966+0000 7fcafad57b80 1 -- 172.21.15.16:0/4267922745 wait complete. 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.967+0000 7fcafad57b80 1 -- start start 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.967+0000 7fcafad57b80 1 Processor -- start 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbf4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbf54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbf5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bad35e0 con 0x55943bbf1c00 2024-05-08T23:20:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bad3280 con 0x55943bbf12e0 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943b9ba720 con 0x55943bbebd70 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcafad57b80 10 client.0 _ll_register_callbacks cb 0x55943badcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbf5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbf5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58180/0 (socket says 172.21.15.16:58180) 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 -- 172.21.15.16:0/4185456604 learned_addr learned my addr 172.21.15.16:0/4185456604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaec939700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbf4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcae7fff700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbf54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 -- 172.21.15.16:0/4185456604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 msgr2=0x55943bbf4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbf4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 -- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 msgr2=0x55943bbf54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbf54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 -- 172.21.15.16:0/4185456604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae0008b50 con 0x55943bbf1c00 2024-05-08T23:20:53.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcae7fff700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbf54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcaed13a700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbf5a10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcae0003dc0 tx=0x7fcae0003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43077936b331389b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.968+0000 7fcac7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcac7fff700 1 -- 172.21.15.16:0/4185456604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000e070 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcac7fff700 1 -- 172.21.15.16:0/4185456604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcae001bcc0 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55943ba902f0 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcac7fff700 1 -- 172.21.15.16:0/4185456604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae0027040 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 10 client.15930 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55943b964be0 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcac7fff700 1 -- 172.21.15.16:0/4185456604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55943ba902f0 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcac7fff700 1 -- 172.21.15.16:0/4185456604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcae0023460 con 0x55943bbf1c00 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 20 client.15930 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc056c0 0x55943bc07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 4 client.15930 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.969+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55943bbb6890 con 0x55943bc056c0 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.970+0000 7fcaec939700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc056c0 0x55943bc07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.970+0000 7fcaec939700 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc056c0 0x55943bc07ae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3cdb38cdc7f548 server_cookie=a8e2256f8a899b27 in_seq=0 out_seq=0 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.970+0000 7fcac7fff700 10 client.15930 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcac7fff700 1 -- 172.21.15.16:0/4185456604 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55943bbb6890 con 0x55943bc056c0 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcac7fff700 10 client.15930 handle_command_reply: tid=0 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 client.15930 shutdown 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 msgr2=0x55943bbf5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbf5a10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcae0003dc0 tx=0x7fcae0003df0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 shutdown_connections 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbf5a10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbf4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc056c0 0x55943bc07ae0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 --2- 172.21.15.16:0/4185456604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbf54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcac7fff700 0 client.15930 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 >> 172.21.15.16:0/4185456604 conn(0x55943bb45290 msgr2=0x55943bbe1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 shutdown_connections 2024-05-08T23:20:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.971+0000 7fcafad57b80 1 -- 172.21.15.16:0/4185456604 wait complete. 2024-05-08T23:20:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.972+0000 7fcafad57b80 20 client.15930 trim_cache size 0 max 16384 2024-05-08T23:20:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 Processor -- start 2024-05-08T23:20:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 -- start start 2024-05-08T23:20:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbebd70 0x55943bbf0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbf12e0 0x55943bbf4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbe1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bc05530 con 0x55943bbf1c00 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bbf7b20 con 0x55943bbebd70 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.990+0000 7fcafad57b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bbda0c0 con 0x55943bbf12e0 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbe1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbe1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58190/0 (socket says 172.21.15.16:58190) 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 -- 172.21.15.16:0/1619003005 learned_addr learned my addr 172.21.15.16:0/1619003005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcae7fff700 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbf12e0 0x55943bbf4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaec939700 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbebd70 0x55943bbf0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 -- 172.21.15.16:0/1619003005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbf12e0 msgr2=0x55943bbf4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbf12e0 0x55943bbf4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 -- 172.21.15.16:0/1619003005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbebd70 msgr2=0x55943bbf0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbebd70 0x55943bbf0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 -- 172.21.15.16:0/1619003005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943bafd5e0 con 0x55943bbf1c00 2024-05-08T23:20:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcaed13a700 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbe1da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009fa0 tx=0x7fcae0023990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb8b894d3dd0cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcacf7fe700 1 -- 172.21.15.16:0/1619003005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000e030 con 0x55943bbf1c00 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcacf7fe700 1 -- 172.21.15.16:0/1619003005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae0023ae0 con 0x55943bbf1c00 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.991+0000 7fcacf7fe700 1 -- 172.21.15.16:0/1619003005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0027040 con 0x55943bbf1c00 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 -- 172.21.15.16:0/1619003005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 msgr2=0x55943bbe1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbe1da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009fa0 tx=0x7fcae0023990 comp rx=0 tx=0).stop 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 -- 172.21.15.16:0/1619003005 shutdown_connections 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbe1da0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbebd70 0x55943bbf0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 --2- 172.21.15.16:0/1619003005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbf12e0 0x55943bbf4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 -- 172.21.15.16:0/1619003005 >> 172.21.15.16:0/1619003005 conn(0x55943bb45290 msgr2=0x55943bbf6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 -- 172.21.15.16:0/1619003005 shutdown_connections 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.992+0000 7fcafad57b80 1 -- 172.21.15.16:0/1619003005 wait complete. 2024-05-08T23:20:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.993+0000 7fcafad57b80 1 -- start start 2024-05-08T23:20:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.993+0000 7fcafad57b80 1 Processor -- start 2024-05-08T23:20:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbfb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbfb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbfbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bc5bbd0 con 0x55943bbf1c00 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bc5bd10 con 0x55943bbf12e0 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bc5be50 con 0x55943bbebd70 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcafad57b80 10 client.0 _ll_register_callbacks cb 0x55943badcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbfbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbfbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58200/0 (socket says 172.21.15.16:58200) 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 -- 172.21.15.16:0/3589945771 learned_addr learned my addr 172.21.15.16:0/3589945771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaec939700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbfb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcae7fff700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbfb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 -- 172.21.15.16:0/3589945771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 msgr2=0x55943bbfb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbfb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 -- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 msgr2=0x55943bbfb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbfb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 -- 172.21.15.16:0/3589945771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55943b964be0 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaec939700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbfb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcaed13a700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbfbb80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcae0008b20 tx=0x7fcae001aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5973969b817c7dcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcac7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.994+0000 7fcac7fff700 1 -- 172.21.15.16:0/3589945771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000e070 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcac7fff700 1 -- 172.21.15.16:0/3589945771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcae001bcc0 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcae000f040 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcac7fff700 1 -- 172.21.15.16:0/3589945771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae0027040 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 10 client.15936 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55943b9bcd50 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcac7fff700 1 -- 172.21.15.16:0/3589945771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcae000f040 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcac7fff700 1 -- 172.21.15.16:0/3589945771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcae0033070 con 0x55943bbf1c00 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 20 client.15936 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc5d290 0x55943bc5f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 4 client.15936 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55943bb2ff40 con 0x55943bc5d290 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.995+0000 7fcaec939700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc5d290 0x55943bc5f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.996+0000 7fcac7fff700 10 client.15936 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.996+0000 7fcaec939700 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc5d290 0x55943bc5f6b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42459ae4c662fe20 server_cookie=6e07cc5b4bd59e0f in_seq=0 out_seq=0 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcac7fff700 1 -- 172.21.15.16:0/3589945771 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55943bb2ff40 con 0x55943bc5d290 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcac7fff700 10 client.15936 handle_command_reply: tid=0 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 client.15936 shutdown 2024-05-08T23:20:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 msgr2=0x55943bbfbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbfbb80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcae0008b20 tx=0x7fcae001aa10 comp rx=0 tx=0).stop 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 shutdown_connections 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55943bbf1c00 0x55943bbfbb80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55943bbebd70 0x55943bbfb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55943bc5d290 0x55943bc5f6b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 --2- 172.21.15.16:0/3589945771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55943bbf12e0 0x55943bbfb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcac7fff700 0 client.15936 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 >> 172.21.15.16:0/3589945771 conn(0x55943bb45290 msgr2=0x55943bbe6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 shutdown_connections 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.997+0000 7fcafad57b80 1 -- 172.21.15.16:0/3589945771 wait complete. 2024-05-08T23:20:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcafad57b80 20 client.15936 trim_cache size 0 max 16384 2024-05-08T23:20:54.004 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:20:54.004 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:20:54.004 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "08dfaae5-01d0-4ee4-9f19-71e748da2220", 2024-05-08T23:20:54.004 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:20:54.004 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad00a13b0 msgr2=0x7fcad00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad00a13b0 0x7fcad00a3870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009fd0 tx=0x7fcae001e000 comp rx=0 tx=0).stop 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 msgr2=0x7fcae81977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 0x7fcae81977b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00f690 tx=0x7fcadc00f6c0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 shutdown_connections 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcad00a13b0 0x7fcad00a3870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcae8105f90 0x7fcae819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae81000f0 0x7fcae8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 --2- 172.21.15.16:0/2062071153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcae8100ab0 0x7fcae81977b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 >> 172.21.15.16:0/2062071153 conn(0x7fcae8076ba0 msgr2=0x7fcae8069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 shutdown_connections 2024-05-08T23:20:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:53.998+0000 7fcaeeb9d700 1 -- 172.21.15.16:0/2062071153 wait complete. 2024-05-08T23:20:54.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:20:54.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:54 smithi016 ceph-mon[42541]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:54.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.159+0000 7f42996fa700 1 -- 172.21.15.16:0/265667976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4294100a10 msgr2=0x7f4294104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.159+0000 7f42996fa700 1 --2- 172.21.15.16:0/265667976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4294100a10 0x7f4294104a30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4288009990 tx=0x7f4288009ca0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.159+0000 7f42996fa700 1 -- 172.21.15.16:0/265667976 shutdown_connections 2024-05-08T23:20:54.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.159+0000 7f42996fa700 1 --2- 172.21.15.16:0/265667976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4294105160 0x7f429410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.159+0000 7f42996fa700 1 --2- 172.21.15.16:0/265667976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294100060 0x7f4294100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.159+0000 7f42996fa700 1 --2- 172.21.15.16:0/265667976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4294100a10 0x7f4294104a30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.160+0000 7f42996fa700 1 -- 172.21.15.16:0/265667976 >> 172.21.15.16:0/265667976 conn(0x7f4294076f40 msgr2=0x7f4294077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.160+0000 7f42996fa700 1 -- 172.21.15.16:0/265667976 shutdown_connections 2024-05-08T23:20:54.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.160+0000 7f42996fa700 1 -- 172.21.15.16:0/265667976 wait complete. 2024-05-08T23:20:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.160+0000 7f42996fa700 1 Processor -- start 2024-05-08T23:20:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 -- start start 2024-05-08T23:20:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4294100060 0x7f4294196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294105160 0x7f42941967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 0x7f429419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419b570 con 0x7f429419aaf0 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419b6e0 con 0x7f4294105160 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42996fa700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419b8a0 con 0x7f4294100060 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 0x7f429419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42927fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294105160 0x7f42941967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 0x7f429419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58216/0 (socket says 172.21.15.16:58216) 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f42937fe700 1 -- 172.21.15.16:0/1819986676 learned_addr learned my addr 172.21.15.16:0/1819986676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.161+0000 7f4292ffd700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4294100060 0x7f4294196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42937fe700 1 -- 172.21.15.16:0/1819986676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4294100060 msgr2=0x7f4294196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42937fe700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4294100060 0x7f4294196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42937fe700 1 -- 172.21.15.16:0/1819986676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294105160 msgr2=0x7f42941967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42937fe700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294105160 0x7f42941967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42937fe700 1 -- 172.21.15.16:0/1819986676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42880095f0 con 0x7f429419aaf0 2024-05-08T23:20:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42927fc700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294105160 0x7f42941967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:20:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42937fe700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 0x7f429419af70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f428c007ee0 tx=0x7f428c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f427bfff700 1 -- 172.21.15.16:0/1819986676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c017070 con 0x7f429419aaf0 2024-05-08T23:20:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f427bfff700 1 -- 172.21.15.16:0/1819986676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f428c00f040 con 0x7f429419aaf0 2024-05-08T23:20:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f427bfff700 1 -- 172.21.15.16:0/1819986676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c005710 con 0x7f429419aaf0 2024-05-08T23:20:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42941a2e00 con 0x7f429419aaf0 2024-05-08T23:20:54.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.162+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42941a32e0 con 0x7f429419aaf0 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 Processor -- start 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 -- start start 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f54489d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f544897350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544897cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5447ee150 con 0x55f54489ce00 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5447ebb10 con 0x55f544897890 2024-05-08T23:20:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5447e7400 con 0x55f54489d720 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f54489d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f54489d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58220/0 (socket says 172.21.15.16:58220) 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42927fc700 1 -- 172.21.15.16:0/919758393 learned_addr learned my addr 172.21.15.16:0/919758393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f4292ffd700 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f544897350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.166+0000 7f42927fc700 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544897cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42927fc700 1 -- 172.21.15.16:0/919758393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 msgr2=0x55f544897350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42927fc700 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f544897350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42927fc700 1 -- 172.21.15.16:0/919758393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 msgr2=0x55f544897cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42927fc700 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544897cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42927fc700 1 -- 172.21.15.16:0/919758393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5446683b0 con 0x55f54489ce00 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42927fc700 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f54489d1e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4288009990 tx=0x7f428801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5a2f544e4dbfe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42797fa700 1 -- 172.21.15.16:0/919758393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800bb20 con 0x55f54489ce00 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42797fa700 1 -- 172.21.15.16:0/919758393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4288004c90 con 0x55f54489ce00 2024-05-08T23:20:54.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.167+0000 7f42797fa700 1 -- 172.21.15.16:0/919758393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288005390 con 0x55f54489ce00 2024-05-08T23:20:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f427bfff700 1 -- 172.21.15.16:0/1819986676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) v1 ==== 136106+0+0 (secure 0 0 0) 0x7f428c005870 con 0x7f429419aaf0 2024-05-08T23:20:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 -- 172.21.15.16:0/919758393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 msgr2=0x55f54489d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f54489d1e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4288009990 tx=0x7f428801a040 comp rx=0 tx=0).stop 2024-05-08T23:20:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 -- 172.21.15.16:0/919758393 shutdown_connections 2024-05-08T23:20:54.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544897cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f54489d1e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/919758393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f544897350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 -- 172.21.15.16:0/919758393 >> 172.21.15.16:0/919758393 conn(0x55f5447f5210 msgr2=0x55f544891df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 -- 172.21.15.16:0/919758393 shutdown_connections 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 -- 172.21.15.16:0/919758393 wait complete. 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f427bfff700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f427c0a1340 0x7f427c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42a58b4b80 1 -- start start 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.168+0000 7f42927fc700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f427c0a1340 0x7f427c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f427bfff700 1 -- 172.21.15.16:0/1819986676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f428c0cc9d0 con 0x7f429419aaf0 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 Processor -- start 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544783560 con 0x55f54489ce00 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544783200 con 0x55f544897890 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f54466a6a0 con 0x55f54489d720 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42a58b4b80 10 client.0 _ll_register_callbacks cb 0x55f54478cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f427c0a1340 0x7f427c0a3800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4288005a40 tx=0x7f4288009ee0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f4292ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448a82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58226/0 (socket says 172.21.15.16:58226) 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 -- 172.21.15.16:0/705014098 learned_addr learned my addr 172.21.15.16:0/705014098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42937fe700 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 -- 172.21.15.16:0/705014098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 msgr2=0x55f5448a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 -- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 msgr2=0x55f5448a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448a7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.169+0000 7f42927fc700 1 -- 172.21.15.16:0/705014098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4288020040 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f42927fc700 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448a82a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f42880048f0 tx=0x7f428801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ecd280062ebe54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f4276ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f4276ff5700 1 -- 172.21.15.16:0/705014098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288038070 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f4276ff5700 1 -- 172.21.15.16:0/705014098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4288005420 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f544740270 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f4276ff5700 1 -- 172.21.15.16:0/705014098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42880055e0 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f42a58b4b80 10 client.15948 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f544614b60 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f4276ff5700 1 -- 172.21.15.16:0/705014098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f544740270 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f4276ff5700 1 -- 172.21.15.16:0/705014098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4288031420 con 0x55f54489ce00 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.170+0000 7f42a58b4b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f42a58b4b80 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f42a58b4b80 20 client.15948 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f5448b2e80 0x55f5448b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f42a58b4b80 4 client.15948 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f544866810 con 0x55f5448b2e80 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f4292ffd700 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f5448b2e80 0x55f5448b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f4292ffd700 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f5448b2e80 0x55f5448b5270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6324b50971419d8a server_cookie=ac4120ab1e8db04f in_seq=0 out_seq=0 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.171+0000 7f4276ff5700 10 client.15948 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f4276ff5700 1 -- 172.21.15.16:0/705014098 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f544866810 con 0x55f5448b2e80 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f4276ff5700 10 client.15948 handle_command_reply: tid=0 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 client.15948 shutdown 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 msgr2=0x55f5448a82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448a82a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f42880048f0 tx=0x7f428801a040 comp rx=0 tx=0).stop 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 shutdown_connections 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f5448b2e80 0x55f5448b5270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/705014098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448a82a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f4276ff5700 0 client.15948 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 >> 172.21.15.16:0/705014098 conn(0x55f5447f5210 msgr2=0x55f5448915a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 shutdown_connections 2024-05-08T23:20:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 1 -- 172.21.15.16:0/705014098 wait complete. 2024-05-08T23:20:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.172+0000 7f42a58b4b80 20 client.15948 trim_cache size 0 max 16384 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 Processor -- start 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 -- start start 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544891d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489ce00 0x55f5448a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 0x55f5448a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544893070 con 0x55f54489d720 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5448a2a90 con 0x55f544897890 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f544893e50 con 0x55f54489ce00 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 0x55f5448a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 0x55f5448a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58234/0 (socket says 172.21.15.16:58234) 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f42937fe700 1 -- 172.21.15.16:0/2491338921 learned_addr learned my addr 172.21.15.16:0/2491338921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.191+0000 7f4292ffd700 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544891d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42927fc700 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489ce00 0x55f5448a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42937fe700 1 -- 172.21.15.16:0/2491338921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489ce00 msgr2=0x55f5448a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42937fe700 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489ce00 0x55f5448a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42937fe700 1 -- 172.21.15.16:0/2491338921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 msgr2=0x55f544891d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42937fe700 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544891d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42937fe700 1 -- 172.21.15.16:0/2491338921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5447ad560 con 0x55f54489d720 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f42937fe700 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 0x55f5448a1ab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f428c0994e0 tx=0x7f428c0b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e8ca4ea8d35f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f4279ffb700 1 -- 172.21.15.16:0/2491338921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c098eb0 con 0x55f54489d720 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f4279ffb700 1 -- 172.21.15.16:0/2491338921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f428c09d170 con 0x55f54489d720 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.192+0000 7f4279ffb700 1 -- 172.21.15.16:0/2491338921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c0995c0 con 0x55f54489d720 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 -- 172.21.15.16:0/2491338921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 msgr2=0x55f5448a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 0x55f5448a1ab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f428c0994e0 tx=0x7f428c0b9040 comp rx=0 tx=0).stop 2024-05-08T23:20:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 -- 172.21.15.16:0/2491338921 shutdown_connections 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489d720 0x55f5448a1ab0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f544891d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/2491338921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489ce00 0x55f5448a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 -- 172.21.15.16:0/2491338921 >> 172.21.15.16:0/2491338921 conn(0x55f5447f5210 msgr2=0x55f54489df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 -- 172.21.15.16:0/2491338921 shutdown_connections 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 -- 172.21.15.16:0/2491338921 wait complete. 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 -- start start 2024-05-08T23:20:54.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.193+0000 7f42a58b4b80 1 Processor -- start 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5448ac470 con 0x55f54489ce00 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5448ac5e0 con 0x55f544897890 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5448ac7a0 con 0x55f54489d720 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 10 client.0 _ll_register_callbacks cb 0x55f54478cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58242/0 (socket says 172.21.15.16:58242) 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 -- 172.21.15.16:0/3584609429 learned_addr learned my addr 172.21.15.16:0/3584609429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f4292ffd700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42937fe700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 -- 172.21.15.16:0/3584609429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 msgr2=0x55f5448abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 -- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 msgr2=0x55f5448af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 -- 172.21.15.16:0/3584609429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428c095040 con 0x55f54489ce00 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42937fe700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42927fc700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448ab9f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4288004c60 tx=0x7f4288031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de732177377e836 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f4276ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f4276ff5700 1 -- 172.21.15.16:0/3584609429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288038040 con 0x55f54489ce00 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f4276ff5700 1 -- 172.21.15.16:0/3584609429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4288031a50 con 0x55f54489ce00 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.194+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4288004c90 con 0x55f54489ce00 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f4276ff5700 1 -- 172.21.15.16:0/3584609429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4288031630 con 0x55f54489ce00 2024-05-08T23:20:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 10 client.15954 fetch_fsmap learned FSMap version 25 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f54466ccd0 con 0x55f54489ce00 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f4276ff5700 1 -- 172.21.15.16:0/3584609429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4288004c90 con 0x55f54489ce00 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f4276ff5700 1 -- 172.21.15.16:0/3584609429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4288004920 con 0x55f54489ce00 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 20 client.15954 populate_metadata read hostname 'smithi016' 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f54490a370 0x55f54490c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 4 client.15954 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5447dfec0 con 0x55f54490a370 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.195+0000 7f4292ffd700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f54490a370 0x55f54490c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f4276ff5700 10 client.15954 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f4292ffd700 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f54490a370 0x55f54490c760 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cce0a07c2c6a66 server_cookie=46b4e93303a35ad in_seq=0 out_seq=0 2024-05-08T23:20:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f4276ff5700 1 -- 172.21.15.16:0/3584609429 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f5447dfec0 con 0x55f54490a370 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f4276ff5700 10 client.15954 handle_command_reply: tid=0 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 client.15954 shutdown 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 msgr2=0x55f5448ab9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448ab9f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4288004c60 tx=0x7f4288031ec0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 shutdown_connections 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f54489d720 0x55f5448abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f54490a370 0x55f54490c760 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f544897890 0x55f5448af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 --2- 172.21.15.16:0/3584609429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f54489ce00 0x55f5448ab9f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f4276ff5700 0 client.15954 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 >> 172.21.15.16:0/3584609429 conn(0x55f5447f5210 msgr2=0x55f54489df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 shutdown_connections 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 1 -- 172.21.15.16:0/3584609429 wait complete. 2024-05-08T23:20:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.196+0000 7f42a58b4b80 20 client.15954 trim_cache size 0 max 16384 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: "08dfaae5-01d0-4ee4-9f19-71e748da2220": { 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "08dfaae5-01d0-4ee4-9f19-71e748da2220", 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:20:54.204 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:20:54.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f427c0a1340 msgr2=0x7f427c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f427c0a1340 0x7f427c0a3800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4288005a40 tx=0x7f4288009ee0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 msgr2=0x7f429419af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:20:54.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 0x7f429419af70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f428c007ee0 tx=0x7f428c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 shutdown_connections 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f427c0a1340 0x7f427c0a3800 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f429419aaf0 0x7f429419af70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4294100060 0x7f4294196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 --2- 172.21.15.16:0/1819986676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4294105160 0x7f42941967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 >> 172.21.15.16:0/1819986676 conn(0x7f4294076f40 msgr2=0x7f42940fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 shutdown_connections 2024-05-08T23:20:54.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:20:54.197+0000 7f42996fa700 1 -- 172.21.15.16:0/1819986676 wait complete. 2024-05-08T23:20:54.211 INFO:tasks.cephfs.filesystem:scrub status for tag:08dfaae5-01d0-4ee4-9f19-71e748da2220 - {'path': '/', 'tag': '08dfaae5-01d0-4ee4-9f19-71e748da2220', 'options': 'recursive,force'} 2024-05-08T23:20:54.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:54 smithi189 ceph-mon[46919]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:54 smithi155 ceph-mon[47105]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:20:55.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:55 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:20:55.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:55 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:55.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:55 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:20:55.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:55 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:55 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:20:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:55 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:55 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:20:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:55 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:55 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:20:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:55 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:55 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:20:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:55 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:20:56.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:56 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:20:56.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:56 smithi016 ceph-mon[42541]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:56.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:56 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:20:56.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:56 smithi189 ceph-mon[46919]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:56.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:56 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:20:56.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:56 smithi155 ceph-mon[47105]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:58.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:20:58 smithi016 ceph-mon[42541]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:58.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:20:58 smithi189 ceph-mon[46919]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:20:58.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:20:58 smithi155 ceph-mon[47105]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-08T23:21:00.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:00 smithi016 ceph-mon[42541]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:21:00.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:00 smithi189 ceph-mon[46919]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:21:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:00 smithi155 ceph-mon[47105]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:21:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:02 smithi016 ceph-mon[42541]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:02.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:02 smithi189 ceph-mon[46919]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:02.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:02 smithi155 ceph-mon[47105]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:04.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:04 smithi016 ceph-mon[42541]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-08T23:21:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:04 smithi189 ceph-mon[46919]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-08T23:21:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:04 smithi155 ceph-mon[47105]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-08T23:21:06.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:06 smithi016 ceph-mon[42541]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:21:06.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:06 smithi189 ceph-mon[46919]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:21:06.467 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:06 smithi155 ceph-mon[47105]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:21:08.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:08 smithi016 ceph-mon[42541]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:21:08.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:08 smithi189 ceph-mon[46919]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:21:08.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:08 smithi155 ceph-mon[47105]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-08T23:21:10.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:10 smithi016 ceph-mon[42541]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-05-08T23:21:10.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:10 smithi016 ceph-mon[42541]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:21:10.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:10 smithi189 ceph-mon[46919]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-05-08T23:21:10.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:10 smithi189 ceph-mon[46919]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:21:10.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:10 smithi155 ceph-mon[47105]: mgrmap e25: x(active, since 91s), standbys: y, z 2024-05-08T23:21:10.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:10 smithi155 ceph-mon[47105]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:21:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:12 smithi016 ceph-mon[42541]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:21:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:12 smithi189 ceph-mon[46919]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:21:12.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:12 smithi155 ceph-mon[47105]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T23:21:14.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:14 smithi016 ceph-mon[42541]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:21:14.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:14 smithi189 ceph-mon[46919]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:21:14.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:14 smithi155 ceph-mon[47105]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-08T23:21:16.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:16 smithi016 ceph-mon[42541]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-08T23:21:16.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:16 smithi189 ceph-mon[46919]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-08T23:21:16.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:16 smithi155 ceph-mon[47105]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-08T23:21:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:18 smithi016 ceph-mon[42541]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-08T23:21:18.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:18 smithi189 ceph-mon[46919]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-08T23:21:18.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:18 smithi155 ceph-mon[47105]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-08T23:21:20.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:20 smithi016 ceph-mon[42541]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-08T23:21:20.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:20 smithi189 ceph-mon[46919]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-08T23:21:20.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:20 smithi155 ceph-mon[47105]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-08T23:21:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:22 smithi016 ceph-mon[42541]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-08T23:21:22.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:22 smithi189 ceph-mon[46919]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-08T23:21:22.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:22 smithi155 ceph-mon[47105]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 85 B/s wr, 4 op/s 2024-05-08T23:21:24.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:24.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:24 smithi016 ceph-mon[42541]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T23:21:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 -- 172.21.15.16:0/1290606067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284073c60 msgr2=0x7fd2841119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/1290606067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284073c60 0x7fd2841119e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd274009990 tx=0x7fd274009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 -- 172.21.15.16:0/1290606067 shutdown_connections 2024-05-08T23:21:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/1290606067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd284111f20 0x7fd284114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/1290606067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd284073340 0x7fd284073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/1290606067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284073c60 0x7fd2841119e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 -- 172.21.15.16:0/1290606067 >> 172.21.15.16:0/1290606067 conn(0x7fd2840fc9e0 msgr2=0x7fd2840fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 -- 172.21.15.16:0/1290606067 shutdown_connections 2024-05-08T23:21:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.396+0000 7fd28b8d3700 1 -- 172.21.15.16:0/1290606067 wait complete. 2024-05-08T23:21:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.397+0000 7fd28b8d3700 1 Processor -- start 2024-05-08T23:21:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.397+0000 7fd28b8d3700 1 -- start start 2024-05-08T23:21:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd28b8d3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd284073340 0x7fd2841a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd28b8d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd284073c60 0x7fd2841a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd28b8d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 0x7fd28419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd28b8d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28419ddd0 con 0x7fd284111f20 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd28b8d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28419df40 con 0x7fd284073c60 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd28b8d3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28419e100 con 0x7fd284073340 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 0x7fd28419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 0x7fd28419d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50942/0 (socket says 172.21.15.16:50942) 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 -- 172.21.15.16:0/3277803983 learned_addr learned my addr 172.21.15.16:0/3277803983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd284073c60 0x7fd2841a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 -- 172.21.15.16:0/3277803983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd284073340 msgr2=0x7fd2841a3640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd284073340 0x7fd2841a3640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 -- 172.21.15.16:0/3277803983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd284073c60 msgr2=0x7fd2841a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd284073c60 0x7fd2841a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.398+0000 7fd289e70700 1 -- 172.21.15.16:0/3277803983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2740095f0 con 0x7fd284111f20 2024-05-08T23:21:24.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.399+0000 7fd289e70700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 0x7fd28419d770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0094f0 tx=0x7fd27c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.399+0000 7fd27a7fc700 1 -- 172.21.15.16:0/3277803983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00ddd0 con 0x7fd284111f20 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.399+0000 7fd27a7fc700 1 -- 172.21.15.16:0/3277803983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd27c00f040 con 0x7fd284111f20 2024-05-08T23:21:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.399+0000 7fd27a7fc700 1 -- 172.21.15.16:0/3277803983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c005a00 con 0x7fd284111f20 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.399+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28419e3e0 con 0x7fd284111f20 2024-05-08T23:21:24.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.399+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284105390 con 0x7fd284111f20 2024-05-08T23:21:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.405+0000 7fd27a7fc700 1 -- 172.21.15.16:0/3277803983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd27c005020 con 0x7fd284111f20 2024-05-08T23:21:24.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.406+0000 7fd297a8db80 1 Processor -- start 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.406+0000 7fd27a7fc700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd2700a1460 0x7fd2700a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd27a7fc700 1 -- 172.21.15.16:0/3277803983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fd27c0cc880 con 0x7fd284111f20 2024-05-08T23:21:24.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd2700a1460 0x7fd2700a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 -- start start 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e26f7150 con 0x5556e27a6720 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e26f4b10 con 0x5556e27a0890 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd297a8db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e26f0400 con 0x5556e27a5e00 2024-05-08T23:21:24.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.407+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd2700a1460 0x7fd2700a3920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd28419ee80 tx=0x7fd274005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd288e6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd288e6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50954/0 (socket says 172.21.15.16:50954) 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd288e6e700 1 -- 172.21.15.16:0/1136206915 learned_addr learned my addr 172.21.15.16:0/1136206915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd28966f700 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 -- 172.21.15.16:0/1136206915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 msgr2=0x5556e27a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 -- 172.21.15.16:0/1136206915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 msgr2=0x5556e27a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 -- 172.21.15.16:0/1136206915 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556e25713b0 con 0x5556e27a0890 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd288e6e700 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd289e70700 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0c9c30 tx=0x7fd27c095a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0fffa2387068dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd26f7fe700 1 -- 172.21.15.16:0/1136206915 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0b90d0 con 0x5556e27a0890 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd26f7fe700 1 -- 172.21.15.16:0/1136206915 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd27c098f50 con 0x5556e27a0890 2024-05-08T23:21:24.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.408+0000 7fd26f7fe700 1 -- 172.21.15.16:0/1136206915 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0a1a70 con 0x5556e27a0890 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.409+0000 7fd297a8db80 1 -- 172.21.15.16:0/1136206915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 msgr2=0x5556e27a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.409+0000 7fd297a8db80 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0c9c30 tx=0x7fd27c095a50 comp rx=0 tx=0).stop 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 -- 172.21.15.16:0/1136206915 shutdown_connections 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0cd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 --2- 172.21.15.16:0/1136206915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 -- 172.21.15.16:0/1136206915 >> 172.21.15.16:0/1136206915 conn(0x5556e26fe210 msgr2=0x5556e279adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 -- 172.21.15.16:0/1136206915 shutdown_connections 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 -- 172.21.15.16:0/1136206915 wait complete. 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 -- start start 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.410+0000 7fd297a8db80 1 Processor -- start 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a0890 0x5556e27a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a5e00 0x5556e27aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e268c560 con 0x5556e27a6720 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e268c200 con 0x5556e27a5e00 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e25736a0 con 0x5556e27a0890 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd297a8db80 10 client.0 _ll_register_callbacks cb 0x5556e2695b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd289e70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd289e70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50958/0 (socket says 172.21.15.16:50958) 2024-05-08T23:21:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.411+0000 7fd289e70700 1 -- 172.21.15.16:0/2718565146 learned_addr learned my addr 172.21.15.16:0/2718565146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd288e6e700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a5e00 0x5556e27aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd289e70700 1 -- 172.21.15.16:0/2718565146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a0890 msgr2=0x5556e27a9b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd289e70700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a0890 0x5556e27a9b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd289e70700 1 -- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a5e00 msgr2=0x5556e27aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd289e70700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a5e00 0x5556e27aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd289e70700 1 -- 172.21.15.16:0/2718565146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284114cc0 con 0x5556e27a6720 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd288e6e700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a5e00 0x5556e27aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd289e70700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27aa600 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009fd0 tx=0x7fd27c007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5cc11cc9a52c52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd26cff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd26cff9700 1 -- 172.21.15.16:0/2718565146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c099cf0 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.412+0000 7fd26cff9700 1 -- 172.21.15.16:0/2718565146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fd27c0a1a70 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556e2649270 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd26cff9700 1 -- 172.21.15.16:0/2718565146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd27c0995f0 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 10 client.15966 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fd284105390 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd26cff9700 1 -- 172.21.15.16:0/2718565146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5556e2649270 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd26cff9700 1 -- 172.21.15.16:0/2718565146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd27c099850 con 0x5556e27a6720 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 10 client.15966 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 20 client.15966 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e27caec0 0x5556e27cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 4 client.15966 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.413+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556e276f810 con 0x5556e27caec0 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.414+0000 7fd28966f700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e27caec0 0x5556e27cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.414+0000 7fd28966f700 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e27caec0 0x5556e27cd2b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373da02900c602d0 server_cookie=968807539f143e96 in_seq=0 out_seq=0 2024-05-08T23:21:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.414+0000 7fd26cff9700 10 client.15966 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd26cff9700 1 -- 172.21.15.16:0/2718565146 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556e276f810 con 0x5556e27caec0 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd26cff9700 10 client.15966 handle_command_reply: tid=0 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 client.15966 shutdown 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 msgr2=0x5556e27aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27aa600 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009fd0 tx=0x7fd27c007490 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 shutdown_connections 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e27aa600 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a0890 0x5556e27a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e27caec0 0x5556e27cd2b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 --2- 172.21.15.16:0/2718565146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a5e00 0x5556e27aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 >> 172.21.15.16:0/2718565146 conn(0x5556e26fe210 msgr2=0x5556e279a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.415+0000 7fd26cff9700 0 client.15966 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.416+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 shutdown_connections 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.416+0000 7fd297a8db80 1 -- 172.21.15.16:0/2718565146 wait complete. 2024-05-08T23:21:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.416+0000 7fd297a8db80 20 client.15966 trim_cache size 0 max 16384 2024-05-08T23:21:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:24 smithi189 ceph-mon[46919]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T23:21:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 Processor -- start 2024-05-08T23:21:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 -- start start 2024-05-08T23:21:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 0x5556e279ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a6720 0x5556e27aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e27ab130 con 0x5556e27a5e00 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e279ce50 con 0x5556e27a0890 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.452+0000 7fd297a8db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e2795800 con 0x5556e27a6720 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 0x5556e279ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 0x5556e279ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50966/0 (socket says 172.21.15.16:50966) 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 -- 172.21.15.16:0/3784468120 learned_addr learned my addr 172.21.15.16:0/3784468120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd28966f700 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd289e70700 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a6720 0x5556e27aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 -- 172.21.15.16:0/3784468120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a6720 msgr2=0x5556e27aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a6720 0x5556e27aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 -- 172.21.15.16:0/3784468120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 msgr2=0x5556e27a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 -- 172.21.15.16:0/3784468120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556e27a0f40 con 0x5556e27a5e00 2024-05-08T23:21:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd289e70700 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a6720 0x5556e27aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.453+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 0x5556e279ad20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd274005cd0 tx=0x7fd274020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790e3fcb48c557be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.454+0000 7fd26ffff700 1 -- 172.21.15.16:0/3784468120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2740056c0 con 0x5556e27a5e00 2024-05-08T23:21:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.454+0000 7fd26ffff700 1 -- 172.21.15.16:0/3784468120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd27401e040 con 0x5556e27a5e00 2024-05-08T23:21:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.454+0000 7fd26ffff700 1 -- 172.21.15.16:0/3784468120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd274024670 con 0x5556e27a5e00 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 -- 172.21.15.16:0/3784468120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 msgr2=0x5556e279ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 0x5556e279ad20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd274005cd0 tx=0x7fd274020000 comp rx=0 tx=0).stop 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 -- 172.21.15.16:0/3784468120 shutdown_connections 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a6720 0x5556e27aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3784468120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a5e00 0x5556e279ad20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.455+0000 7fd297a8db80 1 -- 172.21.15.16:0/3784468120 >> 172.21.15.16:0/3784468120 conn(0x5556e26fe210 msgr2=0x5556e27a8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.457+0000 7fd297a8db80 1 -- 172.21.15.16:0/3784468120 shutdown_connections 2024-05-08T23:21:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.459+0000 7fd297a8db80 1 -- 172.21.15.16:0/3784468120 wait complete. 2024-05-08T23:21:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.459+0000 7fd297a8db80 1 -- start start 2024-05-08T23:21:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.459+0000 7fd297a8db80 1 Processor -- start 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e281a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e281a8b0 con 0x5556e27a6720 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e281aa20 con 0x5556e27a0890 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556e281abe0 con 0x5556e27a5e00 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.460+0000 7fd297a8db80 10 client.0 _ll_register_callbacks cb 0x5556e2695b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27c03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53410/0 (socket says 172.21.15.16:53410) 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 -- 172.21.15.16:0/3898438392 learned_addr learned my addr 172.21.15.16:0/3898438392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd289e70700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e281a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 -- 172.21.15.16:0/3898438392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 msgr2=0x5556e27c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 -- 172.21.15.16:0/3898438392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 msgr2=0x5556e281a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e281a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd28966f700 1 -- 172.21.15.16:0/3898438392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27402c040 con 0x5556e27a0890 2024-05-08T23:21:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.461+0000 7fd289e70700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e281a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.462+0000 7fd28966f700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27c03d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd280016ec0 tx=0x7fd28000cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d5527b60e1d61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.462+0000 7fd26cff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.462+0000 7fd26cff9700 1 -- 172.21.15.16:0/3898438392 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28001f480 con 0x5556e27a0890 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.462+0000 7fd26cff9700 1 -- 172.21.15.16:0/3898438392 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fd280010430 con 0x5556e27a0890 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.462+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd27c098f50 con 0x5556e27a0890 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.462+0000 7fd26cff9700 1 -- 172.21.15.16:0/3898438392 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2800105f0 con 0x5556e27a0890 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.463+0000 7fd297a8db80 10 client.25757 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.463+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fd28410d830 con 0x5556e27a0890 2024-05-08T23:21:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.463+0000 7fd26cff9700 1 -- 172.21.15.16:0/3898438392 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd27c098f50 con 0x5556e27a0890 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd26cff9700 1 -- 172.21.15.16:0/3898438392 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd28001f8e0 con 0x5556e27a0890 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd297a8db80 10 client.25757 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd297a8db80 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd297a8db80 20 client.25757 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e28222f0 0x5556e2824730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd297a8db80 4 client.25757 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556e26e8ec0 con 0x5556e28222f0 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.464+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e28222f0 0x5556e2824730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd26cff9700 10 client.25757 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd288e6e700 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e28222f0 0x5556e2824730 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d5bc2eef173a4b server_cookie=cf2bb3fb3fbedc8c in_seq=0 out_seq=0 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd26cff9700 1 -- 172.21.15.16:0/3898438392 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5556e26e8ec0 con 0x5556e28222f0 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd26cff9700 10 client.25757 handle_command_reply: tid=0 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 client.25757 shutdown 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 msgr2=0x5556e27c03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27c03d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd280016ec0 tx=0x7fd28000cdd0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 shutdown_connections 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556e27a6720 0x5556e281a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5556e28222f0 0x5556e2824730 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5556e27a0890 0x5556e27c03d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd26cff9700 0 client.25757 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 --2- 172.21.15.16:0/3898438392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556e27a5e00 0x5556e27c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.465+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 >> 172.21.15.16:0/3898438392 conn(0x5556e26fe210 msgr2=0x5556e27a4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.467 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:24 smithi155 ceph-mon[47105]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 shutdown_connections 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd297a8db80 1 -- 172.21.15.16:0/3898438392 wait complete. 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd297a8db80 20 client.25757 trim_cache size 0 max 16384 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:21:24.468 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd2700a1460 msgr2=0x7fd2700a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd2700a1460 0x7fd2700a3920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd28419ee80 tx=0x7fd274005fb0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 msgr2=0x7fd28419d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.466+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 0x7fd28419d770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0094f0 tx=0x7fd27c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 shutdown_connections 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd2700a1460 0x7fd2700a3920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd284111f20 0x7fd28419d770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd284073340 0x7fd2841a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 --2- 172.21.15.16:0/3277803983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd284073c60 0x7fd2841a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 >> 172.21.15.16:0/3277803983 conn(0x7fd2840fc9e0 msgr2=0x7fd2840fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 shutdown_connections 2024-05-08T23:21:24.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.469+0000 7fd28b8d3700 1 -- 172.21.15.16:0/3277803983 wait complete. 2024-05-08T23:21:24.483 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:21:24.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 -- 172.21.15.16:0/3917611458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 msgr2=0x7f39ec105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 --2- 172.21.15.16:0/3917611458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec105640 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b0a0 tx=0x7f39e400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 -- 172.21.15.16:0/3917611458 shutdown_connections 2024-05-08T23:21:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 --2- 172.21.15.16:0/3917611458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec105640 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 --2- 172.21.15.16:0/3917611458 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec0ffd90 0x7f39ec100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 --2- 172.21.15.16:0/3917611458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39ec1006b0 0x7f39ec104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 -- 172.21.15.16:0/3917611458 >> 172.21.15.16:0/3917611458 conn(0x7f39ec0fb7e0 msgr2=0x7f39ec0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.628+0000 7f39f1d24700 1 -- 172.21.15.16:0/3917611458 shutdown_connections 2024-05-08T23:21:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.629+0000 7f39f1d24700 1 -- 172.21.15.16:0/3917611458 wait complete. 2024-05-08T23:21:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.629+0000 7f39f1d24700 1 Processor -- start 2024-05-08T23:21:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 -- start start 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39ec0ffd90 0x7f39ec1a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec1006b0 0x7f39ec1a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19dd80 con 0x7f39ec105190 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19def0 con 0x7f39ec1006b0 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39f1d24700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19e0b0 con 0x7f39ec0ffd90 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51002/0 (socket says 172.21.15.16:51002) 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39ebfff700 1 -- 172.21.15.16:0/1482253175 learned_addr learned my addr 172.21.15.16:0/1482253175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.630+0000 7f39eaffd700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec1006b0 0x7f39ec1a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39eb7fe700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39ec0ffd90 0x7f39ec1a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39ebfff700 1 -- 172.21.15.16:0/1482253175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39ec0ffd90 msgr2=0x7f39ec1a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39ebfff700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39ec0ffd90 0x7f39ec1a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39ebfff700 1 -- 172.21.15.16:0/1482253175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec1006b0 msgr2=0x7f39ec1a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39ebfff700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec1006b0 0x7f39ec1a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39ebfff700 1 -- 172.21.15.16:0/1482253175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4009d00 con 0x7f39ec105190 2024-05-08T23:21:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.631+0000 7f39ebfff700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec19d720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b070 tx=0x7f39e4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.632+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400e040 con 0x7f39ec105190 2024-05-08T23:21:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.632+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39e401d480 con 0x7f39ec105190 2024-05-08T23:21:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.632+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e401b6b0 con 0x7f39ec105190 2024-05-08T23:21:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.632+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec19e330 con 0x7f39ec105190 2024-05-08T23:21:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.632+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec1aa5e0 con 0x7f39ec105190 2024-05-08T23:21:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.633+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec069d10 con 0x7f39ec105190 2024-05-08T23:21:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.637+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f39e401d8a0 con 0x7f39ec105190 2024-05-08T23:21:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.637+0000 7f39e8ff9700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f39d40a1310 0x7f39d40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.638+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f39e40cee30 con 0x7f39ec105190 2024-05-08T23:21:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.638+0000 7f39eb7fe700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f39d40a1310 0x7f39d40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.638+0000 7f39eb7fe700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f39d40a1310 0x7f39d40a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009bf0 tx=0x7f39dc009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:24.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.643+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f39e4097f70 con 0x7f39ec105190 2024-05-08T23:21:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.940+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39ec068060 con 0x7f39ec105190 2024-05-08T23:21:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.941+0000 7f39e8ff9700 1 -- 172.21.15.16:0/1482253175 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f39e4021090 con 0x7f39ec105190 2024-05-08T23:21:24.941 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:21:24.941 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.943+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f39d40a1310 msgr2=0x7f39d40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.943+0000 7f39f1d24700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f39d40a1310 0x7f39d40a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009bf0 tx=0x7f39dc009260 comp rx=0 tx=0).stop 2024-05-08T23:21:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 msgr2=0x7f39ec19d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec19d720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b070 tx=0x7f39e4007e00 comp rx=0 tx=0).stop 2024-05-08T23:21:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 shutdown_connections 2024-05-08T23:21:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f39d40a1310 0x7f39d40a37d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39ec105190 0x7f39ec19d720 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39ec0ffd90 0x7f39ec1a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 --2- 172.21.15.16:0/1482253175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39ec1006b0 0x7f39ec1a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 >> 172.21.15.16:0/1482253175 conn(0x7f39ec0fb7e0 msgr2=0x7f39ec0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 shutdown_connections 2024-05-08T23:21:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:24.944+0000 7f39f1d24700 1 -- 172.21.15.16:0/1482253175 wait complete. 2024-05-08T23:21:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:21:24.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 -- 172.21.15.16:0/3193156123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124100a10 msgr2=0x7fe124104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 --2- 172.21.15.16:0/3193156123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124100a10 0x7fe124104a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1180098f0 tx=0x7fe118009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 -- 172.21.15.16:0/3193156123 shutdown_connections 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 --2- 172.21.15.16:0/3193156123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124105160 0x7fe12410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 --2- 172.21.15.16:0/3193156123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe124100060 0x7fe124100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 --2- 172.21.15.16:0/3193156123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124100a10 0x7fe124104a30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 -- 172.21.15.16:0/3193156123 >> 172.21.15.16:0/3193156123 conn(0x7fe124076f40 msgr2=0x7fe124077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 -- 172.21.15.16:0/3193156123 shutdown_connections 2024-05-08T23:21:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.123+0000 7fe129ed0700 1 -- 172.21.15.16:0/3193156123 wait complete. 2024-05-08T23:21:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 Processor -- start 2024-05-08T23:21:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 -- start start 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 0x7fe124199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124105160 0x7fe124199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe12419de00 0x7fe12419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12419e880 con 0x7fe124100060 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12419e9f0 con 0x7fe124105160 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.124+0000 7fe129ed0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12419ebb0 con 0x7fe12419de00 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 0x7fe124199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 0x7fe124199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51018/0 (socket says 172.21.15.16:51018) 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 -- 172.21.15.16:0/1298854422 learned_addr learned my addr 172.21.15.16:0/1298854422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe123fff700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe12419de00 0x7fe12419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe122ffd700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124105160 0x7fe124199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 -- 172.21.15.16:0/1298854422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe12419de00 msgr2=0x7fe12419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe12419de00 0x7fe12419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 -- 172.21.15.16:0/1298854422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124105160 msgr2=0x7fe124199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124105160 0x7fe124199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 -- 172.21.15.16:0/1298854422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c009d00 con 0x7fe124100060 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe122ffd700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124105160 0x7fe124199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe1237fe700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 0x7fe124199520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9a0 tx=0x7fe11400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.125+0000 7fe120ff9700 1 -- 172.21.15.16:0/1298854422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114009790 con 0x7fe124100060 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.126+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe118009520 con 0x7fe124100060 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.126+0000 7fe120ff9700 1 -- 172.21.15.16:0/1298854422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe11400cbd0 con 0x7fe124100060 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.126+0000 7fe120ff9700 1 -- 172.21.15.16:0/1298854422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114018680 con 0x7fe124100060 2024-05-08T23:21:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.126+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1241a64b0 con 0x7fe124100060 2024-05-08T23:21:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.129+0000 7fe120ff9700 1 -- 172.21.15.16:0/1298854422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe11400cd40 con 0x7fe124100060 2024-05-08T23:21:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.130+0000 7fe120ff9700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe10c0a1290 0x7fe10c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.130+0000 7fe120ff9700 1 -- 172.21.15.16:0/1298854422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe114014070 con 0x7fe124100060 2024-05-08T23:21:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 Processor -- start 2024-05-08T23:21:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 -- start start 2024-05-08T23:21:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c88705b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8863f150 con 0x563c886ecf20 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8863cb10 con 0x563c886e5cf0 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.131+0000 7fe13608ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c88638400 con 0x563c886e5330 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe122ffd700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe10c0a1290 0x7fe10c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe122ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886ed300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51022/0 (socket says 172.21.15.16:51022) 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe1237fe700 1 -- 172.21.15.16:0/465972588 learned_addr learned my addr 172.21.15.16:0/465972588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c88705b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 -- 172.21.15.16:0/465972588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 msgr2=0x563c886e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 -- 172.21.15.16:0/465972588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 msgr2=0x563c886ed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886ed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 -- 172.21.15.16:0/465972588 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c884b93b0 con 0x563c886e5cf0 2024-05-08T23:21:25.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe1237fe700 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886ed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.132+0000 7fe123fff700 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c88705b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009470 tx=0x7fe11c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc6bf9d6f83bb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.133+0000 7fe109ffb700 1 -- 172.21.15.16:0/465972588 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c019070 con 0x563c886e5cf0 2024-05-08T23:21:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.133+0000 7fe109ffb700 1 -- 172.21.15.16:0/465972588 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe11c00f040 con 0x563c886e5cf0 2024-05-08T23:21:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.133+0000 7fe109ffb700 1 -- 172.21.15.16:0/465972588 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00e030 con 0x563c886e5cf0 2024-05-08T23:21:25.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 -- 172.21.15.16:0/465972588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 msgr2=0x563c88705b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c88705b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009470 tx=0x7fe11c0099e0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 -- 172.21.15.16:0/465972588 shutdown_connections 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c88705b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886ed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 --2- 172.21.15.16:0/465972588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 -- 172.21.15.16:0/465972588 >> 172.21.15.16:0/465972588 conn(0x563c88646210 msgr2=0x563c886e2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 -- 172.21.15.16:0/465972588 shutdown_connections 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 -- 172.21.15.16:0/465972588 wait complete. 2024-05-08T23:21:25.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe122ffd700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe10c0a1290 0x7fe10c0a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe118000c00 tx=0x7fe118009c00 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.134+0000 7fe13608ab80 1 -- start start 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 Processor -- start 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c886f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c885d4560 con 0x563c886ecf20 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c885d4200 con 0x563c886e5cf0 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c884bb6a0 con 0x563c886e5330 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.135+0000 7fe13608ab80 10 client.0 _ll_register_callbacks cb 0x563c885ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe122ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c886f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe122ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c886f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53462/0 (socket says 172.21.15.16:53462) 2024-05-08T23:21:25.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe122ffd700 1 -- 172.21.15.16:0/1957042573 learned_addr learned my addr 172.21.15.16:0/1957042573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 -- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 msgr2=0x563c886f7be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886f7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 -- 172.21.15.16:0/1957042573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 msgr2=0x563c886f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c886f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 -- 172.21.15.16:0/1957042573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1241a64b0 con 0x563c886ecf20 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe122ffd700 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c886f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe123fff700 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886f1f50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe11c000f80 tx=0x7fe11c003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58299c3646f86c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe102ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe102ffd700 1 -- 172.21.15.16:0/1957042573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00e070 con 0x563c886ecf20 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe102ffd700 1 -- 172.21.15.16:0/1957042573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe11c003e70 con 0x563c886ecf20 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.136+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c88591270 con 0x563c886ecf20 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe102ffd700 1 -- 172.21.15.16:0/1957042573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe11c019040 con 0x563c886ecf20 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 10 client.15990 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:25.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe11c009150 con 0x563c886ecf20 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe102ffd700 1 -- 172.21.15.16:0/1957042573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c88591270 con 0x563c886ecf20 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe102ffd700 1 -- 172.21.15.16:0/1957042573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe11c040450 con 0x563c886ecf20 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 10 client.15990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 20 client.15990 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88701040 0x563c88703460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 4 client.15990 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:25.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.137+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c886b7810 con 0x563c88701040 2024-05-08T23:21:25.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.141+0000 7fe1237fe700 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88701040 0x563c88703460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.142+0000 7fe102ffd700 10 client.15990 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:25.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.142+0000 7fe1237fe700 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88701040 0x563c88703460 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0c5e85c9f1fe6e0 server_cookie=60d926e51dae18c8 in_seq=0 out_seq=0 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe102ffd700 1 -- 172.21.15.16:0/1957042573 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c886b7810 con 0x563c88701040 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe102ffd700 10 client.15990 handle_command_reply: tid=0 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 client.15990 shutdown 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 msgr2=0x563c886f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886f1f50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe11c000f80 tx=0x7fe11c003a00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 shutdown_connections 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886ecf20 0x563c886f1f50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5330 0x563c886f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88701040 0x563c88703460 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 --2- 172.21.15.16:0/1957042573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886e5cf0 0x563c886f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 >> 172.21.15.16:0/1957042573 conn(0x563c88646210 msgr2=0x563c886e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe102ffd700 0 client.15990 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 shutdown_connections 2024-05-08T23:21:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.143+0000 7fe13608ab80 1 -- 172.21.15.16:0/1957042573 wait complete. 2024-05-08T23:21:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.144+0000 7fe13608ab80 20 client.15990 trim_cache size 0 max 16384 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 Processor -- start 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 -- start start 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886e7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886e7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c886e3b30 con 0x563c886e5330 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c8870fc90 con 0x563c886ecf20 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.171+0000 7fe13608ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c886ea890 con 0x563c886e5cf0 2024-05-08T23:21:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886e7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51038/0 (socket says 172.21.15.16:51038) 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 -- 172.21.15.16:0/3257543296 learned_addr learned my addr 172.21.15.16:0/3257543296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe122ffd700 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe123fff700 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 -- 172.21.15.16:0/3257543296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 msgr2=0x563c886e7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886e7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 -- 172.21.15.16:0/3257543296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 msgr2=0x563c886f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.172+0000 7fe1237fe700 1 -- 172.21.15.16:0/3257543296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c88465b60 con 0x563c886e5330 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.173+0000 7fe1237fe700 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886e7030 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe114000c00 tx=0x7fe114010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c800c1c6337cd47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.173+0000 7fe10a7fc700 1 -- 172.21.15.16:0/3257543296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114096040 con 0x563c886e5330 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.173+0000 7fe10a7fc700 1 -- 172.21.15.16:0/3257543296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe11409e160 con 0x563c886e5330 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.173+0000 7fe10a7fc700 1 -- 172.21.15.16:0/3257543296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114096c30 con 0x563c886e5330 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.174+0000 7fe13608ab80 1 -- 172.21.15.16:0/3257543296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 msgr2=0x563c886e7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.174+0000 7fe13608ab80 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886e7030 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe114000c00 tx=0x7fe114010b20 comp rx=0 tx=0).stop 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 -- 172.21.15.16:0/3257543296 shutdown_connections 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886e7030 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 --2- 172.21.15.16:0/3257543296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886e7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 -- 172.21.15.16:0/3257543296 >> 172.21.15.16:0/3257543296 conn(0x563c88646210 msgr2=0x563c88707d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 -- 172.21.15.16:0/3257543296 shutdown_connections 2024-05-08T23:21:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 -- 172.21.15.16:0/3257543296 wait complete. 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 -- start start 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.175+0000 7fe13608ab80 1 Processor -- start 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c886fc060 con 0x563c886e5330 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c886fc1d0 con 0x563c886ecf20 2024-05-08T23:21:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c886fc390 con 0x563c886e5cf0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe13608ab80 10 client.0 _ll_register_callbacks cb 0x563c885ddb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51044/0 (socket says 172.21.15.16:51044) 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 -- 172.21.15.16:0/499890954 learned_addr learned my addr 172.21.15.16:0/499890954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe123fff700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe122ffd700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 -- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 msgr2=0x563c886fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 -- 172.21.15.16:0/499890954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 msgr2=0x563c886fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886fbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.176+0000 7fe1237fe700 1 -- 172.21.15.16:0/499890954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe118009520 con 0x563c886e5330 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe1237fe700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886fe100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e970 tx=0x7fe114005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d7b04c51938460 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe102ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe102ffd700 1 -- 172.21.15.16:0/499890954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114064070 con 0x563c886e5330 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe11c00f040 con 0x563c886e5330 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe102ffd700 1 -- 172.21.15.16:0/499890954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe1140988c0 con 0x563c886e5330 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe102ffd700 1 -- 172.21.15.16:0/499890954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe114096040 con 0x563c886e5330 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe13608ab80 10 client.15996 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563c885fe560 con 0x563c886e5330 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.177+0000 7fe102ffd700 1 -- 172.21.15.16:0/499890954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe11c00f040 con 0x563c886e5330 2024-05-08T23:21:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe102ffd700 1 -- 172.21.15.16:0/499890954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe114099a10 con 0x563c886e5330 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe13608ab80 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe13608ab80 10 client.15996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe13608ab80 20 client.15996 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe13608ab80 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88758cd0 0x563c8875b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe13608ab80 4 client.15996 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.178+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c88630ec0 con 0x563c88758cd0 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.179+0000 7fe122ffd700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88758cd0 0x563c8875b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.179+0000 7fe122ffd700 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88758cd0 0x563c8875b0f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3733e6af84424e7 server_cookie=e6dfdaa04998d134 in_seq=0 out_seq=0 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.179+0000 7fe102ffd700 10 client.15996 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.179+0000 7fe102ffd700 1 -- 172.21.15.16:0/499890954 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c88630ec0 con 0x563c88758cd0 2024-05-08T23:21:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.179+0000 7fe102ffd700 10 client.15996 handle_command_reply: tid=0 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.179+0000 7fe13608ab80 1 client.15996 shutdown 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 msgr2=0x563c886fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886fe100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e970 tx=0x7fe114005d00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 shutdown_connections 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563c88758cd0 0x563c8875b0f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563c886ecf20 0x563c886fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c886e5330 0x563c886fe100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe102ffd700 0 client.15996 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 --2- 172.21.15.16:0/499890954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c886e5cf0 0x563c886fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 >> 172.21.15.16:0/499890954 conn(0x563c88646210 msgr2=0x563c886e7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 shutdown_connections 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 1 -- 172.21.15.16:0/499890954 wait complete. 2024-05-08T23:21:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.180+0000 7fe13608ab80 20 client.15996 trim_cache size 0 max 16384 2024-05-08T23:21:25.183 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:21:25.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe10c0a1290 msgr2=0x7fe10c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe10c0a1290 0x7fe10c0a3750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe118000c00 tx=0x7fe118009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 msgr2=0x7fe124199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 0x7fe124199520 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9a0 tx=0x7fe11400ed60 comp rx=0 tx=0).stop 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 shutdown_connections 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe10c0a1290 0x7fe10c0a3750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe12419de00 0x7fe12419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe124100060 0x7fe124199520 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 --2- 172.21.15.16:0/1298854422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe124105160 0x7fe124199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 >> 172.21.15.16:0/1298854422 conn(0x7fe124076f40 msgr2=0x7fe1240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 shutdown_connections 2024-05-08T23:21:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.181+0000 7fe129ed0700 1 -- 172.21.15.16:0/1298854422 wait complete. 2024-05-08T23:21:25.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:21:25.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1482253175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 -- 172.21.15.16:0/88766366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 msgr2=0x7ff0c81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/88766366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81079e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009990 tx=0x7ff0b8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 -- 172.21.15.16:0/88766366 shutdown_connections 2024-05-08T23:21:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/88766366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 0x7ff0c8114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/88766366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0c810c580 0x7ff0c810c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/88766366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81079e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 -- 172.21.15.16:0/88766366 >> 172.21.15.16:0/88766366 conn(0x7ff0c8076d90 msgr2=0x7ff0c8105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 -- 172.21.15.16:0/88766366 shutdown_connections 2024-05-08T23:21:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.343+0000 7ff0ce03d700 1 -- 172.21.15.16:0/88766366 wait complete. 2024-05-08T23:21:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.344+0000 7ff0ce03d700 1 Processor -- start 2024-05-08T23:21:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.344+0000 7ff0ce03d700 1 -- start start 2024-05-08T23:21:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0ce03d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0ce03d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 0x7ff0c81a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0ce03d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0c810c580 0x7ff0c819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0ce03d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c819ddb0 con 0x7ff0c8107560 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0ce03d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c819df20 con 0x7ff0c810c580 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0ce03d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c819e0e0 con 0x7ff0c8107f20 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51060/0 (socket says 172.21.15.16:51060) 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 -- 172.21.15.16:0/3271483111 learned_addr learned my addr 172.21.15.16:0/3271483111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 0x7ff0c81a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0c810c580 0x7ff0c819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 -- 172.21.15.16:0/3271483111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 msgr2=0x7ff0c81a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 0x7ff0c81a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 -- 172.21.15.16:0/3271483111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0c810c580 msgr2=0x7ff0c819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0c810c580 0x7ff0c819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c77fe700 1 -- 172.21.15.16:0/3271483111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80095f0 con 0x7ff0c8107560 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.345+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 0x7ff0c81a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.346+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81a3620 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000efc0 tx=0x7ff0c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.346+0000 7ff0c4ff9700 1 -- 172.21.15.16:0/3271483111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0018070 con 0x7ff0c8107560 2024-05-08T23:21:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.346+0000 7ff0c4ff9700 1 -- 172.21.15.16:0/3271483111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0c000f040 con 0x7ff0c8107560 2024-05-08T23:21:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.346+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c819e360 con 0x7ff0c8107560 2024-05-08T23:21:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.346+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c806aba0 con 0x7ff0c8107560 2024-05-08T23:21:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.348+0000 7ff0c4ff9700 1 -- 172.21.15.16:0/3271483111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0004280 con 0x7ff0c8107560 2024-05-08T23:21:25.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.351+0000 7ff0da1f7b80 1 Processor -- start 2024-05-08T23:21:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.351+0000 7ff0c4ff9700 1 -- 172.21.15.16:0/3271483111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff0c0003a00 con 0x7ff0c8107560 2024-05-08T23:21:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.351+0000 7ff0da1f7b80 1 -- start start 2024-05-08T23:21:25.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a9641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a95e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 0x56044a95ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a8b5150 con 0x56044a964720 2024-05-08T23:21:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a8b2b10 con 0x56044a95e890 2024-05-08T23:21:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a8ae400 con 0x56044a963e00 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 0x56044a95ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 0x56044a95ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:53524/0 (socket says 172.21.15.16:53524) 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 -- 172.21.15.16:0/1245105806 learned_addr learned my addr 172.21.15.16:0/1245105806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c4ff9700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff0b00a1290 0x7ff0b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 -- 172.21.15.16:0/1245105806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 msgr2=0x56044a9641e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a9641e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 -- 172.21.15.16:0/1245105806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 msgr2=0x56044a95e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a95e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.352+0000 7ff0c6ffd700 1 -- 172.21.15.16:0/1245105806 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044a72f3b0 con 0x56044a95e890 2024-05-08T23:21:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.353+0000 7ff0c4ff9700 1 -- 172.21.15.16:0/3271483111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7ff0c00ccae0 con 0x7ff0c8107560 2024-05-08T23:21:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.353+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 0x56044a95ecd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8015000 tx=0x7ff0b800b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b6c3eedb1a5012 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.353+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff0b00a1290 0x7ff0b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.353+0000 7ff0adffb700 1 -- 172.21.15.16:0/1245105806 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b801d070 con 0x56044a95e890 2024-05-08T23:21:25.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.353+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff0b00a1290 0x7ff0b00a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc005950 tx=0x7ff0bc00b390 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.356+0000 7ff0adffb700 1 -- 172.21.15.16:0/1245105806 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0b801e040 con 0x56044a95e890 2024-05-08T23:21:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.356+0000 7ff0adffb700 1 -- 172.21.15.16:0/1245105806 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8004cf0 con 0x56044a95e890 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.356+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/1245105806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 msgr2=0x56044a95ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.356+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 0x56044a95ecd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8015000 tx=0x7ff0b800b9f0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/1245105806 shutdown_connections 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a95e890 0x56044a95ecd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a9641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/1245105806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a95e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/1245105806 >> 172.21.15.16:0/1245105806 conn(0x56044a8bc210 msgr2=0x56044a958df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/1245105806 shutdown_connections 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/1245105806 wait complete. 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 -- start start 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.357+0000 7ff0da1f7b80 1 Processor -- start 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a95e890 0x56044a967bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a9680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a968630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a84a560 con 0x56044a964720 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a84a200 con 0x56044a963e00 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a7316a0 con 0x56044a95e890 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0da1f7b80 10 client.0 _ll_register_callbacks cb 0x56044a853b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a968630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a968630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51080/0 (socket says 172.21.15.16:51080) 2024-05-08T23:21:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 -- 172.21.15.16:0/2000402332 learned_addr learned my addr 172.21.15.16:0/2000402332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a95e890 0x56044a967bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 -- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a95e890 msgr2=0x56044a967bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a95e890 0x56044a967bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 -- 172.21.15.16:0/2000402332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 msgr2=0x56044a9680f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a9680f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.358+0000 7ff0c7fff700 1 -- 172.21.15.16:0/2000402332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c8114d30 con 0x56044a964720 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a968630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc003b30 tx=0x7ff0bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af9da906712e5a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/2000402332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011c60 con 0x56044a964720 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/2000402332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7ff0bc017070 con 0x56044a964720 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56044a807270 con 0x56044a964720 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/2000402332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0bc011dc0 con 0x56044a964720 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0da1f7b80 10 client.16005 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:25.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56044a6dbb60 con 0x56044a964720 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.359+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/2000402332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56044a807270 con 0x56044a964720 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/2000402332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0bc026410 con 0x56044a964720 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0da1f7b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0da1f7b80 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0da1f7b80 20 client.16005 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a978320 0x56044a97a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0da1f7b80 4 client.16005 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.360+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56044a92d810 con 0x56044a978320 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.361+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a978320 0x56044a97a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.361+0000 7ff0a6ffd700 10 client.16005 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:25.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.361+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a978320 0x56044a97a740 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85b57d0a8574d93b server_cookie=b8adc853cfc1aee8 in_seq=0 out_seq=0 2024-05-08T23:21:25.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/2000402332 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56044a92d810 con 0x56044a978320 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0a6ffd700 10 client.16005 handle_command_reply: tid=0 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 client.16005 shutdown 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 msgr2=0x56044a968630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a968630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc003b30 tx=0x7ff0bc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 shutdown_connections 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a964720 0x56044a968630 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a95e890 0x56044a967bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a978320 0x56044a97a740 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/2000402332 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a9680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0a6ffd700 0 client.16005 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.362+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 >> 172.21.15.16:0/2000402332 conn(0x56044a8bc210 msgr2=0x56044a9585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.364+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 shutdown_connections 2024-05-08T23:21:25.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.367+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/2000402332 wait complete. 2024-05-08T23:21:25.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.368+0000 7ff0da1f7b80 20 client.16005 trim_cache size 0 max 16384 2024-05-08T23:21:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.382+0000 7ff0da1f7b80 1 Processor -- start 2024-05-08T23:21:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.382+0000 7ff0da1f7b80 1 -- start start 2024-05-08T23:21:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a95e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a973bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a964720 0x56044a958d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a807270 con 0x56044a95e890 2024-05-08T23:21:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a974100 con 0x56044a964720 2024-05-08T23:21:25.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a9759b0 con 0x56044a963e00 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a95e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a95e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51092/0 (socket says 172.21.15.16:51092) 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 -- 172.21.15.16:0/143809421 learned_addr learned my addr 172.21.15.16:0/143809421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a973bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a964720 0x56044a958d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 -- 172.21.15.16:0/143809421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 msgr2=0x56044a973bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a973bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 -- 172.21.15.16:0/143809421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a964720 msgr2=0x56044a958d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a964720 0x56044a958d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.383+0000 7ff0c77fe700 1 -- 172.21.15.16:0/143809421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044a874560 con 0x56044a95e890 2024-05-08T23:21:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.384+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a95e350 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00997e0 tx=0x7ff0c00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c2c30a481b7d08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.384+0000 7ff0ae7fc700 1 -- 172.21.15.16:0/143809421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000ce30 con 0x56044a95e890 2024-05-08T23:21:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.384+0000 7ff0ae7fc700 1 -- 172.21.15.16:0/143809421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0c00cce90 con 0x56044a95e890 2024-05-08T23:21:25.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.384+0000 7ff0ae7fc700 1 -- 172.21.15.16:0/143809421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00ca110 con 0x56044a95e890 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/143809421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 msgr2=0x56044a95e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a95e350 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00997e0 tx=0x7ff0c00092e0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/143809421 shutdown_connections 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a964720 0x56044a958d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a95e350 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/143809421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a963e00 0x56044a973bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/143809421 >> 172.21.15.16:0/143809421 conn(0x56044a8bc210 msgr2=0x56044a95a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/143809421 shutdown_connections 2024-05-08T23:21:25.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/143809421 wait complete. 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.385+0000 7ff0da1f7b80 1 -- start start 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 Processor -- start 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a96d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a96d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 0x56044a9cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a96df10 con 0x56044a95e890 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a9cdd60 con 0x56044a963e00 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044a9cdea0 con 0x56044a964720 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0da1f7b80 10 client.0 _ll_register_callbacks cb 0x56044a853b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 0x56044a9cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0c7fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 0x56044a9cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:37622/0 (socket says 172.21.15.16:37622) 2024-05-08T23:21:25.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0c7fff700 1 -- 172.21.15.16:0/812125786 learned_addr learned my addr 172.21.15.16:0/812125786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a96d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.386+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a96d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c7fff700 1 -- 172.21.15.16:0/812125786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 msgr2=0x56044a96d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a96d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c7fff700 1 -- 172.21.15.16:0/812125786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 msgr2=0x56044a96d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a96d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c7fff700 1 -- 172.21.15.16:0/812125786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c806aba0 con 0x56044a964720 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c6ffd700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a96d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0c7fff700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 0x56044a9cd820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc003b30 tx=0x7ff0bc0042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d97b40feefeff1a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/812125786 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc026990 con 0x56044a964720 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/812125786 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7ff0bc017050 con 0x56044a964720 2024-05-08T23:21:25.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.387+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0b801e040 con 0x56044a964720 2024-05-08T23:21:25.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.388+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/812125786 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0bc026af0 con 0x56044a964720 2024-05-08T23:21:25.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.388+0000 7ff0da1f7b80 10 client.25834 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:25.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.388+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56044a733cd0 con 0x56044a964720 2024-05-08T23:21:25.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.388+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/812125786 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0b801e040 con 0x56044a964720 2024-05-08T23:21:25.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/812125786 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0bc011a60 con 0x56044a964720 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0da1f7b80 10 client.25834 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0da1f7b80 10 client.25834 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0da1f7b80 20 client.25834 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a9cf4f0 0x56044a9d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0da1f7b80 4 client.25834 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56044a8a6ec0 con 0x56044a9cf4f0 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.389+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a9cf4f0 0x56044a9d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0c77fe700 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a9cf4f0 0x56044a9d1910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abac3162bd0f304 server_cookie=505d4aaf7830be43 in_seq=0 out_seq=0 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0a6ffd700 10 client.25834 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0a6ffd700 1 -- 172.21.15.16:0/812125786 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56044a8a6ec0 con 0x56044a9cf4f0 2024-05-08T23:21:25.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0a6ffd700 10 client.25834 handle_command_reply: tid=0 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 client.25834 shutdown 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 msgr2=0x56044a9cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 0x56044a9cd820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc003b30 tx=0x7ff0bc0042e0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 shutdown_connections 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56044a964720 0x56044a9cd820 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56044a95e890 0x56044a96d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x56044a9cf4f0 0x56044a9d1910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 --2- 172.21.15.16:0/812125786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56044a963e00 0x56044a96d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 >> 172.21.15.16:0/812125786 conn(0x56044a8bc210 msgr2=0x56044a966310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0a6ffd700 0 client.25834 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 shutdown_connections 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 1 -- 172.21.15.16:0/812125786 wait complete. 2024-05-08T23:21:25.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.390+0000 7ff0da1f7b80 20 client.25834 trim_cache size 0 max 16384 2024-05-08T23:21:25.395 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:21:25.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff0b00a1290 msgr2=0x7ff0b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff0b00a1290 0x7ff0b00a3750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc005950 tx=0x7ff0bc00b390 comp rx=0 tx=0).stop 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 msgr2=0x7ff0c81a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81a3620 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000efc0 tx=0x7ff0c000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 shutdown_connections 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff0b00a1290 0x7ff0b00a3750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0c810c580 0x7ff0c819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0c8107560 0x7ff0c81a3620 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 --2- 172.21.15.16:0/3271483111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0c8107f20 0x7ff0c81a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 >> 172.21.15.16:0/3271483111 conn(0x7ff0c8076d90 msgr2=0x7ff0c80773f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 shutdown_connections 2024-05-08T23:21:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.391+0000 7ff0ce03d700 1 -- 172.21.15.16:0/3271483111 wait complete. 2024-05-08T23:21:25.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:21:25.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1482253175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:25.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1482253175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.592+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1506522970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 msgr2=0x7fce2410d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.592+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1506522970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce2410d400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce140098f0 tx=0x7fce14009c00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1506522970 shutdown_connections 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1506522970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce2410d940 0x7fce2410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1506522970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce240694b0 0x7fce24069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1506522970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce2410d400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1506522970 >> 172.21.15.16:0/1506522970 conn(0x7fce24076f70 msgr2=0x7fce24077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1506522970 shutdown_connections 2024-05-08T23:21:25.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1506522970 wait complete. 2024-05-08T23:21:25.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.593+0000 7fce2c2b7700 1 Processor -- start 2024-05-08T23:21:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 -- start start 2024-05-08T23:21:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce240694b0 0x7fce241994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce24199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2410d940 0x7fce2419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e410 con 0x7fce240694b0 2024-05-08T23:21:25.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e580 con 0x7fce24069dd0 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2c2b7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e740 con 0x7fce2410d940 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2a053700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce240694b0 0x7fce241994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2a053700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce240694b0 0x7fce241994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51122/0 (socket says 172.21.15.16:51122) 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2a053700 1 -- 172.21.15.16:0/1352609194 learned_addr learned my addr 172.21.15.16:0/1352609194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce29852700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce24199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce2a854700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2410d940 0x7fce2419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce29852700 1 -- 172.21.15.16:0/1352609194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2410d940 msgr2=0x7fce2419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce29852700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2410d940 0x7fce2419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce29852700 1 -- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce240694b0 msgr2=0x7fce241994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.594+0000 7fce29852700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce240694b0 0x7fce241994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.595+0000 7fce29852700 1 -- 172.21.15.16:0/1352609194 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20009520 con 0x7fce24069dd0 2024-05-08T23:21:25.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.596+0000 7fce29852700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce24199a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce14009230 tx=0x7fce1400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.597+0000 7fce1b7fe700 1 -- 172.21.15.16:0/1352609194 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1401c070 con 0x7fce24069dd0 2024-05-08T23:21:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.597+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce14009520 con 0x7fce24069dd0 2024-05-08T23:21:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.597+0000 7fce1b7fe700 1 -- 172.21.15.16:0/1352609194 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce1400b9f0 con 0x7fce24069dd0 2024-05-08T23:21:25.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.597+0000 7fce1b7fe700 1 -- 172.21.15.16:0/1352609194 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140038e0 con 0x7fce24069dd0 2024-05-08T23:21:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.597+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2406a9c0 con 0x7fce24069dd0 2024-05-08T23:21:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.600+0000 7fce1b7fe700 1 -- 172.21.15.16:0/1352609194 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fce14020030 con 0x7fce24069dd0 2024-05-08T23:21:25.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.601+0000 7fce38471b80 1 Processor -- start 2024-05-08T23:21:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.601+0000 7fce38471b80 1 -- start start 2024-05-08T23:21:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce38471b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce38471b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032f30 0x563ff502cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce38471b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502d0a0 0x563ff502d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce38471b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4f7b150 con 0x563ff5032610 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce38471b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4f78b10 con 0x563ff5032f30 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce38471b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4f74400 con 0x563ff502d0a0 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce29852700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502d0a0 0x563ff502d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce2a854700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce1b7fe700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce100a1290 0x7fce100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce2a854700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50329f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51132/0 (socket says 172.21.15.16:51132) 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce29852700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502d0a0 0x563ff502d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:37648/0 (socket says 172.21.15.16:37648) 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.602+0000 7fce2a854700 1 -- 172.21.15.16:0/2486347296 learned_addr learned my addr 172.21.15.16:0/2486347296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce1b7fe700 1 -- 172.21.15.16:0/1352609194 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fce140cde20 con 0x7fce24069dd0 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce100a1290 0x7fce100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 -- 172.21.15.16:0/2486347296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502d0a0 msgr2=0x563ff502d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502d0a0 0x563ff502d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 -- 172.21.15.16:0/2486347296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032f30 msgr2=0x563ff502cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032f30 0x563ff502cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 -- 172.21.15.16:0/2486347296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff4df53b0 con 0x563ff5032610 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.603+0000 7fce2a854700 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50329f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007a00 tx=0x7fce1c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6a487d23d2f5ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.604+0000 7fce18da4700 1 -- 172.21.15.16:0/2486347296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00e040 con 0x563ff5032610 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.604+0000 7fce18da4700 1 -- 172.21.15.16:0/2486347296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce1c00f040 con 0x563ff5032610 2024-05-08T23:21:25.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.604+0000 7fce18da4700 1 -- 172.21.15.16:0/2486347296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c023470 con 0x563ff5032610 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 -- 172.21.15.16:0/2486347296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 msgr2=0x563ff50329f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50329f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fce1c007a00 tx=0x7fce1c007d10 comp rx=0 tx=0).stop 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce2a854700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce100a1290 0x7fce100a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0175d0 tx=0x7fce1c01d000 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 -- 172.21.15.16:0/2486347296 shutdown_connections 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502d0a0 0x563ff502d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50329f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 --2- 172.21.15.16:0/2486347296 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032f30 0x563ff502cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.605+0000 7fce38471b80 1 -- 172.21.15.16:0/2486347296 >> 172.21.15.16:0/2486347296 conn(0x563ff4f82210 msgr2=0x563ff501edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 -- 172.21.15.16:0/2486347296 shutdown_connections 2024-05-08T23:21:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 -- 172.21.15.16:0/2486347296 wait complete. 2024-05-08T23:21:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 -- start start 2024-05-08T23:21:25.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 Processor -- start 2024-05-08T23:21:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5026f30 0x563ff502c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502cc50 0x563ff5021da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4f10560 con 0x563ff5032610 2024-05-08T23:21:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4f10200 con 0x563ff5026f30 2024-05-08T23:21:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff4df76a0 con 0x563ff502cc50 2024-05-08T23:21:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.609+0000 7fce38471b80 10 client.0 _ll_register_callbacks cb 0x563ff4f19b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50269f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51142/0 (socket says 172.21.15.16:51142) 2024-05-08T23:21:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 -- 172.21.15.16:0/2992716878 learned_addr learned my addr 172.21.15.16:0/2992716878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a854700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502cc50 0x563ff5021da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce29852700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5026f30 0x563ff502c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 -- 172.21.15.16:0/2992716878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502cc50 msgr2=0x563ff5021da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502cc50 0x563ff5021da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 -- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5026f30 msgr2=0x563ff502c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5026f30 0x563ff502c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.610+0000 7fce2a053700 1 -- 172.21.15.16:0/2992716878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c0075e0 con 0x563ff5032610 2024-05-08T23:21:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce2a053700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50269f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fce200117c0 tx=0x7fce20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632f7c30f848a726 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce0dffb700 1 -- 172.21.15.16:0/2992716878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20004830 con 0x563ff5032610 2024-05-08T23:21:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce0dffb700 1 -- 172.21.15.16:0/2992716878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fce20004990 con 0x563ff5032610 2024-05-08T23:21:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ff4ecd270 con 0x563ff5032610 2024-05-08T23:21:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce0dffb700 1 -- 172.21.15.16:0/2992716878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce20004b50 con 0x563ff5032610 2024-05-08T23:21:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce38471b80 10 client.16023 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563ff4da1b60 con 0x563ff5032610 2024-05-08T23:21:25.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.611+0000 7fce0dffb700 1 -- 172.21.15.16:0/2992716878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563ff4ecd270 con 0x563ff5032610 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce0dffb700 1 -- 172.21.15.16:0/2992716878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce2000f750 con 0x563ff5032610 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce38471b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce38471b80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce38471b80 20 client.16023 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce38471b80 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff504fbb0 0x563ff5051fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce38471b80 4 client.16023 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.612+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ff4ff3810 con 0x563ff504fbb0 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.613+0000 7fce29852700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff504fbb0 0x563ff5051fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.613+0000 7fce29852700 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff504fbb0 0x563ff5051fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d7944ecf36b5c5 server_cookie=185a33e532a2d22 in_seq=0 out_seq=0 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.613+0000 7fce0dffb700 10 client.16023 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce0dffb700 1 -- 172.21.15.16:0/2992716878 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ff4ff3810 con 0x563ff504fbb0 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce0dffb700 10 client.16023 handle_command_reply: tid=0 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 client.16023 shutdown 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 msgr2=0x563ff50269f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50269f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fce200117c0 tx=0x7fce20011ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 shutdown_connections 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff502cc50 0x563ff5021da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff50269f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff504fbb0 0x563ff5051fd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 --2- 172.21.15.16:0/2992716878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5026f30 0x563ff502c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 >> 172.21.15.16:0/2992716878 conn(0x563ff4f82210 msgr2=0x563ff501e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce0dffb700 0 client.16023 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 shutdown_connections 2024-05-08T23:21:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 1 -- 172.21.15.16:0/2992716878 wait complete. 2024-05-08T23:21:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.614+0000 7fce38471b80 20 client.16023 trim_cache size 0 max 16384 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.628+0000 7fce38471b80 1 Processor -- start 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 -- start start 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 0x563ff5034340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff502cc50 0x563ff5022480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff502c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff502caf0 con 0x563ff5032610 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff5037ff0 con 0x563ff502cc50 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce38471b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff5022f50 con 0x563ff5026f30 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce2a854700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff502c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce2a854700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff502c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51148/0 (socket says 172.21.15.16:51148) 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce2a854700 1 -- 172.21.15.16:0/705433997 learned_addr learned my addr 172.21.15.16:0/705433997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce29852700 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff502cc50 0x563ff5022480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.629+0000 7fce2a854700 1 -- 172.21.15.16:0/705433997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 msgr2=0x563ff5034340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce2a854700 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 0x563ff5034340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce2a854700 1 -- 172.21.15.16:0/705433997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff502cc50 msgr2=0x563ff5022480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce2a854700 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff502cc50 0x563ff5022480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce2a854700 1 -- 172.21.15.16:0/705433997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff4f3a560 con 0x563ff5032610 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce2a854700 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff502c5b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fce1c022000 tx=0x7fce1c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd228451f5e0377 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce195a5700 1 -- 172.21.15.16:0/705433997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00e070 con 0x563ff5032610 2024-05-08T23:21:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce195a5700 1 -- 172.21.15.16:0/705433997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce1c009450 con 0x563ff5032610 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.630+0000 7fce195a5700 1 -- 172.21.15.16:0/705433997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c003e20 con 0x563ff5032610 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 -- 172.21.15.16:0/705433997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 msgr2=0x563ff502c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff502c5b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fce1c022000 tx=0x7fce1c009960 comp rx=0 tx=0).stop 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 -- 172.21.15.16:0/705433997 shutdown_connections 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff5032610 0x563ff502c5b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 0x563ff5034340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 --2- 172.21.15.16:0/705433997 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff502cc50 0x563ff5022480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 -- 172.21.15.16:0/705433997 >> 172.21.15.16:0/705433997 conn(0x563ff4f82210 msgr2=0x563ff501b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 -- 172.21.15.16:0/705433997 shutdown_connections 2024-05-08T23:21:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 -- 172.21.15.16:0/705433997 wait complete. 2024-05-08T23:21:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 -- start start 2024-05-08T23:21:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.631+0000 7fce38471b80 1 Processor -- start 2024-05-08T23:21:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 0x563ff5036270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 0x563ff50367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032610 0x563ff5036cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff50a5060 con 0x563ff502cc50 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff50a51a0 con 0x563ff5032610 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff50a5310 con 0x563ff5026f30 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce38471b80 10 client.0 _ll_register_callbacks cb 0x563ff4f19b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 0x563ff50367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 0x563ff50367b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51150/0 (socket says 172.21.15.16:51150) 2024-05-08T23:21:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 -- 172.21.15.16:0/2317788686 learned_addr learned my addr 172.21.15.16:0/2317788686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce2a854700 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032610 0x563ff5036cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 -- 172.21.15.16:0/2317788686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 msgr2=0x563ff5036270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 0x563ff5036270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 -- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032610 msgr2=0x563ff5036cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032610 0x563ff5036cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.632+0000 7fce29852700 1 -- 172.21.15.16:0/2317788686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff4df53b0 con 0x563ff502cc50 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce29852700 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 0x563ff50367b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fce140cb290 tx=0x7fce14003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5284dbd3d8b3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce0dffb700 1 -- 172.21.15.16:0/2317788686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce140a3070 con 0x563ff502cc50 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce0dffb700 1 -- 172.21.15.16:0/2317788686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fce1400bb60 con 0x563ff502cc50 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce1c00f040 con 0x563ff502cc50 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce0dffb700 1 -- 172.21.15.16:0/2317788686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce1409eab0 con 0x563ff502cc50 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce38471b80 10 client.16029 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.633+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563ff4df9cd0 con 0x563ff502cc50 2024-05-08T23:21:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce0dffb700 1 -- 172.21.15.16:0/2317788686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce1c00f040 con 0x563ff502cc50 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce0dffb700 1 -- 172.21.15.16:0/2317788686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce1409aee0 con 0x563ff502cc50 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce38471b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce38471b80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce38471b80 20 client.16029 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce38471b80 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff50a6760 0x563ff50a8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce38471b80 4 client.16029 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ff4f6cec0 con 0x563ff50a6760 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce2a053700 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff50a6760 0x563ff50a8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce2a053700 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff50a6760 0x563ff50a8b80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89922fb392ea056 server_cookie=76c9478355c8b89a in_seq=0 out_seq=0 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.634+0000 7fce0dffb700 10 client.16029 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce0dffb700 1 -- 172.21.15.16:0/2317788686 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ff4f6cec0 con 0x563ff50a6760 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce0dffb700 10 client.16029 handle_command_reply: tid=0 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 client.16029 shutdown 2024-05-08T23:21:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 msgr2=0x563ff50367b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 0x563ff50367b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fce140cb290 tx=0x7fce14003f90 comp rx=0 tx=0).stop 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 shutdown_connections 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563ff50a6760 0x563ff50a8b80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563ff5032610 0x563ff5036cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ff5026f30 0x563ff5036270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 --2- 172.21.15.16:0/2317788686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ff502cc50 0x563ff50367b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 >> 172.21.15.16:0/2317788686 conn(0x563ff4f82210 msgr2=0x563ff5022480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce0dffb700 0 client.16029 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 shutdown_connections 2024-05-08T23:21:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 1 -- 172.21.15.16:0/2317788686 wait complete. 2024-05-08T23:21:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.635+0000 7fce38471b80 20 client.16029 trim_cache size 0 max 16384 2024-05-08T23:21:25.642 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:21:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce100a1290 msgr2=0x7fce100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce100a1290 0x7fce100a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0175d0 tx=0x7fce1c01d000 comp rx=0 tx=0).stop 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 msgr2=0x7fce24199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce24199a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce14009230 tx=0x7fce1400fa00 comp rx=0 tx=0).stop 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 shutdown_connections 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce100a1290 0x7fce100a3750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2410d940 0x7fce2419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce240694b0 0x7fce241994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 --2- 172.21.15.16:0/1352609194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24069dd0 0x7fce24199a30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 >> 172.21.15.16:0/1352609194 conn(0x7fce24076f70 msgr2=0x7fce2410f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 shutdown_connections 2024-05-08T23:21:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:25.636+0000 7fce2c2b7700 1 -- 172.21.15.16:0/1352609194 wait complete. 2024-05-08T23:21:26.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:26 smithi016 ceph-mon[42541]: pgmap v59: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T23:21:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:26 smithi189 ceph-mon[46919]: pgmap v59: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T23:21:26.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:26 smithi155 ceph-mon[47105]: pgmap v59: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T23:21:26.650 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:21:26.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:21:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 -- 172.21.15.16:0/3691973909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc105180 msgr2=0x7f9bcc10cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 --2- 172.21.15.16:0/3691973909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc105180 0x7f9bcc10cd00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b0a0 tx=0x7f9bc400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 -- 172.21.15.16:0/3691973909 shutdown_connections 2024-05-08T23:21:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 --2- 172.21.15.16:0/3691973909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc105180 0x7f9bcc10cd00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 --2- 172.21.15.16:0/3691973909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bcc100060 0x7f9bcc100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 --2- 172.21.15.16:0/3691973909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc100a10 0x7f9bcc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 -- 172.21.15.16:0/3691973909 >> 172.21.15.16:0/3691973909 conn(0x7f9bcc076f40 msgr2=0x7f9bcc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 -- 172.21.15.16:0/3691973909 shutdown_connections 2024-05-08T23:21:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.799+0000 7f9bd337c700 1 -- 172.21.15.16:0/3691973909 wait complete. 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.800+0000 7f9bd337c700 1 Processor -- start 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.800+0000 7f9bd337c700 1 -- start start 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd337c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bcc100060 0x7f9bcc1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd337c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc105180 0x7f9bcc198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd337c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 0x7f9bcc1a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd337c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1a3a10 con 0x7f9bcc198d70 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd337c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1a3b80 con 0x7f9bcc105180 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd337c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1a3d40 con 0x7f9bcc100060 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 0x7f9bcc1a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 0x7f9bcc1a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34994/0 (socket says 172.21.15.16:34994) 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 -- 172.21.15.16:0/2100963613 learned_addr learned my addr 172.21.15.16:0/2100963613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1118700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bcc100060 0x7f9bcc1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 -- 172.21.15.16:0/2100963613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bcc100060 msgr2=0x7f9bcc1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bcc100060 0x7f9bcc1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 -- 172.21.15.16:0/2100963613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc105180 msgr2=0x7f9bcc198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc105180 0x7f9bcc198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.801+0000 7f9bd1919700 1 -- 172.21.15.16:0/2100963613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4009d00 con 0x7f9bcc198d70 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.802+0000 7f9bd1919700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 0x7f9bcc1a3370 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4006130 tx=0x7f9bc4007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.802+0000 7f9bc27fc700 1 -- 172.21.15.16:0/2100963613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400e040 con 0x7f9bcc198d70 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.802+0000 7f9bc27fc700 1 -- 172.21.15.16:0/2100963613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bc400bc50 con 0x7f9bcc198d70 2024-05-08T23:21:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.802+0000 7f9bc27fc700 1 -- 172.21.15.16:0/2100963613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40052a0 con 0x7f9bcc198d70 2024-05-08T23:21:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.803+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc1a3fc0 con 0x7f9bcc198d70 2024-05-08T23:21:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.803+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1a43f0 con 0x7f9bcc198d70 2024-05-08T23:21:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.806+0000 7f9bc27fc700 1 -- 172.21.15.16:0/2100963613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9bc4005610 con 0x7f9bcc198d70 2024-05-08T23:21:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.807+0000 7f9bc27fc700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9bb80a1390 0x7f9bb80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.807+0000 7f9bc27fc700 1 -- 172.21.15.16:0/2100963613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f9bc40ce050 con 0x7f9bcc198d70 2024-05-08T23:21:26.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.809+0000 7f9bd1118700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9bb80a1390 0x7f9bb80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.809+0000 7f9bd1118700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9bb80a1390 0x7f9bb80a3850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8006fd0 tx=0x7f9bc8008040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 Processor -- start 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 -- start start 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab34a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab3447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab345160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab2971d0 con 0x55b6ab34a290 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab294b90 con 0x55b6ab34abb0 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab290480 con 0x55b6ab344d20 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab345160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab345160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:57674/0 (socket says 172.21.15.16:57674) 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.810+0000 7f9bd1919700 1 -- 172.21.15.16:0/4269322734 learned_addr learned my addr 172.21.15.16:0/4269322734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1118700 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab34a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1919700 1 -- 172.21.15.16:0/4269322734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 msgr2=0x55b6ab3447e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1919700 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab3447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1919700 1 -- 172.21.15.16:0/4269322734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 msgr2=0x55b6ab34a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1919700 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab34a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1919700 1 -- 172.21.15.16:0/4269322734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6ab111430 con 0x55b6ab344d20 2024-05-08T23:21:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1118700 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab34a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:26.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bd1919700 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab345160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009850 tx=0x7f9bc4029000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5cd7becb3e2c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bb77fe700 1 -- 172.21.15.16:0/4269322734 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc401a3b0 con 0x55b6ab344d20 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bb77fe700 1 -- 172.21.15.16:0/4269322734 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bc409b550 con 0x55b6ab344d20 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.811+0000 7f9bb77fe700 1 -- 172.21.15.16:0/4269322734 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40cb680 con 0x55b6ab344d20 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4269322734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 msgr2=0x55b6ab345160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab345160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009850 tx=0x7f9bc4029000 comp rx=0 tx=0).stop 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4269322734 shutdown_connections 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab345160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab34a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4269322734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab3447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.812+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4269322734 >> 172.21.15.16:0/4269322734 conn(0x55b6ab29e290 msgr2=0x55b6ab33ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:26.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.813+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4269322734 shutdown_connections 2024-05-08T23:21:26.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.813+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4269322734 wait complete. 2024-05-08T23:21:26.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.813+0000 7f9bdf536b80 1 -- start start 2024-05-08T23:21:26.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.813+0000 7f9bdf536b80 1 Processor -- start 2024-05-08T23:21:26.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab344d20 0x55b6ab34df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab34e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 0x55b6ab34e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab22c5e0 con 0x55b6ab344d20 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab22c280 con 0x55b6ab34a290 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab113720 con 0x55b6ab34abb0 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bdf536b80 10 client.0 _ll_register_callbacks cb 0x55b6ab235bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 0x55b6ab34e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 0x55b6ab34e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:57688/0 (socket says 172.21.15.16:57688) 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bd1919700 1 -- 172.21.15.16:0/4152820129 learned_addr learned my addr 172.21.15.16:0/4152820129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.814+0000 7f9bd1118700 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab344d20 0x55b6ab34df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bd1919700 1 -- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 msgr2=0x55b6ab34e4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bd1919700 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab34e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bd1919700 1 -- 172.21.15.16:0/4152820129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab344d20 msgr2=0x55b6ab34df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bd1919700 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab344d20 0x55b6ab34df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bd1919700 1 -- 172.21.15.16:0/4152820129 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1a43f0 con 0x55b6ab34abb0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bd1919700 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 0x55b6ab34e9e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9bc409ab50 tx=0x7f9bc400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670f906dc1697a1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/4152820129 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40cb890 con 0x55b6ab34abb0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/4152820129 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f9bc409ec80 con 0x55b6ab34abb0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6ab1e92f0 con 0x55b6ab34abb0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.815+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/4152820129 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bc409b0b0 con 0x55b6ab34abb0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.816+0000 7f9bdf536b80 10 client.25855 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.816+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b6ab0bdbe0 con 0x55b6ab34abb0 2024-05-08T23:21:26.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.816+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/4152820129 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b6ab1e92f0 con 0x55b6ab34abb0 2024-05-08T23:21:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.816+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/4152820129 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9bc40a2db0 con 0x55b6ab34abb0 2024-05-08T23:21:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bdf536b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bdf536b80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:26.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bdf536b80 20 client.25855 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab35de20 0x55b6ab360240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bdf536b80 4 client.25855 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6ab30f890 con 0x55b6ab35de20 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bd1118700 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab35de20 0x55b6ab360240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bb4ff9700 10 client.25855 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.817+0000 7f9bd1118700 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab35de20 0x55b6ab360240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5048347fa7bdc44d server_cookie=e737126004562f0e in_seq=0 out_seq=0 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.818+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/4152820129 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6ab30f890 con 0x55b6ab35de20 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.818+0000 7f9bb4ff9700 10 client.25855 handle_command_reply: tid=0 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.818+0000 7f9bdf536b80 1 client.25855 shutdown 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 msgr2=0x55b6ab34e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 0x55b6ab34e9e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9bc409ab50 tx=0x7f9bc400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 shutdown_connections 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab34abb0 0x55b6ab34e9e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab344d20 0x55b6ab34df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab35de20 0x55b6ab360240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/4152820129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab34e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 >> 172.21.15.16:0/4152820129 conn(0x55b6ab29e290 msgr2=0x55b6ab33a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bb4ff9700 0 client.25855 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 shutdown_connections 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 1 -- 172.21.15.16:0/4152820129 wait complete. 2024-05-08T23:21:26.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.819+0000 7f9bdf536b80 20 client.25855 trim_cache size 0 max 16384 2024-05-08T23:21:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.854+0000 7f9bdf536b80 1 Processor -- start 2024-05-08T23:21:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.854+0000 7f9bdf536b80 1 -- start start 2024-05-08T23:21:26.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab3447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab349750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab33ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab1e92f0 con 0x55b6ab34a290 2024-05-08T23:21:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab333190 con 0x55b6ab34abb0 2024-05-08T23:21:26.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab35ae50 con 0x55b6ab344d20 2024-05-08T23:21:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1118700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab3447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1118700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab3447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:57698/0 (socket says 172.21.15.16:57698) 2024-05-08T23:21:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab33ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1118700 1 -- 172.21.15.16:0/2699353917 learned_addr learned my addr 172.21.15.16:0/2699353917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1919700 1 -- 172.21.15.16:0/2699353917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 msgr2=0x55b6ab3447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1919700 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab3447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.855+0000 7f9bd1919700 1 -- 172.21.15.16:0/2699353917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 msgr2=0x55b6ab349750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.856+0000 7f9bd1919700 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab349750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.856+0000 7f9bd1919700 1 -- 172.21.15.16:0/2699353917 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6ab34dac0 con 0x55b6ab34abb0 2024-05-08T23:21:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.856+0000 7f9bd1919700 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab33ada0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4097350 tx=0x7f9bc4007550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5923f7232dd5e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.856+0000 7f9bb7fff700 1 -- 172.21.15.16:0/2699353917 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc409ae50 con 0x55b6ab34abb0 2024-05-08T23:21:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.856+0000 7f9bb7fff700 1 -- 172.21.15.16:0/2699353917 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bc40cb890 con 0x55b6ab34abb0 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 -- 172.21.15.16:0/2699353917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 msgr2=0x55b6ab33ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab33ada0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4097350 tx=0x7f9bc4007550 comp rx=0 tx=0).stop 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 -- 172.21.15.16:0/2699353917 shutdown_connections 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34abb0 0x55b6ab33ada0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab3447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/2699353917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34a290 0x55b6ab349750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 -- 172.21.15.16:0/2699353917 >> 172.21.15.16:0/2699353917 conn(0x55b6ab29e290 msgr2=0x55b6ab349040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 -- 172.21.15.16:0/2699353917 shutdown_connections 2024-05-08T23:21:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.857+0000 7f9bdf536b80 1 -- 172.21.15.16:0/2699353917 wait complete. 2024-05-08T23:21:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.858+0000 7f9bdf536b80 1 -- start start 2024-05-08T23:21:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.858+0000 7f9bdf536b80 1 Processor -- start 2024-05-08T23:21:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab353540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab353a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34abb0 0x55b6ab3b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab3541f0 con 0x55b6ab34abb0 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab3b3820 con 0x55b6ab34a290 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6ab3b39e0 con 0x55b6ab344d20 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bdf536b80 10 client.0 _ll_register_callbacks cb 0x55b6ab235bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34abb0 0x55b6ab3b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd1919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34abb0 0x55b6ab3b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35030/0 (socket says 172.21.15.16:35030) 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd1919700 1 -- 172.21.15.16:0/275666911 learned_addr learned my addr 172.21.15.16:0/275666911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd0917700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab353a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd1118700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab353540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd0917700 1 -- 172.21.15.16:0/275666911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 msgr2=0x55b6ab353540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd0917700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab353540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd0917700 1 -- 172.21.15.16:0/275666911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34abb0 msgr2=0x55b6ab3b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd0917700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34abb0 0x55b6ab3b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.859+0000 7f9bd0917700 1 -- 172.21.15.16:0/275666911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6ab0bdbe0 con 0x55b6ab34a290 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bd0917700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab353a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55b6ab34f530 tx=0x7f9bbc00e6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f369392cb4b059d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/275666911 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc004230 con 0x55b6ab34a290 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/275666911 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f9bbc014070 con 0x55b6ab34a290 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bc40a2fb0 con 0x55b6ab34a290 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/275666911 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bbc00ee80 con 0x55b6ab34a290 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bdf536b80 10 client.25814 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b6ab2565e0 con 0x55b6ab34a290 2024-05-08T23:21:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.860+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/275666911 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9bc40a2fb0 con 0x55b6ab34a290 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/275666911 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9bbc018500 con 0x55b6ab34a290 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bdf536b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bdf536b80 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bdf536b80 20 client.25814 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab3b5220 0x55b6ab3b7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bdf536b80 4 client.25814 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.861+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f9bc409b550 con 0x55b6ab3b5220 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.862+0000 7f9bd1118700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab3b5220 0x55b6ab3b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.862+0000 7f9bb4ff9700 10 client.25814 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.862+0000 7f9bd1118700 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab3b5220 0x55b6ab3b7640 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f576e114498e5b59 server_cookie=feba765a8c7a9ebb in_seq=0 out_seq=0 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bb4ff9700 1 -- 172.21.15.16:0/275666911 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f9bc409b550 con 0x55b6ab3b5220 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bb4ff9700 10 client.25814 handle_command_reply: tid=0 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 client.25814 shutdown 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 msgr2=0x55b6ab353a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab353a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55b6ab34f530 tx=0x7f9bbc00e6a0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 shutdown_connections 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6ab34abb0 0x55b6ab3b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6ab344d20 0x55b6ab353540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b6ab3b5220 0x55b6ab3b7640 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 --2- 172.21.15.16:0/275666911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b6ab34a290 0x55b6ab353a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 >> 172.21.15.16:0/275666911 conn(0x55b6ab29e290 msgr2=0x55b6ab34dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bb4ff9700 0 client.25814 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 shutdown_connections 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 1 -- 172.21.15.16:0/275666911 wait complete. 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.864+0000 7f9bdf536b80 20 client.25814 trim_cache size 0 max 16384 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "591286da-49d2-4416-af17-923b62d13986", 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:21:26.866 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9bb80a1390 msgr2=0x7f9bb80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9bb80a1390 0x7f9bb80a3850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8006fd0 tx=0x7f9bc8008040 comp rx=0 tx=0).stop 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 msgr2=0x7f9bcc1a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 0x7f9bcc1a3370 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4006130 tx=0x7f9bc4007ad0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 shutdown_connections 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9bb80a1390 0x7f9bb80a3850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bcc198d70 0x7f9bcc1a3370 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bcc100060 0x7f9bcc1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 --2- 172.21.15.16:0/2100963613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9bcc105180 0x7f9bcc198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 >> 172.21.15.16:0/2100963613 conn(0x7f9bcc076f40 msgr2=0x7f9bcc106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 shutdown_connections 2024-05-08T23:21:26.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:26.865+0000 7f9bd337c700 1 -- 172.21.15.16:0/2100963613 wait complete. 2024-05-08T23:21:26.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:27.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 -- 172.21.15.16:0/1479139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069e20 msgr2=0x7f4ed010d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 --2- 172.21.15.16:0/1479139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069e20 0x7f4ed010d3b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009990 tx=0x7f4ec0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 -- 172.21.15.16:0/1479139926 shutdown_connections 2024-05-08T23:21:27.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 --2- 172.21.15.16:0/1479139926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed010d8f0 0x7f4ed010fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 --2- 172.21.15.16:0/1479139926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ed0069500 0x7f4ed00698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 --2- 172.21.15.16:0/1479139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069e20 0x7f4ed010d3b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 -- 172.21.15.16:0/1479139926 >> 172.21.15.16:0/1479139926 conn(0x7f4ed0076fa0 msgr2=0x7f4ed00773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 -- 172.21.15.16:0/1479139926 shutdown_connections 2024-05-08T23:21:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.025+0000 7f4ed8337700 1 -- 172.21.15.16:0/1479139926 wait complete. 2024-05-08T23:21:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.026+0000 7f4ed8337700 1 Processor -- start 2024-05-08T23:21:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.026+0000 7f4ed8337700 1 -- start start 2024-05-08T23:21:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed8337700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069500 0x7f4ed0102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed8337700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 0x7f4ed0102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed8337700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ed010d8f0 0x7f4ed0102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed8337700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed01a9b10 con 0x7f4ed0069e20 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed8337700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0199220 con 0x7f4ed010d8f0 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed8337700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0199390 con 0x7f4ed0069500 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 0x7f4ed0102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 0x7f4ed0102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35046/0 (socket says 172.21.15.16:35046) 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 -- 172.21.15.16:0/2266724700 learned_addr learned my addr 172.21.15.16:0/2266724700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069500 0x7f4ed0102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ed010d8f0 0x7f4ed0102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 -- 172.21.15.16:0/2266724700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069500 msgr2=0x7f4ed0102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069500 0x7f4ed0102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 -- 172.21.15.16:0/2266724700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ed010d8f0 msgr2=0x7f4ed0102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ed010d8f0 0x7f4ed0102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.027+0000 7f4ed58d2700 1 -- 172.21.15.16:0/2266724700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00095f0 con 0x7f4ed0069e20 2024-05-08T23:21:27.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.028+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 0x7f4ed0102560 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0005740 tx=0x7f4ec0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.028+0000 7f4ec77fe700 1 -- 172.21.15.16:0/2266724700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec001d070 con 0x7f4ed0069e20 2024-05-08T23:21:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.028+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed0199610 con 0x7f4ed0069e20 2024-05-08T23:21:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.028+0000 7f4ec77fe700 1 -- 172.21.15.16:0/2266724700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec0022470 con 0x7f4ed0069e20 2024-05-08T23:21:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.028+0000 7f4ec77fe700 1 -- 172.21.15.16:0/2266724700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0017650 con 0x7f4ed0069e20 2024-05-08T23:21:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.029+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed0199b10 con 0x7f4ed0069e20 2024-05-08T23:21:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.031+0000 7f4ec77fe700 1 -- 172.21.15.16:0/2266724700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4ec00225e0 con 0x7f4ed0069e20 2024-05-08T23:21:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.032+0000 7f4ec77fe700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4ebc0a1290 0x7f4ebc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.033+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4ebc0a1290 0x7f4ebc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.033+0000 7f4ec77fe700 1 -- 172.21.15.16:0/2266724700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4ec00ce440 con 0x7f4ed0069e20 2024-05-08T23:21:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.033+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4ebc0a1290 0x7f4ebc0a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005fd0 tx=0x7f4ecc005f60 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.034+0000 7f4ee44f1b80 1 Processor -- start 2024-05-08T23:21:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.034+0000 7f4ee44f1b80 1 -- start start 2024-05-08T23:21:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b8f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b89350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b89cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2ae0150 con 0x5654c2b8f720 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2addb10 con 0x5654c2b8ee00 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2ad9400 con 0x5654c2b89890 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b89350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b89350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35058/0 (socket says 172.21.15.16:35058) 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1200587827 learned_addr learned my addr 172.21.15.16:0/1200587827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b8f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1200587827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 msgr2=0x5654c2b89cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b89cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1200587827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 msgr2=0x5654c2b8f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b8f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1200587827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654c295a3b0 con 0x5654c2b8f720 2024-05-08T23:21:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.035+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b8f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.036+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b89350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0097840 tx=0x7f4ec0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79440ea8e12e7c7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.036+0000 7f4ec4da4700 1 -- 172.21.15.16:0/1200587827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec009ba40 con 0x5654c2b8f720 2024-05-08T23:21:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.036+0000 7f4ec4da4700 1 -- 172.21.15.16:0/1200587827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec009b340 con 0x5654c2b8f720 2024-05-08T23:21:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.036+0000 7f4ec4da4700 1 -- 172.21.15.16:0/1200587827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0022900 con 0x5654c2b8f720 2024-05-08T23:21:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1200587827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 msgr2=0x5654c2b89350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b89350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0097840 tx=0x7f4ec0009310 comp rx=0 tx=0).stop 2024-05-08T23:21:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1200587827 shutdown_connections 2024-05-08T23:21:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b89cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b8f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1200587827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b89350 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1200587827 >> 172.21.15.16:0/1200587827 conn(0x5654c2ae7210 msgr2=0x5654c2b83df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1200587827 shutdown_connections 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.037+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1200587827 wait complete. 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 -- start start 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 Processor -- start 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b95750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8ee00 0x5654c2b929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8f720 0x5654c2b93110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2a75560 con 0x5654c2b8ee00 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2a75200 con 0x5654c2b8f720 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c295c6a0 con 0x5654c2b89890 2024-05-08T23:21:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.038+0000 7f4ee44f1b80 10 client.0 _ll_register_callbacks cb 0x5654c2a7eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b95750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b95750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:57768/0 (socket says 172.21.15.16:57768) 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 -- 172.21.15.16:0/569717327 learned_addr learned my addr 172.21.15.16:0/569717327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8f720 0x5654c2b93110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 -- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8f720 msgr2=0x5654c2b93110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8f720 0x5654c2b93110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 -- 172.21.15.16:0/569717327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8ee00 msgr2=0x5654c2b929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8ee00 0x5654c2b929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 -- 172.21.15.16:0/569717327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc00f8b0 con 0x5654c2b89890 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b95750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0116c0 tx=0x7f4ecc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab760c4ef8921c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.039+0000 7f4eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/569717327 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00fb40 con 0x5654c2b89890 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/569717327 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4ecc01f840 con 0x5654c2b89890 2024-05-08T23:21:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654c2a32270 con 0x5654c2b89890 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/569717327 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc01fa00 con 0x5654c2b89890 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 10 client.25870 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f4ed0199b10 con 0x5654c2b89890 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/569717327 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5654c2a32270 con 0x5654c2b89890 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/569717327 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ecc01ebb0 con 0x5654c2b89890 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 10 client.25870 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 20 client.25870 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2ba2ee0 0x5654c2ba5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 4 client.25870 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.040+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654c2b58810 con 0x5654c2ba2ee0 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.041+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2ba2ee0 0x5654c2ba5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.042+0000 7f4eb9ffb700 10 client.25870 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.042+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2ba2ee0 0x5654c2ba5300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc537695c5fbf68 server_cookie=c30c0d1f8c36422a in_seq=0 out_seq=0 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.042+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/569717327 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5654c2b58810 con 0x5654c2ba2ee0 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.042+0000 7f4eb9ffb700 10 client.25870 handle_command_reply: tid=0 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.042+0000 7f4ee44f1b80 1 client.25870 shutdown 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 msgr2=0x5654c2b95750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b95750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0116c0 tx=0x7f4ecc004750 comp rx=0 tx=0).stop 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 shutdown_connections 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2ba2ee0 0x5654c2ba5300 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8f720 0x5654c2b93110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b89890 0x5654c2b95750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/569717327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8ee00 0x5654c2b929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 >> 172.21.15.16:0/569717327 conn(0x5654c2ae7210 msgr2=0x5654c2b835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4eb9ffb700 0 client.25870 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 shutdown_connections 2024-05-08T23:21:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/569717327 wait complete. 2024-05-08T23:21:27.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.043+0000 7f4ee44f1b80 20 client.25870 trim_cache size 0 max 16384 2024-05-08T23:21:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 Processor -- start 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 -- start start 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 0x5654c2b92970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 0x5654c2b83d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b95280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2b7e890 con 0x5654c2b8f720 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2b7c070 con 0x5654c2b89890 2024-05-08T23:21:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2ba05a0 con 0x5654c2b8ee00 2024-05-08T23:21:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ed60d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 0x5654c2b92970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 0x5654c2b83d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ed60d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 0x5654c2b92970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56030/0 (socket says 172.21.15.16:56030) 2024-05-08T23:21:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 0x5654c2b83d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:57778/0 (socket says 172.21.15.16:57778) 2024-05-08T23:21:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.062+0000 7f4ed60d3700 1 -- 172.21.15.16:0/1605886680 learned_addr learned my addr 172.21.15.16:0/1605886680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b95280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1605886680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 msgr2=0x5654c2b92970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 0x5654c2b92970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1605886680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 msgr2=0x5654c2b95280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b95280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed58d2700 1 -- 172.21.15.16:0/1605886680 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654c2906b60 con 0x5654c2b8ee00 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 0x5654c2b92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b95280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ed58d2700 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 0x5654c2b83d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5654c2b95cc0 tx=0x7f4ec0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17abf137d423bc1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ec55a5700 1 -- 172.21.15.16:0/1605886680 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec009f170 con 0x5654c2b8ee00 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ec55a5700 1 -- 172.21.15.16:0/1605886680 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec00a3730 con 0x5654c2b8ee00 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.063+0000 7f4ec55a5700 1 -- 172.21.15.16:0/1605886680 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec009b5a0 con 0x5654c2b8ee00 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1605886680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 msgr2=0x5654c2b83d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 0x5654c2b83d20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5654c2b95cc0 tx=0x7f4ec0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1605886680 shutdown_connections 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b8f720 0x5654c2b95280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b89890 0x5654c2b92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/1605886680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8ee00 0x5654c2b83d20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1605886680 >> 172.21.15.16:0/1605886680 conn(0x5654c2ae7210 msgr2=0x5654c2b92260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1605886680 shutdown_connections 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.064+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/1605886680 wait complete. 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 -- start start 2024-05-08T23:21:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 Processor -- start 2024-05-08T23:21:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b89890 0x5654c2b99e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8f720 0x5654c2b8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2b8b650 con 0x5654c2b89890 2024-05-08T23:21:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2b8b7c0 con 0x5654c2b8ee00 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c2b8b900 con 0x5654c2b8f720 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ee44f1b80 10 client.0 _ll_register_callbacks cb 0x5654c2a7eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.065+0000 7f4ed60d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b89890 0x5654c2b99e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed58d2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b9a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:56044/0 (socket says 172.21.15.16:56044) 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed58d2700 1 -- 172.21.15.16:0/3209429131 learned_addr learned my addr 172.21.15.16:0/3209429131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8f720 0x5654c2b8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 -- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 msgr2=0x5654c2b9a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b9a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 -- 172.21.15.16:0/3209429131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b89890 msgr2=0x5654c2b99e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b89890 0x5654c2b99e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 -- 172.21.15.16:0/3209429131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed0199b10 con 0x5654c2b8f720 2024-05-08T23:21:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4ed68d4700 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8f720 0x5654c2b8af40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8007af0 tx=0x7f4ec8007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca2c267908ba1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.066+0000 7f4eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/3209429131 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800e030 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/3209429131 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4ec8016840 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec009b340 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/3209429131 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec8003a00 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4ee44f1b80 10 client.25876 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5654c295a3b0 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/3209429131 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ec009b340 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.067+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/3209429131 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4ec8004250 con 0x5654c2b8f720 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ee44f1b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ee44f1b80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ee44f1b80 20 client.25876 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2c0b770 0x5654c2c0db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:27.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ee44f1b80 4 client.25876 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5654c2ad1ec0 con 0x5654c2c0b770 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.068+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2c0b770 0x5654c2c0db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ed60d3700 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2c0b770 0x5654c2c0db90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=904f6455d74d626e server_cookie=a758de87d9926283 in_seq=0 out_seq=0 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4eb9ffb700 10 client.25876 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4eb9ffb700 1 -- 172.21.15.16:0/3209429131 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5654c2ad1ec0 con 0x5654c2c0b770 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4eb9ffb700 10 client.25876 handle_command_reply: tid=0 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 client.25876 shutdown 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8f720 msgr2=0x5654c2b8af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8f720 0x5654c2b8af40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8007af0 tx=0x7f4ec8007b20 comp rx=0 tx=0).stop 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 shutdown_connections 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c2b8f720 0x5654c2b8af40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5654c2b89890 0x5654c2b99e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5654c2c0b770 0x5654c2c0db90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 --2- 172.21.15.16:0/3209429131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5654c2b8ee00 0x5654c2b9a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 >> 172.21.15.16:0/3209429131 conn(0x5654c2ae7210 msgr2=0x5654c2b8fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.069+0000 7f4eb9ffb700 0 client.25876 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 shutdown_connections 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ee44f1b80 1 -- 172.21.15.16:0/3209429131 wait complete. 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ee44f1b80 20 client.25876 trim_cache size 0 max 16384 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stdout: "591286da-49d2-4416-af17-923b62d13986": { 2024-05-08T23:21:27.073 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:21:27.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "591286da-49d2-4416-af17-923b62d13986", 2024-05-08T23:21:27.074 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:21:27.074 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:21:27.074 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:21:27.074 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:21:27.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4ebc0a1290 msgr2=0x7f4ebc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4ebc0a1290 0x7f4ebc0a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005fd0 tx=0x7f4ecc005f60 comp rx=0 tx=0).stop 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 msgr2=0x7f4ed0102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 0x7f4ed0102560 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0005740 tx=0x7f4ec0005e80 comp rx=0 tx=0).stop 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 shutdown_connections 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f4ebc0a1290 0x7f4ebc0a3750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ed010d8f0 0x7f4ed0102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed0069500 0x7f4ed0102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 --2- 172.21.15.16:0/2266724700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ed0069e20 0x7f4ed0102560 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.070+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 >> 172.21.15.16:0/2266724700 conn(0x7f4ed0076fa0 msgr2=0x7f4ed010f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.071+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 shutdown_connections 2024-05-08T23:21:27.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:27.071+0000 7f4ed8337700 1 -- 172.21.15.16:0/2266724700 wait complete. 2024-05-08T23:21:27.087 INFO:tasks.cephfs.filesystem:scrub status for tag:591286da-49d2-4416-af17-923b62d13986 - {'path': '/', 'tag': '591286da-49d2-4416-af17-923b62d13986', 'options': 'recursive,force'} 2024-05-08T23:21:27.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:27 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:21:27.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:27 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:27.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:27 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:21:27.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:27 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:27.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:27 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:21:27.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:27 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:27.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:27 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:21:27.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:27 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:27.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:27 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:21:27.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:27 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:27.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:27 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:21:27.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:27 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:21:28.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:28 smithi189 ceph-mon[46919]: pgmap v60: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T23:21:28.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:28 smithi155 ceph-mon[47105]: pgmap v60: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T23:21:28.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:28 smithi016 ceph-mon[42541]: pgmap v60: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-05-08T23:21:30.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:30 smithi016 ceph-mon[42541]: pgmap v61: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:21:30.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:30 smithi189 ceph-mon[46919]: pgmap v61: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:21:30.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:30 smithi155 ceph-mon[47105]: pgmap v61: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-08T23:21:31.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:31 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:21:31.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:31 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:21:31.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:31 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:21:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:32 smithi016 ceph-mon[42541]: pgmap v62: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:32.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:32 smithi189 ceph-mon[46919]: pgmap v62: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:32.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:32 smithi155 ceph-mon[47105]: pgmap v62: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:34 smithi016 ceph-mon[42541]: pgmap v63: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:21:34.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:34 smithi189 ceph-mon[46919]: pgmap v63: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:21:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:34 smithi155 ceph-mon[47105]: pgmap v63: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-08T23:21:34.811 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 84% (10389/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T23:21:35.777 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T23:21:35.777 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.777 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-05-08T23:21:35.778 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T23:21:35.784 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T23:21:35.812 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T23:21:35.814 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:21:35.814 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T23:21:36.295 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T23:21:36.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:36 smithi016 ceph-mon[42541]: pgmap v64: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:36.364 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T23:21:36.407 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T23:21:36.409 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:21:36.409 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T23:21:36.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:36 smithi189 ceph-mon[46919]: pgmap v64: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:36.450 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T23:21:36.454 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:21:36.454 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T23:21:36.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:36 smithi155 ceph-mon[47105]: pgmap v64: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:36.514 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-05-08T23:21:36.516 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-05-08T23:21:36.516 DEBUG:teuthology.orchestra.run.smithi016:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-05-08T23:21:36.588 INFO:tasks.workunit.client.0.smithi016.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-05-08T23:21:36.589 INFO:tasks.workunit.client.0.smithi016.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-05-08T23:21:36.589 INFO:tasks.workunit.client.0.smithi016.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-05-08T23:21:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:38 smithi016 ceph-mon[42541]: pgmap v65: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:38.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:38 smithi189 ceph-mon[46919]: pgmap v65: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:38 smithi155 ceph-mon[47105]: pgmap v65: 129 pgs: 129 active+clean; 1.3 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-08T23:21:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:40 smithi016 ceph-mon[42541]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-08T23:21:40.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:40 smithi189 ceph-mon[46919]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-08T23:21:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:40 smithi155 ceph-mon[47105]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-08T23:21:42.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:42 smithi016 ceph-mon[42541]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-05-08T23:21:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:42 smithi189 ceph-mon[46919]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-05-08T23:21:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:42 smithi155 ceph-mon[47105]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-05-08T23:21:43.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:43 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:21:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:43 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:21:43.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:43 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:21:44.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:21:44.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:21:44.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:21:44.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:44 smithi016 ceph-mon[42541]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-05-08T23:21:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:21:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:21:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:21:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:44 smithi189 ceph-mon[46919]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-05-08T23:21:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:21:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:21:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:21:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:44 smithi155 ceph-mon[47105]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 KiB/s wr, 11 op/s 2024-05-08T23:21:46.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:46 smithi016 ceph-mon[42541]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-05-08T23:21:46.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:46 smithi189 ceph-mon[46919]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-05-08T23:21:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:46 smithi155 ceph-mon[47105]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-05-08T23:21:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:48 smithi016 ceph-mon[42541]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-05-08T23:21:48.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:48 smithi189 ceph-mon[46919]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-05-08T23:21:48.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:48 smithi155 ceph-mon[47105]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 KiB/s wr, 8 op/s 2024-05-08T23:21:50.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:50 smithi016 ceph-mon[42541]: pgmap v71: 129 pgs: 129 active+clean; 1.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-08T23:21:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:50 smithi189 ceph-mon[46919]: pgmap v71: 129 pgs: 129 active+clean; 1.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-08T23:21:50.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:50 smithi155 ceph-mon[47105]: pgmap v71: 129 pgs: 129 active+clean; 1.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-05-08T23:21:52.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:52 smithi016 ceph-mon[42541]: pgmap v72: 129 pgs: 129 active+clean; 1.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:52 smithi189 ceph-mon[46919]: pgmap v72: 129 pgs: 129 active+clean; 1.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:52 smithi155 ceph-mon[47105]: pgmap v72: 129 pgs: 129 active+clean; 1.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:54.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:54 smithi016 ceph-mon[42541]: pgmap v73: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T23:21:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:54 smithi189 ceph-mon[46919]: pgmap v73: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T23:21:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:54 smithi155 ceph-mon[47105]: pgmap v73: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T23:21:56.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:56 smithi016 ceph-mon[42541]: pgmap v74: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:56.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:56 smithi189 ceph-mon[46919]: pgmap v74: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:56.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:56 smithi155 ceph-mon[47105]: pgmap v74: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:57.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 -- 172.21.15.16:0/582052351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c069dd0 msgr2=0x7fdd5c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 --2- 172.21.15.16:0/582052351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c069dd0 0x7fdd5c105b70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009990 tx=0x7fdd58009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 -- 172.21.15.16:0/582052351 shutdown_connections 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 --2- 172.21.15.16:0/582052351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 0x7fdd5c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 --2- 172.21.15.16:0/582052351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 0x7fdd5c069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 --2- 172.21.15.16:0/582052351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c069dd0 0x7fdd5c105b70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 -- 172.21.15.16:0/582052351 >> 172.21.15.16:0/582052351 conn(0x7fdd5c076ba0 msgr2=0x7fdd5c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.236+0000 7fdd6457f700 1 -- 172.21.15.16:0/582052351 shutdown_connections 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 -- 172.21.15.16:0/582052351 wait complete. 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 Processor -- start 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 -- start start 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 0x7fdd5c1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 0x7fdd5c100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 0x7fdd5c1a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1a5ac0 con 0x7fdd5c100c40 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1a5c30 con 0x7fdd5c069420 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.237+0000 7fdd6457f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1a5df0 con 0x7fdd5c106250 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 0x7fdd5c1a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 0x7fdd5c1a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44676/0 (socket says 172.21.15.16:44676) 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 -- 172.21.15.16:0/4032226411 learned_addr learned my addr 172.21.15.16:0/4032226411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 0x7fdd5c100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd6231b700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 0x7fdd5c1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 -- 172.21.15.16:0/4032226411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 msgr2=0x7fdd5c100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 0x7fdd5c100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 -- 172.21.15.16:0/4032226411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 msgr2=0x7fdd5c1001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 0x7fdd5c1001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 -- 172.21.15.16:0/4032226411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580095f0 con 0x7fdd5c100c40 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 0x7fdd5c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:57.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd6231b700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 0x7fdd5c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 0x7fdd5c1a54f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007ee0 tx=0x7fdd5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd537fe700 1 -- 172.21.15.16:0/4032226411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54017070 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd537fe700 1 -- 172.21.15.16:0/4032226411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5400f040 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c1a60d0 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd537fe700 1 -- 172.21.15.16:0/4032226411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54005710 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.238+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1a6560 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.241+0000 7fdd537fe700 1 -- 172.21.15.16:0/4032226411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdd54005870 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.241+0000 7fdd537fe700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdd480a1290 0x7fdd480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.241+0000 7fdd537fe700 1 -- 172.21.15.16:0/4032226411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fdd540cc860 con 0x7fdd5c100c40 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.241+0000 7fdd6231b700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdd480a1290 0x7fdd480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.242+0000 7fdd6231b700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdd480a1290 0x7fdd480a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005950 tx=0x7fdd4c0058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.242+0000 7fdd70739b80 1 Processor -- start 2024-05-08T23:21:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.242+0000 7fdd70739b80 1 -- start start 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae49e00 0x557a9ae4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 0x557a9ae44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae44cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd70739b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ad9b150 con 0x557a9ae4a720 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd70739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ad98b10 con 0x557a9ae49e00 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd70739b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ad94400 con 0x557a9ae44890 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 0x557a9ae44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 0x557a9ae44350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44688/0 (socket says 172.21.15.16:44688) 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 -- 172.21.15.16:0/2321493851 learned_addr learned my addr 172.21.15.16:0/2321493851 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd6231b700 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae49e00 0x557a9ae4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae44cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 -- 172.21.15.16:0/2321493851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 msgr2=0x557a9ae44cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae44cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 -- 172.21.15.16:0/2321493851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae49e00 msgr2=0x557a9ae4a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae49e00 0x557a9ae4a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 -- 172.21.15.16:0/2321493851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9ac153b0 con 0x557a9ae4a720 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd6231b700 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae49e00 0x557a9ae4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 0x557a9ae44350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009990 tx=0x7fdd58005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffeae2cd2c12b8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.243+0000 7fdd50ff9700 1 -- 172.21.15.16:0/2321493851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800b9d0 con 0x557a9ae4a720 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd50ff9700 1 -- 172.21.15.16:0/2321493851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5801e040 con 0x557a9ae4a720 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd50ff9700 1 -- 172.21.15.16:0/2321493851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58004700 con 0x557a9ae4a720 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 -- 172.21.15.16:0/2321493851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 msgr2=0x557a9ae44350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 0x557a9ae44350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009990 tx=0x7fdd58005e80 comp rx=0 tx=0).stop 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 -- 172.21.15.16:0/2321493851 shutdown_connections 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae44cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae49e00 0x557a9ae4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 --2- 172.21.15.16:0/2321493851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae4a720 0x557a9ae44350 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 -- 172.21.15.16:0/2321493851 >> 172.21.15.16:0/2321493851 conn(0x557a9ada2210 msgr2=0x557a9ae3edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 -- 172.21.15.16:0/2321493851 shutdown_connections 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.244+0000 7fdd70739b80 1 -- 172.21.15.16:0/2321493851 wait complete. 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 -- start start 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 Processor -- start 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae4db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae4e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae4e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ad30560 con 0x557a9ae49e00 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ad30200 con 0x557a9ae4a720 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ac176a0 con 0x557a9ae44890 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd70739b80 10 client.0 _ll_register_callbacks cb 0x557a9ad39b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.245+0000 7fdd61b1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae4e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae4e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44692/0 (socket says 172.21.15.16:44692) 2024-05-08T23:21:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 -- 172.21.15.16:0/3878717076 learned_addr learned my addr 172.21.15.16:0/3878717076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd6231b700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae4db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae4e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 -- 172.21.15.16:0/3878717076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 msgr2=0x557a9ae4db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae4db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 -- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 msgr2=0x557a9ae4e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae4e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 -- 172.21.15.16:0/3878717076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c10e680 con 0x557a9ae49e00 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae4e0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdd580098e0 tx=0x7fdd5800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27018ec11283c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd45ffb700 1 -- 172.21.15.16:0/3878717076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800f870 con 0x557a9ae49e00 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd45ffb700 1 -- 172.21.15.16:0/3878717076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fdd5800f9d0 con 0x557a9ae49e00 2024-05-08T23:21:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a9aced270 con 0x557a9ae49e00 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd45ffb700 1 -- 172.21.15.16:0/3878717076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd5800fb90 con 0x557a9ae49e00 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.246+0000 7fdd70739b80 10 client.16071 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a9abc1b60 con 0x557a9ae49e00 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd45ffb700 1 -- 172.21.15.16:0/3878717076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557a9aced270 con 0x557a9ae49e00 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd45ffb700 1 -- 172.21.15.16:0/3878717076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdd58021070 con 0x557a9ae49e00 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 20 client.16071 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9ae5dd80 0x557a9ae601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 4 client.16071 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a9ae13810 con 0x557a9ae5dd80 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.247+0000 7fdd6231b700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9ae5dd80 0x557a9ae601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.248+0000 7fdd6231b700 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9ae5dd80 0x557a9ae601a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d654118afa9b13 server_cookie=3405a830eeb469d3 in_seq=0 out_seq=0 2024-05-08T23:21:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.248+0000 7fdd45ffb700 10 client.16071 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd45ffb700 1 -- 172.21.15.16:0/3878717076 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a9ae13810 con 0x557a9ae5dd80 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd45ffb700 10 client.16071 handle_command_reply: tid=0 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 client.16071 shutdown 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 msgr2=0x557a9ae4e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae4e0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdd580098e0 tx=0x7fdd5800bed0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 shutdown_connections 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9ae5dd80 0x557a9ae601a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae44890 0x557a9ae4db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 --2- 172.21.15.16:0/3878717076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae4e0a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd45ffb700 0 client.16071 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 >> 172.21.15.16:0/3878717076 conn(0x557a9ada2210 msgr2=0x557a9ae3e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 shutdown_connections 2024-05-08T23:21:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 1 -- 172.21.15.16:0/3878717076 wait complete. 2024-05-08T23:21:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.249+0000 7fdd70739b80 20 client.16071 trim_cache size 0 max 16384 2024-05-08T23:21:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 Processor -- start 2024-05-08T23:21:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 -- start start 2024-05-08T23:21:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae44890 0x557a9ae49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae3ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae4a720 0x557a9ae4c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ae49890 con 0x557a9ae49e00 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ae5ad60 con 0x557a9ae44890 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.267+0000 7fdd70739b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9ae4f190 con 0x557a9ae4a720 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae3ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae3ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44708/0 (socket says 172.21.15.16:44708) 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 -- 172.21.15.16:0/865750599 learned_addr learned my addr 172.21.15.16:0/865750599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae4a720 0x557a9ae4c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd6231b700 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae44890 0x557a9ae49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 -- 172.21.15.16:0/865750599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae4a720 msgr2=0x557a9ae4c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae4a720 0x557a9ae4c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 -- 172.21.15.16:0/865750599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae44890 msgr2=0x557a9ae49350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae44890 0x557a9ae49350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 -- 172.21.15.16:0/865750599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9ad5a560 con 0x557a9ae49e00 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae3ed20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c00 tx=0x7fdd58017d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4f38a238e1e52a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd517fa700 1 -- 172.21.15.16:0/865750599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58009ee0 con 0x557a9ae49e00 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd517fa700 1 -- 172.21.15.16:0/865750599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5800ee70 con 0x557a9ae49e00 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.268+0000 7fdd517fa700 1 -- 172.21.15.16:0/865750599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800f8d0 con 0x557a9ae49e00 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 -- 172.21.15.16:0/865750599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 msgr2=0x557a9ae3ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae3ed20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdd58000c00 tx=0x7fdd58017d00 comp rx=0 tx=0).stop 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 -- 172.21.15.16:0/865750599 shutdown_connections 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae4a720 0x557a9ae4c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae44890 0x557a9ae49350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 --2- 172.21.15.16:0/865750599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae49e00 0x557a9ae3ed20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 -- 172.21.15.16:0/865750599 >> 172.21.15.16:0/865750599 conn(0x557a9ada2210 msgr2=0x557a9ae48c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 -- 172.21.15.16:0/865750599 shutdown_connections 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 -- 172.21.15.16:0/865750599 wait complete. 2024-05-08T23:21:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.269+0000 7fdd70739b80 1 -- start start 2024-05-08T23:21:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 Processor -- start 2024-05-08T23:21:57.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 0x557a9ae530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae49e00 0x557a9ae53630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9aeb33c0 con 0x557a9ae44890 2024-05-08T23:21:57.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9aeb3530 con 0x557a9ae4a720 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9aeb36f0 con 0x557a9ae49e00 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd70739b80 10 client.0 _ll_register_callbacks cb 0x557a9ad39b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd6231b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 0x557a9ae530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd6231b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 0x557a9ae530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44712/0 (socket says 172.21.15.16:44712) 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd6231b700 1 -- 172.21.15.16:0/1116501236 learned_addr learned my addr 172.21.15.16:0/1116501236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae49e00 0x557a9ae53630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.270+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd6231b700 1 -- 172.21.15.16:0/1116501236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae49e00 msgr2=0x557a9ae53630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd6231b700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae49e00 0x557a9ae53630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd6231b700 1 -- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 msgr2=0x557a9ae53b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd6231b700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae53b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd6231b700 1 -- 172.21.15.16:0/1116501236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a9abc1b60 con 0x557a9ae44890 2024-05-08T23:21:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae49e00 0x557a9ae53630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd62b1c700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae53b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd6231b700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 0x557a9ae530f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x557a9ae60a40 tx=0x7fdd4c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62546f582163e48b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd45ffb700 1 -- 172.21.15.16:0/1116501236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c017040 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd5801e040 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd45ffb700 1 -- 172.21.15.16:0/1116501236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fdd4c004440 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd45ffb700 1 -- 172.21.15.16:0/1116501236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4c0281d0 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd70739b80 10 client.16077 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a9ac19cd0 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.271+0000 7fdd45ffb700 1 -- 172.21.15.16:0/1116501236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd5801e040 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd45ffb700 1 -- 172.21.15.16:0/1116501236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdd4c003a90 con 0x557a9ae44890 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 20 client.16077 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9aeb4ea0 0x557a9aeb72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 4 client.16077 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a9ad8cec0 con 0x557a9aeb4ea0 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9aeb4ea0 0x557a9aeb72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd61b1a700 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9aeb4ea0 0x557a9aeb72c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9eceb79db04a40 server_cookie=8da0edda4a3ef42 in_seq=0 out_seq=0 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd45ffb700 10 client.16077 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd45ffb700 1 -- 172.21.15.16:0/1116501236 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557a9ad8cec0 con 0x557a9aeb4ea0 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd45ffb700 10 client.16077 handle_command_reply: tid=0 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.272+0000 7fdd70739b80 1 client.16077 shutdown 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 msgr2=0x557a9ae530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 0x557a9ae530f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x557a9ae60a40 tx=0x7fdd4c012000 comp rx=0 tx=0).stop 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 shutdown_connections 2024-05-08T23:21:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557a9aeb4ea0 0x557a9aeb72c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557a9ae4a720 0x557a9ae53b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a9ae44890 0x557a9ae530f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 --2- 172.21.15.16:0/1116501236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a9ae49e00 0x557a9ae53630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd45ffb700 0 client.16077 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 >> 172.21.15.16:0/1116501236 conn(0x557a9ada2210 msgr2=0x557a9ae4c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 shutdown_connections 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 1 -- 172.21.15.16:0/1116501236 wait complete. 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd70739b80 20 client.16077 trim_cache size 0 max 16384 2024-05-08T23:21:57.281 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:21:57.282 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:21:57.282 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:21:57.282 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:21:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdd480a1290 msgr2=0x7fdd480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdd480a1290 0x7fdd480a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005950 tx=0x7fdd4c0058e0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 msgr2=0x7fdd5c1a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 0x7fdd5c1a54f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007ee0 tx=0x7fdd5400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 shutdown_connections 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdd480a1290 0x7fdd480a3750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5c100c40 0x7fdd5c1a54f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd5c069420 0x7fdd5c1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 --2- 172.21.15.16:0/4032226411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd5c106250 0x7fdd5c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 >> 172.21.15.16:0/4032226411 conn(0x7fdd5c076ba0 msgr2=0x7fdd5c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 shutdown_connections 2024-05-08T23:21:57.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.273+0000 7fdd6457f700 1 -- 172.21.15.16:0/4032226411 wait complete. 2024-05-08T23:21:57.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:21:57.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 -- 172.21.15.16:0/2087007780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c1008f0 msgr2=0x7f807c104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 --2- 172.21.15.16:0/2087007780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c1008f0 0x7f807c104930 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8070009990 tx=0x7f8070009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 -- 172.21.15.16:0/2087007780 shutdown_connections 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 --2- 172.21.15.16:0/2087007780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f807c105060 0x7f807c10cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 --2- 172.21.15.16:0/2087007780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f807c0fff40 0x7f807c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 --2- 172.21.15.16:0/2087007780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c1008f0 0x7f807c104930 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 -- 172.21.15.16:0/2087007780 >> 172.21.15.16:0/2087007780 conn(0x7f807c0fb7e0 msgr2=0x7f807c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 -- 172.21.15.16:0/2087007780 shutdown_connections 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.424+0000 7f80829c3700 1 -- 172.21.15.16:0/2087007780 wait complete. 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.425+0000 7f80829c3700 1 Processor -- start 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.425+0000 7f80829c3700 1 -- start start 2024-05-08T23:21:57.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.425+0000 7f80829c3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f807c0fff40 0x7f807c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80829c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 0x7f807c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80829c3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f807c19bbe0 0x7f807c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80829c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c19c660 con 0x7f807c105060 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80829c3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c19c7d0 con 0x7f807c0fff40 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80829c3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c19c990 con 0x7f807c19bbe0 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 0x7f807c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 0x7f807c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44736/0 (socket says 172.21.15.16:44736) 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 -- 172.21.15.16:0/3166223703 learned_addr learned my addr 172.21.15.16:0/3166223703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f8080f60700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f807c19bbe0 0x7f807c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807bfff700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f807c0fff40 0x7f807c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 -- 172.21.15.16:0/3166223703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f807c19bbe0 msgr2=0x7f807c19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f807c19bbe0 0x7f807c19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 -- 172.21.15.16:0/3166223703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f807c0fff40 msgr2=0x7f807c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f807c0fff40 0x7f807c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 -- 172.21.15.16:0/3166223703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700095f0 con 0x7f807c105060 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f807b7fe700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 0x7f807c197840 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8070005950 tx=0x7f8070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807001d070 con 0x7f807c105060 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c1a3e90 con 0x7f807c105060 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8070021410 con 0x7f807c105060 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.426+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070017650 con 0x7f807c105060 2024-05-08T23:21:57.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.427+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c1a42b0 con 0x7f807c105060 2024-05-08T23:21:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.427+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c109340 con 0x7f807c105060 2024-05-08T23:21:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.429+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8070021580 con 0x7f807c105060 2024-05-08T23:21:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.429+0000 7f80797fa700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80640a1290 0x7f80640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.429+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f80700cea60 con 0x7f807c105060 2024-05-08T23:21:57.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.429+0000 7f807bfff700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80640a1290 0x7f80640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.430+0000 7f807bfff700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80640a1290 0x7f80640a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f806c009bf0 tx=0x7f806c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.433+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8070097c00 con 0x7f807c105060 2024-05-08T23:21:57.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.706+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f807c198600 con 0x7f807c105060 2024-05-08T23:21:57.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.707+0000 7f80797fa700 1 -- 172.21.15.16:0/3166223703 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f807009b220 con 0x7f807c105060 2024-05-08T23:21:57.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:21:57.708 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:21:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80640a1290 msgr2=0x7f80640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80640a1290 0x7f80640a3750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f806c009bf0 tx=0x7f806c009260 comp rx=0 tx=0).stop 2024-05-08T23:21:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 msgr2=0x7f807c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 0x7f807c197840 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8070005950 tx=0x7f8070004730 comp rx=0 tx=0).stop 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 shutdown_connections 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80640a1290 0x7f80640a3750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f807c19bbe0 0x7f807c19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f807c0fff40 0x7f807c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 --2- 172.21.15.16:0/3166223703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807c105060 0x7f807c197840 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 >> 172.21.15.16:0/3166223703 conn(0x7f807c0fb7e0 msgr2=0x7f807c068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 shutdown_connections 2024-05-08T23:21:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.709+0000 7f80829c3700 1 -- 172.21.15.16:0/3166223703 wait complete. 2024-05-08T23:21:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:21:57.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:21:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 -- 172.21.15.16:0/31610396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308068590 msgr2=0x7f3308068a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 --2- 172.21.15.16:0/31610396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308068590 0x7f3308068a10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3304009990 tx=0x7f3304009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 -- 172.21.15.16:0/31610396 shutdown_connections 2024-05-08T23:21:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 --2- 172.21.15.16:0/31610396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308103930 0x7f3308103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 --2- 172.21.15.16:0/31610396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308105e30 0x7f3308068050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 --2- 172.21.15.16:0/31610396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308068590 0x7f3308068a10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 -- 172.21.15.16:0/31610396 >> 172.21.15.16:0/31610396 conn(0x7f3308076f40 msgr2=0x7f3308077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 -- 172.21.15.16:0/31610396 shutdown_connections 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.864+0000 7f330f27b700 1 -- 172.21.15.16:0/31610396 wait complete. 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 Processor -- start 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 -- start start 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308068590 0x7f33081a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 0x7f33081a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308105e30 0x7f330819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330819dd70 con 0x7f3308103930 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330819dee0 con 0x7f3308105e30 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330f27b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330819e0a0 con 0x7f3308068590 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 0x7f33081a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 0x7f33081a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44748/0 (socket says 172.21.15.16:44748) 2024-05-08T23:21:57.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 -- 172.21.15.16:0/4076438011 learned_addr learned my addr 172.21.15.16:0/4076438011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330d818700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308105e30 0x7f330819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330d017700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308068590 0x7f33081a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 -- 172.21.15.16:0/4076438011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308068590 msgr2=0x7f33081a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308068590 0x7f33081a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 -- 172.21.15.16:0/4076438011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308105e30 msgr2=0x7f330819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.865+0000 7f330c816700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308105e30 0x7f330819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f330c816700 1 -- 172.21.15.16:0/4076438011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040095f0 con 0x7f3308103930 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f330d818700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308105e30 0x7f330819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:57.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f330d017700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308068590 0x7f33081a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:57.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f330c816700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 0x7f33081a3ad0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f33040036c0 tx=0x7f3304005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f32fe7fc700 1 -- 172.21.15.16:0/4076438011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330401d070 con 0x7f3308103930 2024-05-08T23:21:57.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f32fe7fc700 1 -- 172.21.15.16:0/4076438011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f330400fc00 con 0x7f3308103930 2024-05-08T23:21:57.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330819e320 con 0x7f3308103930 2024-05-08T23:21:57.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f32fe7fc700 1 -- 172.21.15.16:0/4076438011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304022c40 con 0x7f3308103930 2024-05-08T23:21:57.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.866+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33081aa580 con 0x7f3308103930 2024-05-08T23:21:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.868+0000 7f32fe7fc700 1 -- 172.21.15.16:0/4076438011 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3304022470 con 0x7f3308103930 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.869+0000 7f32fe7fc700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32f40a1290 0x7f32f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.869+0000 7f32fe7fc700 1 -- 172.21.15.16:0/4076438011 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f33040ce650 con 0x7f3308103930 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.869+0000 7f330d017700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32f40a1290 0x7f32f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.869+0000 7f330d017700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32f40a1290 0x7f32f40a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009b50 tx=0x7f32f8009550 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 Processor -- start 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 -- start start 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x5578816761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881676720 0x557881670350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881670890 0x557881670cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578815c7150 con 0x557881675e00 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578815c4b10 con 0x557881676720 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f331b435b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578815c0400 con 0x557881670890 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f330d017700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x5578816761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f330d017700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x5578816761e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44760/0 (socket says 172.21.15.16:44760) 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.870+0000 7f330d017700 1 -- 172.21.15.16:0/866844627 learned_addr learned my addr 172.21.15.16:0/866844627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d818700 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881670890 0x557881670cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330c816700 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881676720 0x557881670350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d017700 1 -- 172.21.15.16:0/866844627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881670890 msgr2=0x557881670cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d017700 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881670890 0x557881670cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d017700 1 -- 172.21.15.16:0/866844627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881676720 msgr2=0x557881670350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d017700 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881676720 0x557881670350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d017700 1 -- 172.21.15.16:0/866844627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578814413b0 con 0x557881675e00 2024-05-08T23:21:57.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f330d017700 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x5578816761e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32f800ed90 tx=0x7f32f8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824740e61eebb2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f32f37fe700 1 -- 172.21.15.16:0/866844627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8011be0 con 0x557881675e00 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f32f37fe700 1 -- 172.21.15.16:0/866844627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32f8011d40 con 0x557881675e00 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.871+0000 7f32f37fe700 1 -- 172.21.15.16:0/866844627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8010630 con 0x557881675e00 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 -- 172.21.15.16:0/866844627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 msgr2=0x5578816761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x5578816761e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32f800ed90 tx=0x7f32f8011000 comp rx=0 tx=0).stop 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 -- 172.21.15.16:0/866844627 shutdown_connections 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881670890 0x557881670cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x5578816761e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 --2- 172.21.15.16:0/866844627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881676720 0x557881670350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 -- 172.21.15.16:0/866844627 >> 172.21.15.16:0/866844627 conn(0x5578815ce210 msgr2=0x55788166adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 -- 172.21.15.16:0/866844627 shutdown_connections 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 -- 172.21.15.16:0/866844627 wait complete. 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.872+0000 7f331b435b80 1 -- start start 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 Processor -- start 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881679b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x55788167a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788167a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788155c560 con 0x557881675e00 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788155c200 con 0x557881670890 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578814436a0 con 0x557881676720 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f331b435b80 10 client.0 _ll_register_callbacks cb 0x557881565b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x55788167a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x55788167a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44776/0 (socket says 172.21.15.16:44776) 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 -- 172.21.15.16:0/2884690243 learned_addr learned my addr 172.21.15.16:0/2884690243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330d017700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881679b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330d818700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788167a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 -- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 msgr2=0x55788167a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788167a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 -- 172.21.15.16:0/2884690243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 msgr2=0x557881679b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.873+0000 7f330c816700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881679b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f330c816700 1 -- 172.21.15.16:0/2884690243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f800ea40 con 0x557881675e00 2024-05-08T23:21:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f330c816700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x55788167a040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f33040cc060 tx=0x7f330409b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36419534be4ffb46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f32f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2884690243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33040a39c0 con 0x557881675e00 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2884690243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f330400fd70 con 0x557881675e00 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557881519270 con 0x557881675e00 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2884690243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f330409b830 con 0x557881675e00 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f331b435b80 10 client.16095 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5578813edb60 con 0x557881675e00 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2884690243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557881519270 con 0x557881675e00 2024-05-08T23:21:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2884690243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f33040a3530 con 0x557881675e00 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f331b435b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f331b435b80 10 client.16095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.874+0000 7f331b435b80 20 client.16095 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.875+0000 7f331b435b80 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557881689d50 0x55788168c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.875+0000 7f331b435b80 4 client.16095 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.875+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55788163f810 con 0x557881689d50 2024-05-08T23:21:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.875+0000 7f330d017700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557881689d50 0x55788168c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.875+0000 7f330d017700 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557881689d50 0x55788168c170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16058fba362853a7 server_cookie=65693812a81d5d1d in_seq=0 out_seq=0 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.875+0000 7f32f0ff9700 10 client.16095 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2884690243 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55788163f810 con 0x557881689d50 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f32f0ff9700 10 client.16095 handle_command_reply: tid=0 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 client.16095 shutdown 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 msgr2=0x55788167a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x55788167a040 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f33040cc060 tx=0x7f330409b5d0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 shutdown_connections 2024-05-08T23:21:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788167a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557881689d50 0x55788168c170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881679b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 --2- 172.21.15.16:0/2884690243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x55788167a040 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f32f0ff9700 0 client.16095 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 >> 172.21.15.16:0/2884690243 conn(0x5578815ce210 msgr2=0x55788166a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 shutdown_connections 2024-05-08T23:21:57.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 1 -- 172.21.15.16:0/2884690243 wait complete. 2024-05-08T23:21:57.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.876+0000 7f331b435b80 20 client.16095 trim_cache size 0 max 16384 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.890+0000 7f331b435b80 1 Processor -- start 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 -- start start 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881670350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x557881675300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788166ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788167cab0 con 0x557881675e00 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788167b100 con 0x557881670890 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f331b435b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557881691380 con 0x557881676720 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x557881675300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x557881675300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44784/0 (socket says 172.21.15.16:44784) 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 -- 172.21.15.16:0/4115457384 learned_addr learned my addr 172.21.15.16:0/4115457384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330d017700 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881670350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330d818700 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788166ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 -- 172.21.15.16:0/4115457384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 msgr2=0x55788166ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788166ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 -- 172.21.15.16:0/4115457384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 msgr2=0x557881670350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881670350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 -- 172.21.15.16:0/4115457384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557881586560 con 0x557881675e00 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330d818700 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788166ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.891+0000 7f330c816700 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x557881675300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f330409b560 tx=0x7f3304005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65741762bb87da48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f32f3fff700 1 -- 172.21.15.16:0/4115457384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33040a39c0 con 0x557881675e00 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f32f3fff700 1 -- 172.21.15.16:0/4115457384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f330400fd70 con 0x557881675e00 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f32f3fff700 1 -- 172.21.15.16:0/4115457384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330409abc0 con 0x557881675e00 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 -- 172.21.15.16:0/4115457384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 msgr2=0x557881675300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x557881675300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f330409b560 tx=0x7f3304005bc0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 -- 172.21.15.16:0/4115457384 shutdown_connections 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881676720 0x55788166ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x557881670350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 --2- 172.21.15.16:0/4115457384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881675e00 0x557881675300 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 -- 172.21.15.16:0/4115457384 >> 172.21.15.16:0/4115457384 conn(0x5578815ce210 msgr2=0x557881674bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 -- 172.21.15.16:0/4115457384 shutdown_connections 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.892+0000 7f331b435b80 1 -- 172.21.15.16:0/4115457384 wait complete. 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 -- start start 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 Processor -- start 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x5578816814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881675e00 0x55788167f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 0x55788167f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788167fda0 con 0x557881676720 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55788167ff10 con 0x557881670890 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578816df250 con 0x557881675e00 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f331b435b80 10 client.0 _ll_register_callbacks cb 0x557881565b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f330d818700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 0x55788167f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f330d818700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 0x55788167f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44790/0 (socket says 172.21.15.16:44790) 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f330d818700 1 -- 172.21.15.16:0/2900031972 learned_addr learned my addr 172.21.15.16:0/2900031972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f330d017700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x5578816814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.893+0000 7f330c816700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881675e00 0x55788167f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f330d818700 1 -- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881675e00 msgr2=0x55788167f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f330d818700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881675e00 0x55788167f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f330d818700 1 -- 172.21.15.16:0/2900031972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 msgr2=0x5578816814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f330d818700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x5578816814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f330d818700 1 -- 172.21.15.16:0/2900031972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578813edb60 con 0x557881676720 2024-05-08T23:21:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f330d818700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 0x55788167f690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3300015e60 tx=0x7f33000135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dab04e19713564 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f32f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2900031972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300013e10 con 0x557881676720 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2900031972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f3300003a00 con 0x557881676720 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32f8011d40 con 0x557881676720 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2900031972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3300003bc0 con 0x557881676720 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f331b435b80 10 client.16101 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.894+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557881445cd0 con 0x557881676720 2024-05-08T23:21:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2900031972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32f8011d40 con 0x557881676720 2024-05-08T23:21:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2900031972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f330001d020 con 0x557881676720 2024-05-08T23:21:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f331b435b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f331b435b80 10 client.16101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:21:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f331b435b80 20 client.16101 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f331b435b80 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5578816e0a10 0x5578816e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f331b435b80 4 client.16101 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578815b8ec0 con 0x5578816e0a10 2024-05-08T23:21:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.895+0000 7f330d017700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5578816e0a10 0x5578816e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f32f0ff9700 10 client.16101 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f330d017700 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5578816e0a10 0x5578816e2e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87f1ed6b186eb8eb server_cookie=6b821539a47845a4 in_seq=0 out_seq=0 2024-05-08T23:21:57.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f32f0ff9700 1 -- 172.21.15.16:0/2900031972 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578815b8ec0 con 0x5578816e0a10 2024-05-08T23:21:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f32f0ff9700 10 client.16101 handle_command_reply: tid=0 2024-05-08T23:21:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 client.16101 shutdown 2024-05-08T23:21:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 msgr2=0x55788167f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 0x55788167f690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3300015e60 tx=0x7f33000135b0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 shutdown_connections 2024-05-08T23:21:57.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557881676720 0x55788167f690 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5578816e0a10 0x5578816e2e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557881670890 0x5578816814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 --2- 172.21.15.16:0/2900031972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557881675e00 0x55788167f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f32f0ff9700 0 client.16101 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 >> 172.21.15.16:0/2900031972 conn(0x5578815ce210 msgr2=0x5578816783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 shutdown_connections 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 1 -- 172.21.15.16:0/2900031972 wait complete. 2024-05-08T23:21:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.896+0000 7f331b435b80 20 client.16101 trim_cache size 0 max 16384 2024-05-08T23:21:57.908 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:21:57.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32f40a1290 msgr2=0x7f32f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32f40a1290 0x7f32f40a3750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009b50 tx=0x7f32f8009550 comp rx=0 tx=0).stop 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 msgr2=0x7f33081a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 0x7f33081a3ad0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f33040036c0 tx=0x7f3304005e80 comp rx=0 tx=0).stop 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 shutdown_connections 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32f40a1290 0x7f32f40a3750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3308105e30 0x7f330819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3308068590 0x7f33081a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 --2- 172.21.15.16:0/4076438011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308103930 0x7f33081a3ad0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 >> 172.21.15.16:0/4076438011 conn(0x7f3308076f40 msgr2=0x7f33080fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 shutdown_connections 2024-05-08T23:21:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:57.897+0000 7f330f27b700 1 -- 172.21.15.16:0/4076438011 wait complete. 2024-05-08T23:21:57.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 -- 172.21.15.16:0/4286756006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18106280 msgr2=0x7fce1810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 --2- 172.21.15.16:0/4286756006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18106280 0x7fce1810de00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fce1000b0a0 tx=0x7fce1000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 -- 172.21.15.16:0/4286756006 shutdown_connections 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 --2- 172.21.15.16:0/4286756006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18106280 0x7fce1810de00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 --2- 172.21.15.16:0/4286756006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18101160 0x7fce18101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 --2- 172.21.15.16:0/4286756006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce18101b10 0x7fce18105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 -- 172.21.15.16:0/4286756006 >> 172.21.15.16:0/4286756006 conn(0x7fce180fc9e0 msgr2=0x7fce180fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 -- 172.21.15.16:0/4286756006 shutdown_connections 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 -- 172.21.15.16:0/4286756006 wait complete. 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 Processor -- start 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.050+0000 7fce1f565700 1 -- start start 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1f565700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 0x7fce18197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1f565700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18106280 0x7fce18197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1f565700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce1819bd10 0x7fce1819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1f565700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1819c790 con 0x7fce18101160 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1f565700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1819c900 con 0x7fce18106280 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1f565700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1819cac0 con 0x7fce1819bd10 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 0x7fce18197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 0x7fce18197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44812/0 (socket says 172.21.15.16:44812) 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 -- 172.21.15.16:0/3252377749 learned_addr learned my addr 172.21.15.16:0/3252377749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1cb00700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18106280 0x7fce18197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1db02700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce1819bd10 0x7fce1819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 -- 172.21.15.16:0/3252377749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce1819bd10 msgr2=0x7fce1819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce1819bd10 0x7fce1819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 -- 172.21.15.16:0/3252377749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18106280 msgr2=0x7fce18197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18106280 0x7fce18197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 -- 172.21.15.16:0/3252377749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10009d00 con 0x7fce18101160 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.051+0000 7fce1d301700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 0x7fce18197430 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce1400d950 tx=0x7fce1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.052+0000 7fce0e7fc700 1 -- 172.21.15.16:0/3252377749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14009940 con 0x7fce18101160 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.052+0000 7fce0e7fc700 1 -- 172.21.15.16:0/3252377749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce14010460 con 0x7fce18101160 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.052+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce181a3fc0 con 0x7fce18101160 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.052+0000 7fce0e7fc700 1 -- 172.21.15.16:0/3252377749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1400f620 con 0x7fce18101160 2024-05-08T23:21:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.052+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce181a44f0 con 0x7fce18101160 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.054+0000 7fce0e7fc700 1 -- 172.21.15.16:0/3252377749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fce140105d0 con 0x7fce18101160 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.055+0000 7fce0e7fc700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce040a1310 0x7fce040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.055+0000 7fce1cb00700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce040a1310 0x7fce040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.055+0000 7fce0e7fc700 1 -- 172.21.15.16:0/3252377749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fce140cce00 con 0x7fce18101160 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.056+0000 7fce1cb00700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce040a1310 0x7fce040a37d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce18198980 tx=0x7fce0800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.056+0000 7fce2b71fb80 1 Processor -- start 2024-05-08T23:21:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.056+0000 7fce2b71fb80 1 -- start start 2024-05-08T23:21:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9b210 0x5628fba9b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fba960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fb9e8150 con 0x5628fba9bb30 2024-05-08T23:21:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fb9e5b10 con 0x5628fba95ca0 2024-05-08T23:21:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fb9e1400 con 0x5628fba9b210 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba95760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44822/0 (socket says 172.21.15.16:44822) 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 -- 172.21.15.16:0/4051728789 learned_addr learned my addr 172.21.15.16:0/4051728789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1d301700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9b210 0x5628fba9b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1db02700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fba960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 -- 172.21.15.16:0/4051728789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9b210 msgr2=0x5628fba9b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9b210 0x5628fba9b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 -- 172.21.15.16:0/4051728789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 msgr2=0x5628fba960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fba960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 -- 172.21.15.16:0/4051728789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628fb8623b0 con 0x5628fba9bb30 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1d301700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9b210 0x5628fba9b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1db02700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fba960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce1cb00700 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba95760 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fce08010dc0 tx=0x7fce08013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15ed15b0a8d7c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce037fe700 1 -- 172.21.15.16:0/4051728789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0801e420 con 0x5628fba9bb30 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce037fe700 1 -- 172.21.15.16:0/4051728789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce08015070 con 0x5628fba9bb30 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.057+0000 7fce037fe700 1 -- 172.21.15.16:0/4051728789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0801c8b0 con 0x5628fba9bb30 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 -- 172.21.15.16:0/4051728789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 msgr2=0x5628fba95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba95760 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fce08010dc0 tx=0x7fce08013000 comp rx=0 tx=0).stop 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 -- 172.21.15.16:0/4051728789 shutdown_connections 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fba960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9b210 0x5628fba9b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/4051728789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba95760 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 -- 172.21.15.16:0/4051728789 >> 172.21.15.16:0/4051728789 conn(0x5628fb9ef210 msgr2=0x5628fba8bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 -- 172.21.15.16:0/4051728789 shutdown_connections 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.058+0000 7fce2b71fb80 1 -- 172.21.15.16:0/4051728789 wait complete. 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 -- start start 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 Processor -- start 2024-05-08T23:21:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fb97d560 con 0x5628fba9bb30 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fb97d200 con 0x5628fba9b210 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fb8646a0 con 0x5628fba95ca0 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.059+0000 7fce2b71fb80 10 client.0 _ll_register_callbacks cb 0x5628fb986b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44828/0 (socket says 172.21.15.16:44828) 2024-05-08T23:21:58.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 -- 172.21.15.16:0/1557262741 learned_addr learned my addr 172.21.15.16:0/1557262741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1d301700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1cb00700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 -- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 msgr2=0x5628fba9ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba9ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 -- 172.21.15.16:0/1557262741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 msgr2=0x5628fba9f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 -- 172.21.15.16:0/1557262741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce08010aa0 con 0x5628fba9bb30 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce1db02700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9fa10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fce10009300 tx=0x7fce10009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e5f2a0e7a1443a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce00ff9700 1 -- 172.21.15.16:0/1557262741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000e040 con 0x5628fba9bb30 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce00ff9700 1 -- 172.21.15.16:0/1557262741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fce100076f0 con 0x5628fba9bb30 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628fb93a270 con 0x5628fba9bb30 2024-05-08T23:21:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.060+0000 7fce00ff9700 1 -- 172.21.15.16:0/1557262741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce10009b80 con 0x5628fba9bb30 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 10 client.16113 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5628fb80eb60 con 0x5628fba9bb30 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce00ff9700 1 -- 172.21.15.16:0/1557262741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5628fb93a270 con 0x5628fba9bb30 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce00ff9700 1 -- 172.21.15.16:0/1557262741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce10019070 con 0x5628fba9bb30 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 10 client.16113 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 10 client.16113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 20 client.16113 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbaaf5a0 0x5628fbab19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 4 client.16113 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628fba60810 con 0x5628fbaaf5a0 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.061+0000 7fce1d301700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbaaf5a0 0x5628fbab19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.062+0000 7fce1d301700 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbaaf5a0 0x5628fbab19c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2643fb785955c33c server_cookie=bedf5c5bf8d69792 in_seq=0 out_seq=0 2024-05-08T23:21:58.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.062+0000 7fce00ff9700 10 client.16113 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.062+0000 7fce00ff9700 1 -- 172.21.15.16:0/1557262741 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628fba60810 con 0x5628fbaaf5a0 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.062+0000 7fce00ff9700 10 client.16113 handle_command_reply: tid=0 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.062+0000 7fce2b71fb80 1 client.16113 shutdown 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 msgr2=0x5628fba9fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9fa10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fce10009300 tx=0x7fce10009330 comp rx=0 tx=0).stop 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 shutdown_connections 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9fa10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba9ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbaaf5a0 0x5628fbab19c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1557262741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce00ff9700 0 client.16113 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 >> 172.21.15.16:0/1557262741 conn(0x5628fb9ef210 msgr2=0x5628fba8b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 shutdown_connections 2024-05-08T23:21:58.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1557262741 wait complete. 2024-05-08T23:21:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.063+0000 7fce2b71fb80 20 client.16113 trim_cache size 0 max 16384 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 Processor -- start 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 -- start start 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba8bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fbaa05b0 con 0x5628fba9bb30 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fba840d0 con 0x5628fba9b210 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.077+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fba88dc0 con 0x5628fba95ca0 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44834/0 (socket says 172.21.15.16:44834) 2024-05-08T23:21:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 -- 172.21.15.16:0/1239422489 learned_addr learned my addr 172.21.15.16:0/1239422489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1d301700 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba8bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1cb00700 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 -- 172.21.15.16:0/1239422489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 msgr2=0x5628fba8bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba8bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 -- 172.21.15.16:0/1239422489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 msgr2=0x5628fba9a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 -- 172.21.15.16:0/1239422489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628fb9a7560 con 0x5628fba9bb30 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1cb00700 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce1db02700 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9a550 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fce1000b3f0 tx=0x7fce10009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bce95ce20819d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce03fff700 1 -- 172.21.15.16:0/1239422489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000e030 con 0x5628fba9bb30 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce03fff700 1 -- 172.21.15.16:0/1239422489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce1001cb10 con 0x5628fba9bb30 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.078+0000 7fce03fff700 1 -- 172.21.15.16:0/1239422489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10003d10 con 0x5628fba9bb30 2024-05-08T23:21:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1239422489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 msgr2=0x5628fba9a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9a550 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fce1000b3f0 tx=0x7fce10009b80 comp rx=0 tx=0).stop 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1239422489 shutdown_connections 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9bb30 0x5628fba9a550 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba95ca0 0x5628fba8bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1239422489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba9b210 0x5628fba9a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1239422489 >> 172.21.15.16:0/1239422489 conn(0x5628fb9ef210 msgr2=0x5628fba96080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1239422489 shutdown_connections 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1239422489 wait complete. 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 -- start start 2024-05-08T23:21:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.079+0000 7fce2b71fb80 1 Processor -- start 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fbaa4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 0x5628fbaa4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9bb30 0x5628fbaa5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fbb04c50 con 0x5628fba9b210 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fbb04dc0 con 0x5628fba95ca0 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628fbb04f80 con 0x5628fba9bb30 2024-05-08T23:21:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce2b71fb80 10 client.0 _ll_register_callbacks cb 0x5628fb986b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 0x5628fbaa4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 0x5628fbaa4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44848/0 (socket says 172.21.15.16:44848) 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 -- 172.21.15.16:0/1294896342 learned_addr learned my addr 172.21.15.16:0/1294896342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1db02700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9bb30 0x5628fbaa5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1d301700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fbaa4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 -- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9bb30 msgr2=0x5628fbaa5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9bb30 0x5628fbaa5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 -- 172.21.15.16:0/1294896342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 msgr2=0x5628fbaa4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fbaa4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 -- 172.21.15.16:0/1294896342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628fb80eb60 con 0x5628fba9b210 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1d301700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fbaa4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce1cb00700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 0x5628fbaa4ac0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fce08005950 tx=0x7fce08005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686e031bfb65ed75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.080+0000 7fce00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce00ff9700 1 -- 172.21.15.16:0/1294896342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0802d050 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce00ff9700 1 -- 172.21.15.16:0/1294896342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fce0801ee40 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce08015070 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce00ff9700 1 -- 172.21.15.16:0/1294896342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce08005720 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 10 client.16119 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5628fb866cd0 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce00ff9700 1 -- 172.21.15.16:0/1294896342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce08015070 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce00ff9700 1 -- 172.21.15.16:0/1294896342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce0801e7e0 con 0x5628fba9b210 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 10 client.16119 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 10 client.16119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 20 client.16119 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbb066b0 0x5628fbb08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 4 client.16119 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.081+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628fb9d9ec0 con 0x5628fbb066b0 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.082+0000 7fce1d301700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbb066b0 0x5628fbb08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.082+0000 7fce1d301700 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbb066b0 0x5628fbb08ad0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c00325371cf73c58 server_cookie=b63f37b987ccd040 in_seq=0 out_seq=0 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.082+0000 7fce00ff9700 10 client.16119 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.082+0000 7fce00ff9700 1 -- 172.21.15.16:0/1294896342 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628fb9d9ec0 con 0x5628fbb066b0 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.082+0000 7fce00ff9700 10 client.16119 handle_command_reply: tid=0 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.082+0000 7fce2b71fb80 1 client.16119 shutdown 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 msgr2=0x5628fbaa4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 0x5628fbaa4ac0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fce08005950 tx=0x7fce08005270 comp rx=0 tx=0).stop 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 shutdown_connections 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5628fba9bb30 0x5628fbaa5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5628fbb066b0 0x5628fbb08ad0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5628fba95ca0 0x5628fbaa4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 --2- 172.21.15.16:0/1294896342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5628fba9b210 0x5628fbaa4ac0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce00ff9700 0 client.16119 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:21:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 >> 172.21.15.16:0/1294896342 conn(0x5628fb9ef210 msgr2=0x5628fba9a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 shutdown_connections 2024-05-08T23:21:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 1 -- 172.21.15.16:0/1294896342 wait complete. 2024-05-08T23:21:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce2b71fb80 20 client.16119 trim_cache size 0 max 16384 2024-05-08T23:21:58.090 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce040a1310 msgr2=0x7fce040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce040a1310 0x7fce040a37d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce18198980 tx=0x7fce0800b410 comp rx=0 tx=0).stop 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 msgr2=0x7fce18197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 0x7fce18197430 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fce1400d950 tx=0x7fce1400dc60 comp rx=0 tx=0).stop 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 shutdown_connections 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fce040a1310 0x7fce040a37d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce1819bd10 0x7fce1819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce18101160 0x7fce18197430 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 --2- 172.21.15.16:0/3252377749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce18106280 0x7fce18197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.083+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 >> 172.21.15.16:0/3252377749 conn(0x7fce180fc9e0 msgr2=0x7fce180fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.084+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 shutdown_connections 2024-05-08T23:21:58.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.084+0000 7fce1f565700 1 -- 172.21.15.16:0/3252377749 wait complete. 2024-05-08T23:21:58.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:57 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3166223703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:58.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:21:58.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:57 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3166223703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:58.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:57 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3166223703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:21:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 -- 172.21.15.16:0/3571453607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100a10 msgr2=0x7f2cfc104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 --2- 172.21.15.16:0/3571453607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100a10 0x7f2cfc104a30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009990 tx=0x7f2ce4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 -- 172.21.15.16:0/3571453607 shutdown_connections 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 --2- 172.21.15.16:0/3571453607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cfc105160 0x7f2cfc10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 --2- 172.21.15.16:0/3571453607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cfc100060 0x7f2cfc100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 --2- 172.21.15.16:0/3571453607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100a10 0x7f2cfc104a30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 -- 172.21.15.16:0/3571453607 >> 172.21.15.16:0/3571453607 conn(0x7f2cfc076f40 msgr2=0x7f2cfc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 -- 172.21.15.16:0/3571453607 shutdown_connections 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.237+0000 7f2d0162b700 1 -- 172.21.15.16:0/3571453607 wait complete. 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 Processor -- start 2024-05-08T23:21:58.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 -- start start 2024-05-08T23:21:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 0x7f2cfc197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cfc105160 0x7f2cfc197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cfc19bc30 0x7f2cfc19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc19c6b0 con 0x7f2cfc100060 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc19c820 con 0x7f2cfc105160 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.238+0000 7f2d0162b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc19c9e0 con 0x7f2cfc19bc30 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 0x7f2cfc197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 0x7f2cfc197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44872/0 (socket says 172.21.15.16:44872) 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 -- 172.21.15.16:0/2467062291 learned_addr learned my addr 172.21.15.16:0/2467062291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cfc19bc30 0x7f2cfc19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cfc105160 0x7f2cfc197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 -- 172.21.15.16:0/2467062291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cfc19bc30 msgr2=0x7f2cfc19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cfc19bc30 0x7f2cfc19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 -- 172.21.15.16:0/2467062291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cfc105160 msgr2=0x7f2cfc197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cfc105160 0x7f2cfc197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 -- 172.21.15.16:0/2467062291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce40095f0 con 0x7f2cfc100060 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 0x7f2cfc197350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00e9a0 tx=0x7f2cec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cf8ff9700 1 -- 172.21.15.16:0/2467062291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec009790 con 0x7f2cfc100060 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cf8ff9700 1 -- 172.21.15.16:0/2467062291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cec00cbd0 con 0x7f2cfc100060 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc1a3f40 con 0x7f2cfc100060 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2cf8ff9700 1 -- 172.21.15.16:0/2467062291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec018680 con 0x7f2cfc100060 2024-05-08T23:21:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.239+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc1a4410 con 0x7f2cfc100060 2024-05-08T23:21:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.242+0000 7f2cf8ff9700 1 -- 172.21.15.16:0/2467062291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2cec00cd40 con 0x7f2cfc100060 2024-05-08T23:21:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.242+0000 7f2cf8ff9700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2ce80a1310 0x7f2ce80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.242+0000 7f2cf8ff9700 1 -- 172.21.15.16:0/2467062291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2cec014070 con 0x7f2cfc100060 2024-05-08T23:21:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.242+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2ce80a1310 0x7f2ce80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.243+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2ce80a1310 0x7f2ce80a37d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc1988a0 tx=0x7f2ce4005940 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.243+0000 7f2d0d7e5b80 1 Processor -- start 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.243+0000 7f2d0d7e5b80 1 -- start start 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275b2210 0x55ee275b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 0x55ee275ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee274ff150 con 0x55ee275acca0 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee274fcb10 con 0x55ee275b2b30 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee274f8400 con 0x55ee275b2210 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 0x55ee275ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 0x55ee275ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44882/0 (socket says 172.21.15.16:44882) 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/4288370323 learned_addr learned my addr 172.21.15.16:0/4288370323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275b2210 0x55ee275b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/4288370323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275b2210 msgr2=0x55ee275b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275b2210 0x55ee275b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/4288370323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 msgr2=0x55ee275ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/4288370323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee273793b0 con 0x55ee275acca0 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.244+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 0x55ee275ad0e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4011680 tx=0x7f2cf4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c3ce90db4e43f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2cf17fa700 1 -- 172.21.15.16:0/4288370323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e040 con 0x55ee275acca0 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2cf17fa700 1 -- 172.21.15.16:0/4288370323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cf400f040 con 0x55ee275acca0 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2cf17fa700 1 -- 172.21.15.16:0/4288370323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4004730 con 0x55ee275acca0 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/4288370323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 msgr2=0x55ee275ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 0x55ee275ad0e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4011680 tx=0x7f2cf4009920 comp rx=0 tx=0).stop 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/4288370323 shutdown_connections 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275acca0 0x55ee275ad0e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.245+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275b2210 0x55ee275b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/4288370323 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/4288370323 >> 172.21.15.16:0/4288370323 conn(0x55ee27506210 msgr2=0x55ee275a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/4288370323 shutdown_connections 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/4288370323 wait complete. 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- start start 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 Processor -- start 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2210 0x55ee275b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 0x55ee275b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27494560 con 0x55ee275b2b30 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27494200 con 0x55ee275b2210 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2737b6a0 con 0x55ee275acca0 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2d0d7e5b80 10 client.0 _ll_register_callbacks cb 0x55ee2749db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.246+0000 7f2cfb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 0x55ee275b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 0x55ee275b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44884/0 (socket says 172.21.15.16:44884) 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/1244708205 learned_addr learned my addr 172.21.15.16:0/1244708205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2210 0x55ee275b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/1244708205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 msgr2=0x55ee275b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2210 msgr2=0x55ee275b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2210 0x55ee275b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 -- 172.21.15.16:0/1244708205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf4011330 con 0x55ee275b2b30 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 0x55ee275b6960 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4012010 tx=0x7f2cf40045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc4b0296d02f64a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2ce27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1244708205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e030 con 0x55ee275b2b30 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1244708205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2cf40093a0 con 0x55ee275b2b30 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.247+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee27451270 con 0x55ee275b2b30 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1244708205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf4003ec0 con 0x55ee275b2b30 2024-05-08T23:21:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 10 client.16131 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee27325b60 con 0x55ee275b2b30 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1244708205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee27451270 con 0x55ee275b2b30 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1244708205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cf4024020 con 0x55ee275b2b30 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 10 client.16131 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 20 client.16131 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee275c65a0 0x55ee275c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 4 client.16131 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee27577810 con 0x55ee275c65a0 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.248+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee275c65a0 0x55ee275c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.249+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee275c65a0 0x55ee275c89c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c833737032fcbdb server_cookie=2a360108d624548d in_seq=0 out_seq=0 2024-05-08T23:21:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.249+0000 7f2ce27fc700 10 client.16131 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.249+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1244708205 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee27577810 con 0x55ee275c65a0 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.249+0000 7f2ce27fc700 10 client.16131 handle_command_reply: tid=0 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.249+0000 7f2d0d7e5b80 1 client.16131 shutdown 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 msgr2=0x55ee275b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 0x55ee275b6960 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4012010 tx=0x7f2cf40045c0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 shutdown_connections 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2b30 0x55ee275b6960 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee275c65a0 0x55ee275c89c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1244708205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2210 0x55ee275b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2ce27fc700 0 client.16131 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 >> 172.21.15.16:0/1244708205 conn(0x55ee27506210 msgr2=0x55ee275a25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 shutdown_connections 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1244708205 wait complete. 2024-05-08T23:21:58.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.250+0000 7f2d0d7e5b80 20 client.16131 trim_cache size 0 max 16384 2024-05-08T23:21:58.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.264+0000 7f2d0d7e5b80 1 Processor -- start 2024-05-08T23:21:58.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.264+0000 7f2d0d7e5b80 1 -- start start 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee275b3f20 con 0x55ee275b2210 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee275b74e0 con 0x55ee275b2b30 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee275a38e0 con 0x55ee275acca0 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275bd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44896/0 (socket says 172.21.15.16:44896) 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1549996249 learned_addr learned my addr 172.21.15.16:0/1549996249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1549996249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 msgr2=0x55ee275a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1549996249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 msgr2=0x55ee275bdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275bdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1549996249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee274be560 con 0x55ee275b2210 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275bd5c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009990 tx=0x7f2ce401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbbf94a5ee98af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf1ffb700 1 -- 172.21.15.16:0/1549996249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400bca0 con 0x55ee275b2210 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.265+0000 7f2cf1ffb700 1 -- 172.21.15.16:0/1549996249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ce402c040 con 0x55ee275b2210 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2cf1ffb700 1 -- 172.21.15.16:0/1549996249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400f6c0 con 0x55ee275b2210 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1549996249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 msgr2=0x55ee275bd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275bd5c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009990 tx=0x7f2ce401f000 comp rx=0 tx=0).stop 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1549996249 shutdown_connections 2024-05-08T23:21:58.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee275bdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1549996249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275bd5c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1549996249 >> 172.21.15.16:0/1549996249 conn(0x55ee27506210 msgr2=0x55ee275cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1549996249 shutdown_connections 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.266+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1549996249 wait complete. 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 -- start start 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 Processor -- start 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee27614a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27615160 con 0x55ee275b2210 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee276152d0 con 0x55ee275b2b30 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee27615490 con 0x55ee275acca0 2024-05-08T23:21:58.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2d0d7e5b80 10 client.0 _ll_register_callbacks cb 0x55ee2749db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2cf27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2cf27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275ba380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44912/0 (socket says 172.21.15.16:44912) 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1273685484 learned_addr learned my addr 172.21.15.16:0/1273685484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee27614a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.267+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1273685484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 msgr2=0x55ee275b9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 msgr2=0x55ee27614a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:58.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee27614a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cf27fc700 1 -- 172.21.15.16:0/1273685484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc1a4410 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2cf27fc700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275ba380 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40093c0 tx=0x7f2ce4004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cbd8d2e8e1b07c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2ce27fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1273685484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004810 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1273685484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2ce4004970 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cf400f040 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1273685484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce400f460 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2d0d7e5b80 10 client.16137 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f2cfc067ff0 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.268+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1273685484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2cf400f040 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1273685484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ce4042070 con 0x55ee275b2210 2024-05-08T23:21:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2d0d7e5b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2d0d7e5b80 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2d0d7e5b80 20 client.16137 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee2761d750 0x55ee2761fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2d0d7e5b80 4 client.16137 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee274f0ec0 con 0x55ee2761d750 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee2761d750 0x55ee2761fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2ce27fc700 10 client.16137 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:58.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.269+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee2761d750 0x55ee2761fb70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f897ea0898afd116 server_cookie=d978e709bb305674 in_seq=0 out_seq=0 2024-05-08T23:21:58.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2ce27fc700 1 -- 172.21.15.16:0/1273685484 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee274f0ec0 con 0x55ee2761d750 2024-05-08T23:21:58.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2ce27fc700 10 client.16137 handle_command_reply: tid=0 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 client.16137 shutdown 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 msgr2=0x55ee275ba380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275ba380 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40093c0 tx=0x7f2ce4004f50 comp rx=0 tx=0).stop 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 shutdown_connections 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ee2761d750 0x55ee2761fb70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee275b2b30 0x55ee27614a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee275acca0 0x55ee275b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 --2- 172.21.15.16:0/1273685484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee275b2210 0x55ee275ba380 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2ce27fc700 0 client.16137 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 >> 172.21.15.16:0/1273685484 conn(0x55ee27506210 msgr2=0x55ee275b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 shutdown_connections 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 1 -- 172.21.15.16:0/1273685484 wait complete. 2024-05-08T23:21:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0d7e5b80 20 client.16137 trim_cache size 0 max 16384 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2ce80a1310 msgr2=0x7f2ce80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.270+0000 7f2d0162b700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2ce80a1310 0x7f2ce80a37d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc1988a0 tx=0x7f2ce4005940 comp rx=0 tx=0).stop 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 msgr2=0x7f2cfc197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 0x7f2cfc197350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00e9a0 tx=0x7f2cec00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 shutdown_connections 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2ce80a1310 0x7f2ce80a37d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cfc19bc30 0x7f2cfc19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc100060 0x7f2cfc197350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 --2- 172.21.15.16:0/2467062291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cfc105160 0x7f2cfc197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 >> 172.21.15.16:0/2467062291 conn(0x7f2cfc076f40 msgr2=0x7f2cfc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 shutdown_connections 2024-05-08T23:21:58.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:58.271+0000 7f2d0162b700 1 -- 172.21.15.16:0/2467062291 wait complete. 2024-05-08T23:21:59.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:21:58 smithi016 ceph-mon[42541]: pgmap v75: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:59.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:21:58 smithi189 ceph-mon[46919]: pgmap v75: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:59.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:21:58 smithi155 ceph-mon[47105]: pgmap v75: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-08T23:21:59.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:21:59.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:21:59.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 -- 172.21.15.16:0/4024554874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24101b10 msgr2=0x7f1f24105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/4024554874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24101b10 0x7f1f24105b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009990 tx=0x7f1f10009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 -- 172.21.15.16:0/4024554874 shutdown_connections 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/4024554874 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f24106280 0x7f1f2410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/4024554874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 0x7f1f24101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/4024554874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24101b10 0x7f1f24105b50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 -- 172.21.15.16:0/4024554874 >> 172.21.15.16:0/4024554874 conn(0x7f1f240fc9e0 msgr2=0x7f1f240fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 -- 172.21.15.16:0/4024554874 shutdown_connections 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.435+0000 7f1f2ade4700 1 -- 172.21.15.16:0/4024554874 wait complete. 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 Processor -- start 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 -- start start 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 0x7f1f241973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 0x7f1f241978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f2419bc30 0x7f1f2419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2419c6b0 con 0x7f1f24106280 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2419c820 con 0x7f1f2419bc30 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f2ade4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2419c9e0 con 0x7f1f24101160 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 0x7f1f241978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 0x7f1f241978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44928/0 (socket says 172.21.15.16:44928) 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f23fff700 1 -- 172.21.15.16:0/2562157356 learned_addr learned my addr 172.21.15.16:0/2562157356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f29381700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f2419bc30 0x7f1f2419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.436+0000 7f1f28b80700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 0x7f1f241973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f23fff700 1 -- 172.21.15.16:0/2562157356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 msgr2=0x7f1f241973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f23fff700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 0x7f1f241973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f23fff700 1 -- 172.21.15.16:0/2562157356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f2419bc30 msgr2=0x7f1f2419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f23fff700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f2419bc30 0x7f1f2419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f23fff700 1 -- 172.21.15.16:0/2562157356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100095f0 con 0x7f1f24106280 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f28b80700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 0x7f1f241973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f23fff700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 0x7f1f241978e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f10006010 tx=0x7f1f10005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f21ffb700 1 -- 172.21.15.16:0/2562157356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1001d070 con 0x7f1f24106280 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f21ffb700 1 -- 172.21.15.16:0/2562157356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f1000fbc0 con 0x7f1f24106280 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f241a3ee0 con 0x7f1f24106280 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f21ffb700 1 -- 172.21.15.16:0/2562157356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10022b50 con 0x7f1f24106280 2024-05-08T23:21:59.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.437+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f241a42a0 con 0x7f1f24106280 2024-05-08T23:21:59.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.440+0000 7f1f21ffb700 1 -- 172.21.15.16:0/2562157356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1f1000fd30 con 0x7f1f24106280 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.440+0000 7f1f21ffb700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f140a1290 0x7f1f140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.440+0000 7f1f21ffb700 1 -- 172.21.15.16:0/2562157356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1f100ce730 con 0x7f1f24106280 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.440+0000 7f1f28b80700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f140a1290 0x7f1f140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.441+0000 7f1f28b80700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f140a1290 0x7f1f140a3750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bc10 tx=0x7f1f1800b5f0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.441+0000 7f1f36f9eb80 1 Processor -- start 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.441+0000 7f1f36f9eb80 1 -- start start 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f7f71d0 con 0x55f84f8a5e80 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f7f4b90 con 0x55f84f8a67a0 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f7f0480 con 0x55f84f8a0910 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8a6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44944/0 (socket says 172.21.15.16:44944) 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 -- 172.21.15.16:0/3943475745 learned_addr learned my addr 172.21.15.16:0/3943475745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f29381700 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 -- 172.21.15.16:0/3943475745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 msgr2=0x55f84f8a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f23fff700 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 -- 172.21.15.16:0/3943475745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 msgr2=0x55f84f8a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 -- 172.21.15.16:0/3943475745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84f671430 con 0x55f84f8a5e80 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8a6260 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800fd10 tx=0x7f1f18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ece3a02cf6b22e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f0effd700 1 -- 172.21.15.16:0/3943475745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18011ad0 con 0x55f84f8a5e80 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f0effd700 1 -- 172.21.15.16:0/3943475745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f18011c30 con 0x55f84f8a5e80 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.442+0000 7f1f0effd700 1 -- 172.21.15.16:0/3943475745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1801d5b0 con 0x55f84f8a5e80 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3943475745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 msgr2=0x55f84f8a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8a6260 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800fd10 tx=0x7f1f18012000 comp rx=0 tx=0).stop 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3943475745 shutdown_connections 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8a6260 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3943475745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3943475745 >> 172.21.15.16:0/3943475745 conn(0x55f84f7fe290 msgr2=0x55f84f89ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3943475745 shutdown_connections 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.443+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3943475745 wait complete. 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 -- start start 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 Processor -- start 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 0x55f84f8a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a5e80 0x55f84f8a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f78c5e0 con 0x55f84f8a0910 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f78c280 con 0x55f84f8a67a0 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f673720 con 0x55f84f8a5e80 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f36f9eb80 10 client.0 _ll_register_callbacks cb 0x55f84f795bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f28b80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 0x55f84f8a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f28b80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 0x55f84f8a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44960/0 (socket says 172.21.15.16:44960) 2024-05-08T23:21:59.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.444+0000 7f1f28b80700 1 -- 172.21.15.16:0/1769915470 learned_addr learned my addr 172.21.15.16:0/1769915470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f23fff700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a5e80 0x55f84f8a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f29381700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f28b80700 1 -- 172.21.15.16:0/1769915470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a5e80 msgr2=0x55f84f8a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f28b80700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a5e80 0x55f84f8a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f28b80700 1 -- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 msgr2=0x55f84f8aa530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f28b80700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8aa530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f28b80700 1 -- 172.21.15.16:0/1769915470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1800f9f0 con 0x55f84f8a0910 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f23fff700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a5e80 0x55f84f8a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:59.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f28b80700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 0x55f84f8a9ab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f18011b00 tx=0x7f1f180057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b736f0e9932be91e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f03fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f03fff700 1 -- 172.21.15.16:0/1769915470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1802d070 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f03fff700 1 -- 172.21.15.16:0/1769915470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1f1801e450 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f84f7492f0 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f03fff700 1 -- 172.21.15.16:0/1769915470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f1801e610 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f36f9eb80 10 client.16149 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.445+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f84f61dbe0 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f03fff700 1 -- 172.21.15.16:0/1769915470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f84f7492f0 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f03fff700 1 -- 172.21.15.16:0/1769915470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f18027c50 con 0x55f84f8a0910 2024-05-08T23:21:59.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f36f9eb80 10 client.16149 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f36f9eb80 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f36f9eb80 20 client.16149 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f8ba0e0 0x55f84f8bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f36f9eb80 4 client.16149 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f84f86f890 con 0x55f84f8ba0e0 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.446+0000 7f1f23fff700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f8ba0e0 0x55f84f8bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f23fff700 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f8ba0e0 0x55f84f8bc4d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ceb3e00fcbc37ab server_cookie=42c5ad8674f14778 in_seq=0 out_seq=0 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f03fff700 10 client.16149 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f03fff700 1 -- 172.21.15.16:0/1769915470 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f84f86f890 con 0x55f84f8ba0e0 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f03fff700 10 client.16149 handle_command_reply: tid=0 2024-05-08T23:21:59.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 client.16149 shutdown 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 msgr2=0x55f84f8a9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 0x55f84f8a9ab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f18011b00 tx=0x7f1f180057b0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 shutdown_connections 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f8ba0e0 0x55f84f8bc4d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a67a0 0x55f84f8aa530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a0910 0x55f84f8a9ab0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f03fff700 0 client.16149 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/1769915470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a5e80 0x55f84f8a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.447+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 >> 172.21.15.16:0/1769915470 conn(0x55f84f7fe290 msgr2=0x55f84f89a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.448+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 shutdown_connections 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.448+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/1769915470 wait complete. 2024-05-08T23:21:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.448+0000 7f1f36f9eb80 20 client.16149 trim_cache size 0 max 16384 2024-05-08T23:21:59.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 Processor -- start 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 -- start start 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a5e80 0x55f84f89ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 0x55f84f8a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f8c16e0 con 0x55f84f8a67a0 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f895880 con 0x55f84f8a5e80 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.466+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f8a6260 con 0x55f84f8a0910 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 0x55f84f8a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 0x55f84f8a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44962/0 (socket says 172.21.15.16:44962) 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 -- 172.21.15.16:0/3603663133 learned_addr learned my addr 172.21.15.16:0/3603663133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f23fff700 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a5e80 0x55f84f89ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 -- 172.21.15.16:0/3603663133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 msgr2=0x55f84f8a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 -- 172.21.15.16:0/3603663133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a5e80 msgr2=0x55f84f89ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a5e80 0x55f84f89ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 -- 172.21.15.16:0/3603663133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84f897560 con 0x55f84f8a67a0 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f23fff700 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a5e80 0x55f84f89ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f29381700 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 0x55f84f8a52b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009e00 tx=0x7f1f1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b07f114c0e9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f0f7fe700 1 -- 172.21.15.16:0/3603663133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c007d80 con 0x55f84f8a67a0 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f0f7fe700 1 -- 172.21.15.16:0/3603663133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f1c00f040 con 0x55f84f8a67a0 2024-05-08T23:21:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.467+0000 7f1f0f7fe700 1 -- 172.21.15.16:0/3603663133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017700 con 0x55f84f8a67a0 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3603663133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 msgr2=0x55f84f8a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 0x55f84f8a52b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009e00 tx=0x7f1f1c009e30 comp rx=0 tx=0).stop 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3603663133 shutdown_connections 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a67a0 0x55f84f8a52b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a0910 0x55f84f8a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3603663133 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a5e80 0x55f84f89ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3603663133 >> 172.21.15.16:0/3603663133 conn(0x55f84f7fe290 msgr2=0x55f84f8a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3603663133 shutdown_connections 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3603663133 wait complete. 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 -- start start 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.468+0000 7f1f36f9eb80 1 Processor -- start 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a0910 0x55f84f8b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a67a0 0x55f84f8af210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f8af920 con 0x55f84f8a5e80 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f8afa90 con 0x55f84f8a0910 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84f8afc50 con 0x55f84f8a67a0 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f36f9eb80 10 client.0 _ll_register_callbacks cb 0x55f84f795bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44968/0 (socket says 172.21.15.16:44968) 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 -- 172.21.15.16:0/3587059294 learned_addr learned my addr 172.21.15.16:0/3587059294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a0910 0x55f84f8b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f29381700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a67a0 0x55f84f8af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 -- 172.21.15.16:0/3587059294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a67a0 msgr2=0x55f84f8af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a67a0 0x55f84f8af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 -- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a0910 msgr2=0x55f84f8b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a0910 0x55f84f8b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.469+0000 7f1f23fff700 1 -- 172.21.15.16:0/3587059294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c009ab0 con 0x55f84f8a5e80 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f23fff700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8b1730 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1f10098120 tx=0x7f1f10022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0693d0f78e1f98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f03fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f03fff700 1 -- 172.21.15.16:0/3587059294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1009f4e0 con 0x55f84f8a5e80 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f03fff700 1 -- 172.21.15.16:0/3587059294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1f100a3aa0 con 0x55f84f8a5e80 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f18011c30 con 0x55f84f8a5e80 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f03fff700 1 -- 172.21.15.16:0/3587059294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f10067120 con 0x55f84f8a5e80 2024-05-08T23:21:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 10 client.16155 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f84f8a0fc0 con 0x55f84f8a5e80 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f03fff700 1 -- 172.21.15.16:0/3587059294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f18011c30 con 0x55f84f8a5e80 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f03fff700 1 -- 172.21.15.16:0/3587059294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f1009b910 con 0x55f84f8a5e80 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 10 client.16155 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 20 client.16155 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f9168e0 0x55f84f918d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 4 client.16155 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.470+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f84f7e8f40 con 0x55f84f9168e0 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.471+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f9168e0 0x55f84f918d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.471+0000 7f1f28b80700 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f9168e0 0x55f84f918d00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82287c40576a04 server_cookie=192f61a386a93f3f in_seq=0 out_seq=0 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.471+0000 7f1f03fff700 10 client.16155 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f03fff700 1 -- 172.21.15.16:0/3587059294 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f84f7e8f40 con 0x55f84f9168e0 2024-05-08T23:21:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f03fff700 10 client.16155 handle_command_reply: tid=0 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 client.16155 shutdown 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 msgr2=0x55f84f8b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8b1730 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1f10098120 tx=0x7f1f10022470 comp rx=0 tx=0).stop 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 shutdown_connections 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84f8a67a0 0x55f84f8af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55f84f9168e0 0x55f84f918d00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f84f8a0910 0x55f84f8b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 --2- 172.21.15.16:0/3587059294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84f8a5e80 0x55f84f8b1730 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 >> 172.21.15.16:0/3587059294 conn(0x55f84f7fe290 msgr2=0x55f84f8a7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f03fff700 0 client.16155 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 shutdown_connections 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.472+0000 7f1f36f9eb80 1 -- 172.21.15.16:0/3587059294 wait complete. 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f36f9eb80 20 client.16155 trim_cache size 0 max 16384 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f3ecc856-71dd-4371-b146-f286e5e20b5a", 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:21:59.487 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:21:59.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f140a1290 msgr2=0x7f1f140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f140a1290 0x7f1f140a3750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bc10 tx=0x7f1f1800b5f0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 msgr2=0x7f1f241978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 0x7f1f241978e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f10006010 tx=0x7f1f10005740 comp rx=0 tx=0).stop 2024-05-08T23:21:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 shutdown_connections 2024-05-08T23:21:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f140a1290 0x7f1f140a3750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f2419bc30 0x7f1f2419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f24101160 0x7f1f241973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 --2- 172.21.15.16:0/2562157356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f24106280 0x7f1f241978e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 >> 172.21.15.16:0/2562157356 conn(0x7f1f240fc9e0 msgr2=0x7f1f240fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 shutdown_connections 2024-05-08T23:21:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.473+0000 7f1f2ade4700 1 -- 172.21.15.16:0/2562157356 wait complete. 2024-05-08T23:21:59.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:21:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 -- 172.21.15.16:0/1354338937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8100a10 msgr2=0x7fe6f8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 --2- 172.21.15.16:0/1354338937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8100a10 0x7fe6f8104a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009990 tx=0x7fe6ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 -- 172.21.15.16:0/1354338937 shutdown_connections 2024-05-08T23:21:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 --2- 172.21.15.16:0/1354338937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8105160 0x7fe6f810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 --2- 172.21.15.16:0/1354338937 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8100060 0x7fe6f8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 --2- 172.21.15.16:0/1354338937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f8100a10 0x7fe6f8104a30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 -- 172.21.15.16:0/1354338937 >> 172.21.15.16:0/1354338937 conn(0x7fe6f8076f40 msgr2=0x7fe6f8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 -- 172.21.15.16:0/1354338937 shutdown_connections 2024-05-08T23:21:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 -- 172.21.15.16:0/1354338937 wait complete. 2024-05-08T23:21:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.632+0000 7fe6fda85700 1 Processor -- start 2024-05-08T23:21:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 -- start start 2024-05-08T23:21:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100060 0x7fe6f81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8105160 0x7fe6f8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 0x7fe6f819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f819c5f0 con 0x7fe6f819bb70 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f819c760 con 0x7fe6f8105160 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6fda85700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f819c920 con 0x7fe6f8100060 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 0x7fe6f819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 0x7fe6f819bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44978/0 (socket says 172.21.15.16:44978) 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 -- 172.21.15.16:0/2785523786 learned_addr learned my addr 172.21.15.16:0/2785523786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8105160 0x7fe6f8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100060 0x7fe6f81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 -- 172.21.15.16:0/2785523786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100060 msgr2=0x7fe6f81972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100060 0x7fe6f81972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 -- 172.21.15.16:0/2785523786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8105160 msgr2=0x7fe6f8197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8105160 0x7fe6f8197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f7fff700 1 -- 172.21.15.16:0/2785523786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec0095f0 con 0x7fe6f819bb70 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.633+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100060 0x7fe6f81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:59.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.634+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 0x7fe6f819bff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0007d70 tx=0x7fe6f000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.634+0000 7fe6f4ff9700 1 -- 172.21.15.16:0/2785523786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000ad40 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.634+0000 7fe6f4ff9700 1 -- 172.21.15.16:0/2785523786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6f00128b0 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.634+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f81a3e80 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.634+0000 7fe6f4ff9700 1 -- 172.21.15.16:0/2785523786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00115a0 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.634+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f81a4310 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.636+0000 7fe6f4ff9700 1 -- 172.21.15.16:0/2785523786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe6f0012a20 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.637+0000 7fe6f4ff9700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe6e00a1290 0x7fe6e00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.637+0000 7fe6f4ff9700 1 -- 172.21.15.16:0/2785523786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe6f00cd1c0 con 0x7fe6f819bb70 2024-05-08T23:21:59.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.637+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe6e00a1290 0x7fe6e00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe6e00a1290 0x7fe6e00a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800bc10 tx=0x7fe6e800b5f0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 Processor -- start 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 -- start start 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b0751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b06f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2afc6150 con 0x564c2b074e00 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2afc3b10 con 0x564c2b06f890 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2afbf400 con 0x564c2b075720 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.638+0000 7fe6f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b0751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f6ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b06f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b0751e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44986/0 (socket says 172.21.15.16:44986) 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 -- 172.21.15.16:0/3070299882 learned_addr learned my addr 172.21.15.16:0/3070299882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 -- 172.21.15.16:0/3070299882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 msgr2=0x564c2b06f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b06f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 -- 172.21.15.16:0/3070299882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 msgr2=0x564c2b06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 -- 172.21.15.16:0/3070299882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2ae403b0 con 0x564c2b074e00 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b06f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b0751e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800fd10 tx=0x7fe6e8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60764050651c048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6ddffb700 1 -- 172.21.15.16:0/3070299882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8011ad0 con 0x564c2b074e00 2024-05-08T23:21:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6ddffb700 1 -- 172.21.15.16:0/3070299882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6e8011c30 con 0x564c2b074e00 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.639+0000 7fe6ddffb700 1 -- 172.21.15.16:0/3070299882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e801d5d0 con 0x564c2b074e00 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3070299882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 msgr2=0x564c2b0751e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b0751e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800fd10 tx=0x7fe6e8012000 comp rx=0 tx=0).stop 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3070299882 shutdown_connections 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b0751e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3070299882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b06f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3070299882 >> 172.21.15.16:0/3070299882 conn(0x564c2afcd210 msgr2=0x564c2b069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3070299882 shutdown_connections 2024-05-08T23:21:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3070299882 wait complete. 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.640+0000 7fe709c3fb80 1 -- start start 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 Processor -- start 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b06f890 0x564c2b078ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b079010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b079550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2af5b560 con 0x564c2b074e00 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2af5b200 con 0x564c2b075720 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ae426a0 con 0x564c2b06f890 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe709c3fb80 10 client.0 _ll_register_callbacks cb 0x564c2af64b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b079010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b079010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44996/0 (socket says 172.21.15.16:44996) 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/3283802989 learned_addr learned my addr 172.21.15.16:0/3283802989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b079550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b06f890 0x564c2b078ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/3283802989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b06f890 msgr2=0x564c2b078ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.641+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b06f890 0x564c2b078ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 msgr2=0x564c2b079550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b079550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/3283802989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f810d550 con 0x564c2b074e00 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b06f890 0x564c2b078ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b079010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec015000 tx=0x7fe6ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68994ce12ea90f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/3283802989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec01d030 con 0x564c2b074e00 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/3283802989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe6ec00bd40 con 0x564c2b074e00 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2af18270 con 0x564c2b074e00 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/3283802989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6ec004500 con 0x564c2b074e00 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe709c3fb80 10 client.16167 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fe6e800f9f0 con 0x564c2b074e00 2024-05-08T23:21:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/3283802989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564c2af18270 con 0x564c2b074e00 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/3283802989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6ec00f660 con 0x564c2b074e00 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe709c3fb80 10 client.16167 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.642+0000 7fe709c3fb80 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe709c3fb80 20 client.16167 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b089250 0x564c2b08b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe709c3fb80 4 client.16167 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2b03e810 con 0x564c2b089250 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b089250 0x564c2b08b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe6d6ffd700 10 client.16167 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.643+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b089250 0x564c2b08b670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f42c5f7f4b64fb server_cookie=4e07a0fc75971520 in_seq=0 out_seq=0 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/3283802989 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c2b03e810 con 0x564c2b089250 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe6d6ffd700 10 client.16167 handle_command_reply: tid=0 2024-05-08T23:21:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 client.16167 shutdown 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 msgr2=0x564c2b079010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b079010 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec015000 tx=0x7fe6ec005800 comp rx=0 tx=0).stop 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 shutdown_connections 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b089250 0x564c2b08b670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b079550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b06f890 0x564c2b078ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/3283802989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b079010 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe6d6ffd700 0 client.16167 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 >> 172.21.15.16:0/3283802989 conn(0x564c2afcd210 msgr2=0x564c2b0695a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 shutdown_connections 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 1 -- 172.21.15.16:0/3283802989 wait complete. 2024-05-08T23:21:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.644+0000 7fe709c3fb80 20 client.16167 trim_cache size 0 max 16384 2024-05-08T23:21:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 Processor -- start 2024-05-08T23:21:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 -- start start 2024-05-08T23:21:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 0x564c2b06fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b074e00 0x564c2b084c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b0908a0 con 0x564c2b06f890 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b06ac70 con 0x564c2b075720 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b06b640 con 0x564c2b074e00 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe6f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 0x564c2b06fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe6f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 0x564c2b06fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45002/0 (socket says 172.21.15.16:45002) 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe6f77fe700 1 -- 172.21.15.16:0/417209527 learned_addr learned my addr 172.21.15.16:0/417209527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.663+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b074e00 0x564c2b084c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f77fe700 1 -- 172.21.15.16:0/417209527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b074e00 msgr2=0x564c2b084c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b074e00 0x564c2b084c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f77fe700 1 -- 172.21.15.16:0/417209527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 msgr2=0x564c2b069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f77fe700 1 -- 172.21.15.16:0/417209527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2adecb60 con 0x564c2b06f890 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b074e00 0x564c2b084c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 0x564c2b06fc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800fd10 tx=0x7fe6e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600f8c00392422c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6de7fc700 1 -- 172.21.15.16:0/417209527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8038070 con 0x564c2b06f890 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6de7fc700 1 -- 172.21.15.16:0/417209527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6e8011a90 con 0x564c2b06f890 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe6de7fc700 1 -- 172.21.15.16:0/417209527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e802e040 con 0x564c2b06f890 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe709c3fb80 1 -- 172.21.15.16:0/417209527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 msgr2=0x564c2b06fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.664+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 0x564c2b06fc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800fd10 tx=0x7fe6e8005fd0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- 172.21.15.16:0/417209527 shutdown_connections 2024-05-08T23:21:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b075720 0x564c2b069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b06f890 0x564c2b06fc70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/417209527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b074e00 0x564c2b084c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- 172.21.15.16:0/417209527 >> 172.21.15.16:0/417209527 conn(0x564c2afcd210 msgr2=0x564c2b0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- 172.21.15.16:0/417209527 shutdown_connections 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- 172.21.15.16:0/417209527 wait complete. 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- start start 2024-05-08T23:21:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 Processor -- start 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b07e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b07e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b07ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b0de770 con 0x564c2b074e00 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b0de8b0 con 0x564c2b06f890 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2b0de9f0 con 0x564c2b075720 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.665+0000 7fe709c3fb80 10 client.0 _ll_register_callbacks cb 0x564c2af64b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b07e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b07e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45006/0 (socket says 172.21.15.16:45006) 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/420592702 learned_addr learned my addr 172.21.15.16:0/420592702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f7fff700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b07ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b07e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/420592702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 msgr2=0x564c2b07ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b07ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 msgr2=0x564c2b07e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b07e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 -- 172.21.15.16:0/420592702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2ae403b0 con 0x564c2b074e00 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6f6ffd700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b07e690 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00fa10 tx=0x7fe6ec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a751f34b4d326d7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/420592702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec01d040 con 0x564c2b074e00 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/420592702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe6ec000e40 con 0x564c2b074e00 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6e8011c30 con 0x564c2b074e00 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/420592702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6ec00b6f0 con 0x564c2b074e00 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe709c3fb80 10 client.16173 fetch_fsmap learned FSMap version 25 2024-05-08T23:21:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.666+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564c2af85560 con 0x564c2b074e00 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/420592702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6e8011c30 con 0x564c2b074e00 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/420592702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe6ec004be0 con 0x564c2b074e00 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe709c3fb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe709c3fb80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe709c3fb80 20 client.16173 populate_metadata read hostname 'smithi016' 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b0dfc30 0x564c2b0e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe709c3fb80 4 client.16173 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c2afb7ec0 con 0x564c2b0dfc30 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.667+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b0dfc30 0x564c2b0e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe6d6ffd700 10 client.16173 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe6f77fe700 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b0dfc30 0x564c2b0e2050 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9e5d0bb9678db2 server_cookie=283e8b0d233c8556 in_seq=0 out_seq=0 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe6d6ffd700 1 -- 172.21.15.16:0/420592702 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c2afb7ec0 con 0x564c2b0dfc30 2024-05-08T23:21:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe6d6ffd700 10 client.16173 handle_command_reply: tid=0 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 client.16173 shutdown 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 msgr2=0x564c2b07e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b07e690 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00fa10 tx=0x7fe6ec005f10 comp rx=0 tx=0).stop 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 shutdown_connections 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c2b075720 0x564c2b07ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564c2b0dfc30 0x564c2b0e2050 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564c2b06f890 0x564c2b07e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 --2- 172.21.15.16:0/420592702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c2b074e00 0x564c2b07e690 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe6d6ffd700 0 client.16173 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 >> 172.21.15.16:0/420592702 conn(0x564c2afcd210 msgr2=0x564c2b073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 shutdown_connections 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 1 -- 172.21.15.16:0/420592702 wait complete. 2024-05-08T23:21:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.668+0000 7fe709c3fb80 20 client.16173 trim_cache size 0 max 16384 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: "f3ecc856-71dd-4371-b146-f286e5e20b5a": { 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f3ecc856-71dd-4371-b146-f286e5e20b5a", 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:21:59.679 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:21:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe6e00a1290 msgr2=0x7fe6e00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe6e00a1290 0x7fe6e00a3750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800bc10 tx=0x7fe6e800b5f0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 msgr2=0x7fe6f819bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:21:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 0x7fe6f819bff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0007d70 tx=0x7fe6f000a340 comp rx=0 tx=0).stop 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 shutdown_connections 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe6e00a1290 0x7fe6e00a3750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f819bb70 0x7fe6f819bff0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6f8100060 0x7fe6f81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 --2- 172.21.15.16:0/2785523786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6f8105160 0x7fe6f8197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 >> 172.21.15.16:0/2785523786 conn(0x7fe6f8076f40 msgr2=0x7fe6f8068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 shutdown_connections 2024-05-08T23:21:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:21:59.669+0000 7fe6fda85700 1 -- 172.21.15.16:0/2785523786 wait complete. 2024-05-08T23:21:59.683 INFO:tasks.cephfs.filesystem:scrub status for tag:f3ecc856-71dd-4371-b146-f286e5e20b5a - {'path': '/', 'tag': 'f3ecc856-71dd-4371-b146-f286e5e20b5a', 'options': 'recursive,force'} 2024-05-08T23:22:00.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:00 smithi016 ceph-mon[42541]: pgmap v76: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T23:22:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:00 smithi189 ceph-mon[46919]: pgmap v76: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T23:22:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:00 smithi155 ceph-mon[47105]: pgmap v76: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-05-08T23:22:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:01 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:22:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:01 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:01 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:22:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:01 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:01.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:01 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:22:01.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:01 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:22:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:01 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:01 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:22:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:01 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:01.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:01 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:22:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:01 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:22:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:01 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:01 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:22:01.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:01 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:01.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:01 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:22:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:02 smithi016 ceph-mon[42541]: pgmap v77: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-08T23:22:02.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:02 smithi189 ceph-mon[46919]: pgmap v77: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-08T23:22:02.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:02 smithi155 ceph-mon[47105]: pgmap v77: 129 pgs: 129 active+clean; 1.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-05-08T23:22:04.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:04 smithi016 ceph-mon[42541]: pgmap v78: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-05-08T23:22:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:04 smithi189 ceph-mon[46919]: pgmap v78: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-05-08T23:22:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:04 smithi155 ceph-mon[47105]: pgmap v78: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-05-08T23:22:06.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:06 smithi016 ceph-mon[42541]: pgmap v79: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-05-08T23:22:06.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:06 smithi189 ceph-mon[46919]: pgmap v79: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-05-08T23:22:06.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:06 smithi155 ceph-mon[47105]: pgmap v79: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-05-08T23:22:08.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:08 smithi016 ceph-mon[42541]: pgmap v80: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-05-08T23:22:08.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:08 smithi189 ceph-mon[46919]: pgmap v80: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-05-08T23:22:08.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:08 smithi155 ceph-mon[47105]: pgmap v80: 129 pgs: 129 active+clean; 1.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 25 op/s 2024-05-08T23:22:10.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:10 smithi016 ceph-mon[42541]: pgmap v81: 129 pgs: 129 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-05-08T23:22:10.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:10 smithi189 ceph-mon[46919]: pgmap v81: 129 pgs: 129 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-05-08T23:22:10.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:10 smithi155 ceph-mon[47105]: pgmap v81: 129 pgs: 129 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-05-08T23:22:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:12 smithi016 ceph-mon[42541]: pgmap v82: 129 pgs: 129 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-05-08T23:22:12.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:12 smithi189 ceph-mon[46919]: pgmap v82: 129 pgs: 129 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-05-08T23:22:12.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:12 smithi155 ceph-mon[47105]: pgmap v82: 129 pgs: 129 active+clean; 1.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-05-08T23:22:14.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:14 smithi016 ceph-mon[42541]: pgmap v83: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-05-08T23:22:14.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:14 smithi189 ceph-mon[46919]: pgmap v83: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-05-08T23:22:14.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:14 smithi155 ceph-mon[47105]: pgmap v83: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-05-08T23:22:16.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:16 smithi016 ceph-mon[42541]: pgmap v84: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:16.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:16 smithi189 ceph-mon[46919]: pgmap v84: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:16.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:16 smithi155 ceph-mon[47105]: pgmap v84: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:18 smithi016 ceph-mon[42541]: pgmap v85: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:18.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:18 smithi189 ceph-mon[46919]: pgmap v85: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:18.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:18 smithi155 ceph-mon[47105]: pgmap v85: 129 pgs: 129 active+clean; 1.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:20.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:20 smithi016 ceph-mon[42541]: pgmap v86: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-05-08T23:22:20.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:20 smithi189 ceph-mon[46919]: pgmap v86: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-05-08T23:22:20.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:20 smithi155 ceph-mon[47105]: pgmap v86: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-05-08T23:22:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:22 smithi016 ceph-mon[42541]: pgmap v87: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:22.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:22 smithi189 ceph-mon[46919]: pgmap v87: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:22.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:22 smithi155 ceph-mon[47105]: pgmap v87: 129 pgs: 129 active+clean; 2.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:24.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:24 smithi016 ceph-mon[42541]: pgmap v88: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-05-08T23:22:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:24 smithi189 ceph-mon[46919]: pgmap v88: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-05-08T23:22:24.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:24 smithi155 ceph-mon[47105]: pgmap v88: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-05-08T23:22:26.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:26 smithi016 ceph-mon[42541]: pgmap v89: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:26 smithi189 ceph-mon[46919]: pgmap v89: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:26.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:26 smithi155 ceph-mon[47105]: pgmap v89: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:28.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:28 smithi016 ceph-mon[42541]: pgmap v90: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:28.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:28 smithi189 ceph-mon[46919]: pgmap v90: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:28.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:28 smithi155 ceph-mon[47105]: pgmap v90: 129 pgs: 129 active+clean; 2.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-08T23:22:29.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.833+0000 7fb57f4f5700 1 -- 172.21.15.16:0/4286527228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 msgr2=0x7fb5781015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.833+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/4286527228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb5781015c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb574009990 tx=0x7fb574009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.833+0000 7fb57f4f5700 1 -- 172.21.15.16:0/4286527228 shutdown_connections 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/4286527228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb578106300 0x7fb57810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/4286527228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb5781015c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/4286527228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb578101b90 0x7fb578105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 -- 172.21.15.16:0/4286527228 >> 172.21.15.16:0/4286527228 conn(0x7fb5780fca80 msgr2=0x7fb5780feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 -- 172.21.15.16:0/4286527228 shutdown_connections 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 -- 172.21.15.16:0/4286527228 wait complete. 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 Processor -- start 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.834+0000 7fb57f4f5700 1 -- start start 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb578197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb578106300 0x7fb5781979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb57819bd00 0x7fb57819c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57819c780 con 0x7fb5781011e0 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57819c8f0 con 0x7fb57819bd00 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57819cab0 con 0x7fb578106300 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb578197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb578197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49328/0 (socket says 172.21.15.16:49328) 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 -- 172.21.15.16:0/2551857757 learned_addr learned my addr 172.21.15.16:0/2551857757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:29.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb578106300 0x7fb5781979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57da92700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb57819bd00 0x7fb57819c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 -- 172.21.15.16:0/2551857757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb578106300 msgr2=0x7fb5781979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb578106300 0x7fb5781979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 -- 172.21.15.16:0/2551857757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb57819bd00 msgr2=0x7fb57819c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb57819bd00 0x7fb57819c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 -- 172.21.15.16:0/2551857757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740095f0 con 0x7fb5781011e0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57d291700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb578197470 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb5740059d0 tx=0x7fb574005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb56e7fc700 1 -- 172.21.15.16:0/2551857757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57401d070 con 0x7fb5781011e0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb56e7fc700 1 -- 172.21.15.16:0/2551857757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb57400fd40 con 0x7fb5781011e0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.835+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5781a3fb0 con 0x7fb5781011e0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.836+0000 7fb56e7fc700 1 -- 172.21.15.16:0/2551857757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574022d90 con 0x7fb5781011e0 2024-05-08T23:22:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.836+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781a43d0 con 0x7fb5781011e0 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.838+0000 7fb56e7fc700 1 -- 172.21.15.16:0/2551857757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb57400f8b0 con 0x7fb5781011e0 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.838+0000 7fb56e7fc700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5640a1310 0x7fb5640a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.839+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5640a1310 0x7fb5640a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.839+0000 7fb56e7fc700 1 -- 172.21.15.16:0/2551857757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb5740ce320 con 0x7fb5781011e0 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.839+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5640a1310 0x7fb5640a37d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb568005fd0 tx=0x7fb56800a560 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.839+0000 7fb58b6afb80 1 Processor -- start 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.839+0000 7fb58b6afb80 1 -- start start 2024-05-08T23:22:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 0x556970d3e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d38890 0x556970d38cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970c8f150 con 0x556970d3de00 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970c8cb10 con 0x556970d38890 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970c88400 con 0x556970d3e720 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 0x556970d3e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 0x556970d3e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49338/0 (socket says 172.21.15.16:49338) 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 -- 172.21.15.16:0/2136481105 learned_addr learned my addr 172.21.15.16:0/2136481105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57da92700 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d38890 0x556970d38cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 -- 172.21.15.16:0/2136481105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 msgr2=0x556970d38350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d38350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 -- 172.21.15.16:0/2136481105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d38890 msgr2=0x556970d38cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d38890 0x556970d38cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 -- 172.21.15.16:0/2136481105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556970b093b0 con 0x556970d3de00 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d38350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.840+0000 7fb57d291700 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 0x556970d3e1e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb574097ca0 tx=0x7fb57409b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6670654d208d065 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb5637fe700 1 -- 172.21.15.16:0/2136481105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57409b930 con 0x556970d3de00 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb5637fe700 1 -- 172.21.15.16:0/2136481105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb57409b230 con 0x556970d3de00 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb5637fe700 1 -- 172.21.15.16:0/2136481105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57409b6f0 con 0x556970d3de00 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2136481105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 msgr2=0x556970d3e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 0x556970d3e1e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb574097ca0 tx=0x7fb57409b100 comp rx=0 tx=0).stop 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2136481105 shutdown_connections 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d38890 0x556970d38cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3de00 0x556970d3e1e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2136481105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d38350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2136481105 >> 172.21.15.16:0/2136481105 conn(0x556970c96210 msgr2=0x556970d32df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2136481105 shutdown_connections 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.841+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2136481105 wait complete. 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 -- start start 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 Processor -- start 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d44710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d41a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d42160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970c24560 con 0x556970d38890 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970c24200 con 0x556970d3de00 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970b0b6a0 con 0x556970d3e720 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb58b6afb80 10 client.0 _ll_register_callbacks cb 0x556970c2db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.842+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d44710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d44710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49340/0 (socket says 172.21.15.16:49340) 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 -- 172.21.15.16:0/2948593370 learned_addr learned my addr 172.21.15.16:0/2948593370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d41a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57da92700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d42160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 -- 172.21.15.16:0/2948593370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 msgr2=0x556970d42160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d42160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 -- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 msgr2=0x556970d41a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d41a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 -- 172.21.15.16:0/2948593370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781a43d0 con 0x556970d38890 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57da92700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d42160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb57d291700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d44710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb57409b1c0 tx=0x7fb5740cb7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405cf5dd2cb07180 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb560ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb560ff9700 1 -- 172.21.15.16:0/2948593370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740a3620 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb560ff9700 1 -- 172.21.15.16:0/2948593370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb574095e40 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556970be1270 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb560ff9700 1 -- 172.21.15.16:0/2948593370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb57409a720 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb58b6afb80 10 client.16185 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.843+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556970ab5b60 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb560ff9700 1 -- 172.21.15.16:0/2948593370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556970be1270 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb560ff9700 1 -- 172.21.15.16:0/2948593370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb5740a32b0 con 0x556970d38890 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb58b6afb80 10 client.16185 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb58b6afb80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb58b6afb80 20 client.16185 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970d51900 0x556970d53d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb58b6afb80 4 client.16185 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556970d07810 con 0x556970d51900 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970d51900 0x556970d53d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb57ca90700 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970d51900 0x556970d53d20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ec777b637f22b3 server_cookie=e7dbeee490d5bb84 in_seq=0 out_seq=0 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.844+0000 7fb560ff9700 10 client.16185 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb560ff9700 1 -- 172.21.15.16:0/2948593370 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556970d07810 con 0x556970d51900 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb560ff9700 10 client.16185 handle_command_reply: tid=0 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 client.16185 shutdown 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 msgr2=0x556970d44710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d44710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb57409b1c0 tx=0x7fb5740cb7a0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 shutdown_connections 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d42160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d44710 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970d51900 0x556970d53d20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/2948593370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d41a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb560ff9700 0 client.16185 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 >> 172.21.15.16:0/2948593370 conn(0x556970c96210 msgr2=0x556970d325a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 shutdown_connections 2024-05-08T23:22:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.845+0000 7fb58b6afb80 1 -- 172.21.15.16:0/2948593370 wait complete. 2024-05-08T23:22:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.846+0000 7fb58b6afb80 20 client.16185 trim_cache size 0 max 16384 2024-05-08T23:22:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 Processor -- start 2024-05-08T23:22:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 -- start start 2024-05-08T23:22:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d32d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970be1270 con 0x556970d38890 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970d4e9c0 con 0x556970d3de00 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970d2fdf0 con 0x556970d3e720 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d38350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49354/0 (socket says 172.21.15.16:49354) 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 -- 172.21.15.16:0/154085326 learned_addr learned my addr 172.21.15.16:0/154085326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57da92700 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57ca90700 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d32d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 -- 172.21.15.16:0/154085326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 msgr2=0x556970d40340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d40340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 -- 172.21.15.16:0/154085326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 msgr2=0x556970d32d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d32d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.864+0000 7fb57d291700 1 -- 172.21.15.16:0/154085326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556970d38f40 con 0x556970d38890 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb57d291700 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d38350 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb574099580 tx=0x7fb5740ce250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6751c45c351f88d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb563fff700 1 -- 172.21.15.16:0/154085326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740b94d0 con 0x556970d38890 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb563fff700 1 -- 172.21.15.16:0/154085326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5740a3620 con 0x556970d38890 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb563fff700 1 -- 172.21.15.16:0/154085326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57409a720 con 0x556970d38890 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 -- 172.21.15.16:0/154085326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 msgr2=0x556970d38350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d38350 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb574099580 tx=0x7fb5740ce250 comp rx=0 tx=0).stop 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 -- 172.21.15.16:0/154085326 shutdown_connections 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d3e720 0x556970d40340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d38890 0x556970d38350 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/154085326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d32d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.865+0000 7fb58b6afb80 1 -- 172.21.15.16:0/154085326 >> 172.21.15.16:0/154085326 conn(0x556970c96210 msgr2=0x556970d4d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 -- 172.21.15.16:0/154085326 shutdown_connections 2024-05-08T23:22:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 -- 172.21.15.16:0/154085326 wait complete. 2024-05-08T23:22:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 -- start start 2024-05-08T23:22:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 Processor -- start 2024-05-08T23:22:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d38890 0x556970d48f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 0x556970d459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970d45f30 con 0x556970d3e720 2024-05-08T23:22:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970d460a0 con 0x556970d3de00 2024-05-08T23:22:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556970d46260 con 0x556970d38890 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.866+0000 7fb58b6afb80 10 client.0 _ll_register_callbacks cb 0x556970c2db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 0x556970d459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 0x556970d459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49368/0 (socket says 172.21.15.16:49368) 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 -- 172.21.15.16:0/1234561604 learned_addr learned my addr 172.21.15.16:0/1234561604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57d291700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d38890 0x556970d48f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57ca90700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 -- 172.21.15.16:0/1234561604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d38890 msgr2=0x556970d48f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d38890 0x556970d48f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 -- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 msgr2=0x556970d454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 -- 172.21.15.16:0/1234561604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556970ab5b60 con 0x556970d3e720 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57ca90700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb57da92700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 0x556970d459f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb570007b20 tx=0x7fb570007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c231e867ac6fb13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb560ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb560ff9700 1 -- 172.21.15.16:0/1234561604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570016a40 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb560ff9700 1 -- 172.21.15.16:0/1234561604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb570016ba0 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb57409b230 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb560ff9700 1 -- 172.21.15.16:0/1234561604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb570016da0 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb58b6afb80 10 client.16191 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.867+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556970c4e560 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb560ff9700 1 -- 172.21.15.16:0/1234561604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb57409b230 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb560ff9700 1 -- 172.21.15.16:0/1234561604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb570018420 con 0x556970d3e720 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb58b6afb80 10 client.16191 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb58b6afb80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb58b6afb80 20 client.16191 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970dad790 0x556970dafbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb58b6afb80 4 client.16191 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556970c80ec0 con 0x556970dad790 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.868+0000 7fb57d291700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970dad790 0x556970dafbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb57d291700 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970dad790 0x556970dafbb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff04e33b37ca00a server_cookie=466a6510cc79f718 in_seq=0 out_seq=0 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb560ff9700 10 client.16191 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb560ff9700 1 -- 172.21.15.16:0/1234561604 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556970c80ec0 con 0x556970dad790 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb560ff9700 10 client.16191 handle_command_reply: tid=0 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 client.16191 shutdown 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 msgr2=0x556970d459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 0x556970d459f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb570007b20 tx=0x7fb570007e30 comp rx=0 tx=0).stop 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 shutdown_connections 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556970d3e720 0x556970d459f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556970d38890 0x556970d48f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556970dad790 0x556970dafbb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 --2- 172.21.15.16:0/1234561604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556970d3de00 0x556970d454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb560ff9700 0 client.16191 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 >> 172.21.15.16:0/1234561604 conn(0x556970c96210 msgr2=0x556970d3cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 shutdown_connections 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 1 -- 172.21.15.16:0/1234561604 wait complete. 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.869+0000 7fb58b6afb80 20 client.16191 trim_cache size 0 max 16384 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:22:29.877 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:22:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5640a1310 msgr2=0x7fb5640a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5640a1310 0x7fb5640a37d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb568005fd0 tx=0x7fb56800a560 comp rx=0 tx=0).stop 2024-05-08T23:22:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 msgr2=0x7fb578197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb578197470 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb5740059d0 tx=0x7fb574005ef0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 shutdown_connections 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5640a1310 0x7fb5640a37d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb57819bd00 0x7fb57819c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5781011e0 0x7fb578197470 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 --2- 172.21.15.16:0/2551857757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb578106300 0x7fb5781979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 >> 172.21.15.16:0/2551857757 conn(0x7fb5780fca80 msgr2=0x7fb5780fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 shutdown_connections 2024-05-08T23:22:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:29.870+0000 7fb57f4f5700 1 -- 172.21.15.16:0/2551857757 wait complete. 2024-05-08T23:22:29.883 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:22:29.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:22:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 -- 172.21.15.16:0/1462127059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 msgr2=0x7f323410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 --2- 172.21.15.16:0/1462127059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f323410ade0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f321c009990 tx=0x7f321c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 -- 172.21.15.16:0/1462127059 shutdown_connections 2024-05-08T23:22:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 --2- 172.21.15.16:0/1462127059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f323410b400 0x7f323410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 --2- 172.21.15.16:0/1462127059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3234101fe0 0x7f32341023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 --2- 172.21.15.16:0/1462127059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f323410ade0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 -- 172.21.15.16:0/1462127059 >> 172.21.15.16:0/1462127059 conn(0x7f32340fb840 msgr2=0x7f32340fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 -- 172.21.15.16:0/1462127059 shutdown_connections 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.021+0000 7f323a8fc700 1 -- 172.21.15.16:0/1462127059 wait complete. 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 Processor -- start 2024-05-08T23:22:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 -- start start 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3234101fe0 0x7f32341971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f3234197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f323410b400 0x7f323419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419c110 con 0x7f3234102900 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419c280 con 0x7f3234101fe0 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f323a8fc700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323419c440 con 0x7f323410b400 2024-05-08T23:22:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f3234197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f3234197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49392/0 (socket says 172.21.15.16:49392) 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 -- 172.21.15.16:0/4186987818 learned_addr learned my addr 172.21.15.16:0/4186987818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 -- 172.21.15.16:0/4186987818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f323410b400 msgr2=0x7f323419bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f323410b400 0x7f323419bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 -- 172.21.15.16:0/4186987818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3234101fe0 msgr2=0x7f32341971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3234101fe0 0x7f32341971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.022+0000 7f32337fe700 1 -- 172.21.15.16:0/4186987818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0095f0 con 0x7f3234102900 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.023+0000 7f32337fe700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f3234197730 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f321c004760 tx=0x7f321c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.023+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c01d070 con 0x7f3234102900 2024-05-08T23:22:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.023+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f321c005030 con 0x7f3234102900 2024-05-08T23:22:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.023+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323419c6c0 con 0x7f3234102900 2024-05-08T23:22:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.023+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c004b30 con 0x7f3234102900 2024-05-08T23:22:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.023+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323419c930 con 0x7f3234102900 2024-05-08T23:22:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.024+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32341064d0 con 0x7f3234102900 2024-05-08T23:22:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.025+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f321c0051a0 con 0x7f3234102900 2024-05-08T23:22:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.026+0000 7f32317fa700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32200a1290 0x7f32200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.026+0000 7f3233fff700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32200a1290 0x7f32200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.026+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f321c0ce250 con 0x7f3234102900 2024-05-08T23:22:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.026+0000 7f3233fff700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32200a1290 0x7f32200a3750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3224005950 tx=0x7f32240058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.030+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f321c05bd60 con 0x7f3234102900 2024-05-08T23:22:30.067 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:30 smithi016 ceph-mon[42541]: pgmap v91: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-05-08T23:22:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.307+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3234067380 con 0x7f3234102900 2024-05-08T23:22:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.307+0000 7f32317fa700 1 -- 172.21.15.16:0/4186987818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f321c097390 con 0x7f3234102900 2024-05-08T23:22:30.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:22:30.307 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:22:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32200a1290 msgr2=0x7f32200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32200a1290 0x7f32200a3750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3224005950 tx=0x7f32240058e0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 msgr2=0x7f3234197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f3234197730 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f321c004760 tx=0x7f321c009570 comp rx=0 tx=0).stop 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 shutdown_connections 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f32200a1290 0x7f32200a3750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f323410b400 0x7f323419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3234101fe0 0x7f32341971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 --2- 172.21.15.16:0/4186987818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234102900 0x7f3234197730 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 >> 172.21.15.16:0/4186987818 conn(0x7f32340fb840 msgr2=0x7f32340fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 shutdown_connections 2024-05-08T23:22:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.310+0000 7f323a8fc700 1 -- 172.21.15.16:0/4186987818 wait complete. 2024-05-08T23:22:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:22:30.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:22:30.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:30 smithi189 ceph-mon[46919]: pgmap v91: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-05-08T23:22:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 -- 172.21.15.16:0/607426646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2118106300 msgr2=0x7f211810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 --2- 172.21.15.16:0/607426646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2118106300 0x7f211810de80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f211000b0a0 tx=0x7f211000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 -- 172.21.15.16:0/607426646 shutdown_connections 2024-05-08T23:22:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 --2- 172.21.15.16:0/607426646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2118106300 0x7f211810de80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 --2- 172.21.15.16:0/607426646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f21181011e0 0x7f21181015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 --2- 172.21.15.16:0/607426646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2118101b90 0x7f2118105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 -- 172.21.15.16:0/607426646 >> 172.21.15.16:0/607426646 conn(0x7f21180fca80 msgr2=0x7f21180feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 -- 172.21.15.16:0/607426646 shutdown_connections 2024-05-08T23:22:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 -- 172.21.15.16:0/607426646 wait complete. 2024-05-08T23:22:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 Processor -- start 2024-05-08T23:22:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.464+0000 7f211d9b3700 1 -- start start 2024-05-08T23:22:30.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:30 smithi155 ceph-mon[47105]: pgmap v91: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-05-08T23:22:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f211d9b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 0x7f21180732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f211d9b3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2118106300 0x7f21180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f211d9b3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2118073d20 0x7f21181a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f211d9b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21181a5cc0 con 0x7f21181011e0 2024-05-08T23:22:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f211d9b3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21181a5e30 con 0x7f2118073d20 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f211d9b3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21181a5ff0 con 0x7f2118106300 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 0x7f21180732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 0x7f21180732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49406/0 (socket says 172.21.15.16:49406) 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 -- 172.21.15.16:0/9932481 learned_addr learned my addr 172.21.15.16:0/9932481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f21177fe700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2118073d20 0x7f21181a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f210e7fc700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2118106300 0x7f21180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 -- 172.21.15.16:0/9932481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2118106300 msgr2=0x7f21180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2118106300 0x7f21180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 -- 172.21.15.16:0/9932481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2118073d20 msgr2=0x7f21181a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2118073d20 0x7f21181a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.465+0000 7f2116ffd700 1 -- 172.21.15.16:0/9932481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009d00 con 0x7f21181011e0 2024-05-08T23:22:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.466+0000 7f2116ffd700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 0x7f21180732a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f210800d980 tx=0x7f210800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.466+0000 7f211c9b1700 1 -- 172.21.15.16:0/9932481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108004220 con 0x7f21181011e0 2024-05-08T23:22:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.466+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21181a62d0 con 0x7f21181011e0 2024-05-08T23:22:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.466+0000 7f211c9b1700 1 -- 172.21.15.16:0/9932481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2108007640 con 0x7f21181011e0 2024-05-08T23:22:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.466+0000 7f211c9b1700 1 -- 172.21.15.16:0/9932481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080055e0 con 0x7f21181011e0 2024-05-08T23:22:30.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.466+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21181a6830 con 0x7f21181011e0 2024-05-08T23:22:30.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.468+0000 7f211c9b1700 1 -- 172.21.15.16:0/9932481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2108004750 con 0x7f21181011e0 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.469+0000 7f211c9b1700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f21040a1290 0x7f21040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.469+0000 7f210e7fc700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f21040a1290 0x7f21040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.469+0000 7f211c9b1700 1 -- 172.21.15.16:0/9932481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f21080cce40 con 0x7f21181011e0 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.470+0000 7f210e7fc700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f21040a1290 0x7f21040a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21180749d0 tx=0x7f2100009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.470+0000 7f2129b6db80 1 Processor -- start 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.470+0000 7f2129b6db80 1 -- start start 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e9231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e923720 0x562c1e91d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2129b6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e874150 con 0x562c1e922e00 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2129b6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e871b10 con 0x562c1e91d890 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2129b6db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e86d400 con 0x562c1e923720 2024-05-08T23:22:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e9231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e9231e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49416/0 (socket says 172.21.15.16:49416) 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 -- 172.21.15.16:0/4278321124 learned_addr learned my addr 172.21.15.16:0/4278321124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f21177fe700 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f210e7fc700 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e923720 0x562c1e91d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 -- 172.21.15.16:0/4278321124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e923720 msgr2=0x562c1e91d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e923720 0x562c1e91d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 -- 172.21.15.16:0/4278321124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 msgr2=0x562c1e91dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 -- 172.21.15.16:0/4278321124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1e6ee3b0 con 0x562c1e922e00 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f210e7fc700 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e923720 0x562c1e91d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.471+0000 7f2116ffd700 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e9231e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2108096110 tx=0x7f21080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f9895b436d962b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f210dffb700 1 -- 172.21.15.16:0/4278321124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108099c10 con 0x562c1e922e00 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f210dffb700 1 -- 172.21.15.16:0/4278321124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f210809da40 con 0x562c1e922e00 2024-05-08T23:22:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f210dffb700 1 -- 172.21.15.16:0/4278321124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108099100 con 0x562c1e922e00 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 -- 172.21.15.16:0/4278321124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 msgr2=0x562c1e9231e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e9231e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2108096110 tx=0x7f21080057c0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 -- 172.21.15.16:0/4278321124 shutdown_connections 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e9231e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 --2- 172.21.15.16:0/4278321124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e923720 0x562c1e91d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 -- 172.21.15.16:0/4278321124 >> 172.21.15.16:0/4278321124 conn(0x562c1e87b210 msgr2=0x562c1e917df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 -- 172.21.15.16:0/4278321124 shutdown_connections 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.472+0000 7f2129b6db80 1 -- 172.21.15.16:0/4278321124 wait complete. 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 -- start start 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 Processor -- start 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e926aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e922e00 0x562c1e926fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e927520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e809560 con 0x562c1e923720 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e809200 con 0x562c1e922e00 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.473+0000 7f2129b6db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e6f06a0 con 0x562c1e91d890 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f2129b6db80 10 client.0 _ll_register_callbacks cb 0x562c1e812b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e927520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e927520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49432/0 (socket says 172.21.15.16:49432) 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 -- 172.21.15.16:0/567261141 learned_addr learned my addr 172.21.15.16:0/567261141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f2116ffd700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e926aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f210e7fc700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e922e00 0x562c1e926fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 -- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 msgr2=0x562c1e926aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e926aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 -- 172.21.15.16:0/567261141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e922e00 msgr2=0x562c1e926fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e922e00 0x562c1e926fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 -- 172.21.15.16:0/567261141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21181a6830 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f21177fe700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e927520 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f211000bc80 tx=0x7f211000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb11f39e42dd5465 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f20f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f20f6ffd700 1 -- 172.21.15.16:0/567261141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000e040 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f20f6ffd700 1 -- 172.21.15.16:0/567261141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2110007e30 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.474+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c1e7c6270 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f20f6ffd700 1 -- 172.21.15.16:0/567261141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2110009490 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 10 client.16209 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562c1e69ab60 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f20f6ffd700 1 -- 172.21.15.16:0/567261141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562c1e7c6270 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f20f6ffd700 1 -- 172.21.15.16:0/567261141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f211000b0d0 con 0x562c1e923720 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 10 client.16209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:22:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 20 client.16209 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e937070 0x562c1e939490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 4 client.16209 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c1e8ec810 con 0x562c1e937070 2024-05-08T23:22:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.475+0000 7f2116ffd700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e937070 0x562c1e939490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.476+0000 7f20f6ffd700 10 client.16209 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:22:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.476+0000 7f2116ffd700 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e937070 0x562c1e939490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=349a56b9aefef906 server_cookie=eff89ad1951a40b in_seq=0 out_seq=0 2024-05-08T23:22:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f20f6ffd700 1 -- 172.21.15.16:0/567261141 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c1e8ec810 con 0x562c1e937070 2024-05-08T23:22:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f20f6ffd700 10 client.16209 handle_command_reply: tid=0 2024-05-08T23:22:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 client.16209 shutdown 2024-05-08T23:22:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 msgr2=0x562c1e927520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e927520 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f211000bc80 tx=0x7f211000bcb0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 shutdown_connections 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e927520 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e926aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e937070 0x562c1e939490 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 --2- 172.21.15.16:0/567261141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e922e00 0x562c1e926fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 >> 172.21.15.16:0/567261141 conn(0x562c1e87b210 msgr2=0x562c1e9175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f20f6ffd700 0 client.16209 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 shutdown_connections 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 1 -- 172.21.15.16:0/567261141 wait complete. 2024-05-08T23:22:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.477+0000 7f2129b6db80 20 client.16209 trim_cache size 0 max 16384 2024-05-08T23:22:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 Processor -- start 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 -- start start 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e922e00 0x562c1e917d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e92e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e919a20 con 0x562c1e923720 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e9188e0 con 0x562c1e91d890 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2129b6db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e929c20 con 0x562c1e922e00 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e92e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e92e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 -- 172.21.15.16:0/1412988313 learned_addr learned my addr 172.21.15.16:0/1412988313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2116ffd700 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f210e7fc700 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e922e00 0x562c1e917d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 -- 172.21.15.16:0/1412988313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e922e00 msgr2=0x562c1e917d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e922e00 0x562c1e917d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 -- 172.21.15.16:0/1412988313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 msgr2=0x562c1e91d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f21177fe700 1 -- 172.21.15.16:0/1412988313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1e833560 con 0x562c1e923720 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.491+0000 7f2116ffd700 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f21177fe700 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e92e000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f21100057f0 tx=0x7f2110006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1443a9d57b77aea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f210effd700 1 -- 172.21.15.16:0/1412988313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211002a030 con 0x562c1e923720 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f210effd700 1 -- 172.21.15.16:0/1412988313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2110003e90 con 0x562c1e923720 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f210effd700 1 -- 172.21.15.16:0/1412988313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000e070 con 0x562c1e923720 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 -- 172.21.15.16:0/1412988313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 msgr2=0x562c1e92e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e92e000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f21100057f0 tx=0x7f2110006160 comp rx=0 tx=0).stop 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 -- 172.21.15.16:0/1412988313 shutdown_connections 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e923720 0x562c1e92e000 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e91d890 0x562c1e91d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1412988313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e922e00 0x562c1e917d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.492+0000 7f2129b6db80 1 -- 172.21.15.16:0/1412988313 >> 172.21.15.16:0/1412988313 conn(0x562c1e87b210 msgr2=0x562c1e925370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 -- 172.21.15.16:0/1412988313 shutdown_connections 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 -- 172.21.15.16:0/1412988313 wait complete. 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 -- start start 2024-05-08T23:22:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 Processor -- start 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e9203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e92e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e923720 0x562c1e91e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e92e900 con 0x562c1e922e00 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e91ee70 con 0x562c1e923720 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.493+0000 7f2129b6db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1e91f000 con 0x562c1e91d890 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f2129b6db80 10 client.0 _ll_register_callbacks cb 0x562c1e812b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e92e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e92e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49448/0 (socket says 172.21.15.16:49448) 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 -- 172.21.15.16:0/1458811990 learned_addr learned my addr 172.21.15.16:0/1458811990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f2116ffd700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e9203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f21177fe700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e923720 0x562c1e91e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 -- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 msgr2=0x562c1e9203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e9203d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 -- 172.21.15.16:0/1458811990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e923720 msgr2=0x562c1e91e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e923720 0x562c1e91e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 -- 172.21.15.16:0/1458811990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c1e69ab60 con 0x562c1e922e00 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f21177fe700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e923720 0x562c1e91e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f210e7fc700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e92e010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21000097b0 tx=0x7f21000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0acbf3fd7d07f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f20f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f20f6ffd700 1 -- 172.21.15.16:0/1458811990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100025070 con 0x562c1e922e00 2024-05-08T23:22:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f20f6ffd700 1 -- 172.21.15.16:0/1458811990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2100017070 con 0x562c1e922e00 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f210809da40 con 0x562c1e922e00 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f20f6ffd700 1 -- 172.21.15.16:0/1458811990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2100011810 con 0x562c1e922e00 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.494+0000 7f2129b6db80 10 client.16215 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562c1e6f2cd0 con 0x562c1e922e00 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f20f6ffd700 1 -- 172.21.15.16:0/1458811990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f210809da40 con 0x562c1e922e00 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f20f6ffd700 1 -- 172.21.15.16:0/1458811990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2100029960 con 0x562c1e922e00 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 10 client.16215 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 10 client.16215 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 20 client.16215 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e98e4c0 0x562c1e9908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 4 client.16215 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c1e865ec0 con 0x562c1e98e4c0 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.495+0000 7f2116ffd700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e98e4c0 0x562c1e9908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2116ffd700 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e98e4c0 0x562c1e9908e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=692b7307226d8925 server_cookie=8162050e0eedf04e in_seq=0 out_seq=0 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f20f6ffd700 10 client.16215 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f20f6ffd700 1 -- 172.21.15.16:0/1458811990 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c1e865ec0 con 0x562c1e98e4c0 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f20f6ffd700 10 client.16215 handle_command_reply: tid=0 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 client.16215 shutdown 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 msgr2=0x562c1e92e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e92e010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f21000097b0 tx=0x7f21000047c0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 shutdown_connections 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562c1e98e4c0 0x562c1e9908e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1e923720 0x562c1e91e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f20f6ffd700 0 client.16215 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:22:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1e91d890 0x562c1e9203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 --2- 172.21.15.16:0/1458811990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1e922e00 0x562c1e92e010 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 >> 172.21.15.16:0/1458811990 conn(0x562c1e87b210 msgr2=0x562c1e926a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 shutdown_connections 2024-05-08T23:22:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 1 -- 172.21.15.16:0/1458811990 wait complete. 2024-05-08T23:22:30.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.496+0000 7f2129b6db80 20 client.16215 trim_cache size 0 max 16384 2024-05-08T23:22:30.503 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f21040a1290 msgr2=0x7f21040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f21040a1290 0x7f21040a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21180749d0 tx=0x7f2100009260 comp rx=0 tx=0).stop 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 msgr2=0x7f21180732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 0x7f21180732a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f210800d980 tx=0x7f210800dc90 comp rx=0 tx=0).stop 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 shutdown_connections 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f21040a1290 0x7f21040a3750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2118073d20 0x7f21181a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21181011e0 0x7f21180732a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 --2- 172.21.15.16:0/9932481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2118106300 0x7f21180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 >> 172.21.15.16:0/9932481 conn(0x7f21180fca80 msgr2=0x7f21180fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 shutdown_connections 2024-05-08T23:22:30.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.497+0000 7f211d9b3700 1 -- 172.21.15.16:0/9932481 wait complete. 2024-05-08T23:22:30.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 -- 172.21.15.16:0/1062376512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab4100060 msgr2=0x7f0ab4100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 --2- 172.21.15.16:0/1062376512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab4100060 0x7f0ab4100440 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009990 tx=0x7f0aa4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 -- 172.21.15.16:0/1062376512 shutdown_connections 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 --2- 172.21.15.16:0/1062376512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4105160 0x7f0ab410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 --2- 172.21.15.16:0/1062376512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab4100060 0x7f0ab4100440 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 --2- 172.21.15.16:0/1062376512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ab4100a10 0x7f0ab4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.650+0000 7f0abaa53700 1 -- 172.21.15.16:0/1062376512 >> 172.21.15.16:0/1062376512 conn(0x7f0ab4076f40 msgr2=0x7f0ab4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.651+0000 7f0abaa53700 1 -- 172.21.15.16:0/1062376512 shutdown_connections 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.651+0000 7f0abaa53700 1 -- 172.21.15.16:0/1062376512 wait complete. 2024-05-08T23:22:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.651+0000 7f0abaa53700 1 Processor -- start 2024-05-08T23:22:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.651+0000 7f0abaa53700 1 -- start start 2024-05-08T23:22:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4100060 0x7f0ab41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ab4105160 0x7f0ab4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 0x7f0ab419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419c650 con 0x7f0ab419bbd0 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419c7c0 con 0x7f0ab4100060 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab419c980 con 0x7f0ab4105160 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 0x7f0ab419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 0x7f0ab419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49468/0 (socket says 172.21.15.16:49468) 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 -- 172.21.15.16:0/2284323623 learned_addr learned my addr 172.21.15.16:0/2284323623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ab4105160 0x7f0ab4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4100060 0x7f0ab41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 -- 172.21.15.16:0/2284323623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ab4105160 msgr2=0x7f0ab4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ab4105160 0x7f0ab4197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 -- 172.21.15.16:0/2284323623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4100060 msgr2=0x7f0ab41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4100060 0x7f0ab41972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 -- 172.21.15.16:0/2284323623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40095f0 con 0x7f0ab419bbd0 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4100060 0x7f0ab41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 0x7f0ab419c050 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0aac007ee0 tx=0x7f0aac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab17fa700 1 -- 172.21.15.16:0/2284323623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac017070 con 0x7f0ab419bbd0 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0ab17fa700 1 -- 172.21.15.16:0/2284323623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aac00f040 con 0x7f0ab419bbd0 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.652+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab41a3ee0 con 0x7f0ab419bbd0 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.653+0000 7f0ab17fa700 1 -- 172.21.15.16:0/2284323623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac005710 con 0x7f0ab419bbd0 2024-05-08T23:22:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.653+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab41a43c0 con 0x7f0ab419bbd0 2024-05-08T23:22:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.655+0000 7f0ab17fa700 1 -- 172.21.15.16:0/2284323623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0aac005870 con 0x7f0ab419bbd0 2024-05-08T23:22:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.655+0000 7f0ab17fa700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a9c0a1310 0x7f0a9c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.655+0000 7f0ab17fa700 1 -- 172.21.15.16:0/2284323623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f0aac0cc700 con 0x7f0ab419bbd0 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.656+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a9c0a1310 0x7f0a9c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.656+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a9c0a1310 0x7f0a9c0a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b2e0 tx=0x7f0aa4005c00 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.656+0000 7f0ac6c0db80 1 Processor -- start 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.656+0000 7f0ac6c0db80 1 -- start start 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x56437867e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378678760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x5643786790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643785cb150 con 0x56437867e210 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643785c8b10 con 0x56437867eb30 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643785c4400 con 0x564378678ca0 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x56437867e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x56437867e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49480/0 (socket says 172.21.15.16:49480) 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 -- 172.21.15.16:0/2175698174 learned_addr learned my addr 172.21.15.16:0/2175698174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378678760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x5643786790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 -- 172.21.15.16:0/2175698174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 msgr2=0x5643786790e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x5643786790e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 -- 172.21.15.16:0/2175698174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 msgr2=0x564378678760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378678760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 -- 172.21.15.16:0/2175698174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643784453b0 con 0x56437867e210 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.657+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x56437867e5f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4015000 tx=0x7f0aa401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6798c9aa215dfba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0a9a7fc700 1 -- 172.21.15.16:0/2175698174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa402b070 con 0x56437867e210 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0a9a7fc700 1 -- 172.21.15.16:0/2175698174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aa401e040 con 0x56437867e210 2024-05-08T23:22:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0a9a7fc700 1 -- 172.21.15.16:0/2175698174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4030460 con 0x56437867e210 2024-05-08T23:22:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2175698174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 msgr2=0x56437867e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x56437867e5f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4015000 tx=0x7f0aa401f000 comp rx=0 tx=0).stop 2024-05-08T23:22:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2175698174 shutdown_connections 2024-05-08T23:22:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x5643786790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x56437867e5f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2175698174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378678760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.658+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2175698174 >> 172.21.15.16:0/2175698174 conn(0x5643785d2210 msgr2=0x56437866edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.659+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2175698174 shutdown_connections 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.659+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2175698174 wait complete. 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.659+0000 7f0ac6c0db80 1 -- start start 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.659+0000 7f0ac6c0db80 1 Processor -- start 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.659+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564378678ca0 0x564378681ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 0x564378682400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56437867eb30 0x564378682940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564378560560 con 0x564378678ca0 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564378560200 con 0x56437867e210 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643784476a0 con 0x56437867eb30 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ac6c0db80 10 client.0 _ll_register_callbacks cb 0x564378569b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:30.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 0x564378682400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 0x564378682400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:32788/0 (socket says 172.21.15.16:32788) 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 -- 172.21.15.16:0/3167921096 learned_addr learned my addr 172.21.15.16:0/3167921096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56437867eb30 0x564378682940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 -- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56437867eb30 msgr2=0x564378682940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56437867eb30 0x564378682940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 -- 172.21.15.16:0/3167921096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564378678ca0 msgr2=0x564378681ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564378678ca0 0x564378681ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564378678ca0 0x564378681ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 -- 172.21.15.16:0/3167921096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab410d550 con 0x56437867e210 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564378678ca0 0x564378681ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 0x564378682400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80046e0 tx=0x7f0aa80129b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503bbc7bc2b948be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.660+0000 7f0a937fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:22:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0a937fe700 1 -- 172.21.15.16:0/3167921096 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8017070 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0a937fe700 1 -- 172.21.15.16:0/3167921096 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f0aa8010430 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56437851d270 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0a937fe700 1 -- 172.21.15.16:0/3167921096 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa80122e0 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 10 client.25973 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5643783f1b60 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0a937fe700 1 -- 172.21.15.16:0/3167921096 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56437851d270 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0a937fe700 1 -- 172.21.15.16:0/3167921096 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0aa801c780 con 0x56437867e210 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 10 client.25973 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 10 client.25973 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 20 client.25973 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786925e0 0x564378694a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 4 client.25973 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.661+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564378643810 con 0x5643786925e0 2024-05-08T23:22:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.662+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786925e0 0x564378694a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.662+0000 7f0a937fe700 10 client.25973 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.662+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786925e0 0x564378694a00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=609998cc85987bf2 server_cookie=e734ffe4f447e67e in_seq=0 out_seq=0 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0a937fe700 1 -- 172.21.15.16:0/3167921096 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564378643810 con 0x5643786925e0 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0a937fe700 10 client.25973 handle_command_reply: tid=0 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 client.25973 shutdown 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 msgr2=0x564378682400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 0x564378682400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80046e0 tx=0x7f0aa80129b0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 shutdown_connections 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56437867eb30 0x564378682940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564378678ca0 0x564378681ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786925e0 0x564378694a00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/3167921096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867e210 0x564378682400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 >> 172.21.15.16:0/3167921096 conn(0x5643785d2210 msgr2=0x56437866e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0a937fe700 0 client.25973 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:22:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 shutdown_connections 2024-05-08T23:22:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/3167921096 wait complete. 2024-05-08T23:22:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.663+0000 7f0ac6c0db80 20 client.25973 trim_cache size 0 max 16384 2024-05-08T23:22:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 Processor -- start 2024-05-08T23:22:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 -- start start 2024-05-08T23:22:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x56437868e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378679080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x56437866ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564378683530 con 0x56437867e210 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564378667060 con 0x56437867eb30 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564378699c30 con 0x564378678ca0 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.677+0000 7f0ab37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378679080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378679080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49492/0 (socket says 172.21.15.16:49492) 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2881574373 learned_addr learned my addr 172.21.15.16:0/2881574373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x56437868e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x56437866ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2881574373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 msgr2=0x56437868e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x56437868e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2881574373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 msgr2=0x56437866ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x56437866ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2881574373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437858a560 con 0x56437867e210 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x56437866ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378679080 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80046b0 tx=0x7f0aa8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b3eb952f92bfa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0a9affd700 1 -- 172.21.15.16:0/2881574373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8017040 con 0x56437867e210 2024-05-08T23:22:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0a9affd700 1 -- 172.21.15.16:0/2881574373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aa80122e0 con 0x56437867e210 2024-05-08T23:22:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.678+0000 7f0a9affd700 1 -- 172.21.15.16:0/2881574373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8010e30 con 0x56437867e210 2024-05-08T23:22:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2881574373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 msgr2=0x564378679080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378679080 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80046b0 tx=0x7f0aa8009750 comp rx=0 tx=0).stop 2024-05-08T23:22:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2881574373 shutdown_connections 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x56437866ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x56437868e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2881574373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378679080 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2881574373 >> 172.21.15.16:0/2881574373 conn(0x5643785d2210 msgr2=0x56437866b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2881574373 shutdown_connections 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2881574373 wait complete. 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.679+0000 7f0ac6c0db80 1 -- start start 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 Processor -- start 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x564378685540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378685a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378685fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643786e7af0 con 0x56437867e210 2024-05-08T23:22:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643786e7c30 con 0x56437867eb30 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643786e7df0 con 0x564378678ca0 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ac6c0db80 10 client.0 _ll_register_callbacks cb 0x564378569b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378685a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378685a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49498/0 (socket says 172.21.15.16:49498) 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2208483375 learned_addr learned my addr 172.21.15.16:0/2208483375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab8ff0700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378685fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x564378685540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 msgr2=0x564378685540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x564378685540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2208483375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 msgr2=0x564378685fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378685fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.680+0000 7f0ab37fe700 1 -- 172.21.15.16:0/2208483375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643783f1b60 con 0x56437867e210 2024-05-08T23:22:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ab37fe700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378685a80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009e00 tx=0x7f0aa802a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b7cab69ddc5535 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0a937fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0a937fe700 1 -- 172.21.15.16:0/2208483375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa802acb0 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0a937fe700 1 -- 172.21.15.16:0/2208483375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f0aa8000e40 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aa401e040 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0a937fe700 1 -- 172.21.15.16:0/2208483375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa802ae10 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 10 client.16230 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564378449cd0 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0a937fe700 1 -- 172.21.15.16:0/2208483375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0aa401e040 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0a937fe700 1 -- 172.21.15.16:0/2208483375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0aa801cd20 con 0x56437867e210 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 10 client.16230 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 10 client.16230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 20 client.16230 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.681+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786e91f0 0x5643786eb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0ac6c0db80 4 client.16230 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643785bcec0 con 0x5643786e91f0 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786e91f0 0x5643786eb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0ab3fff700 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786e91f0 0x5643786eb5e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3c37cef7f7477e8 server_cookie=a73381afb04303ed in_seq=0 out_seq=0 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0a937fe700 10 client.16230 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0a937fe700 1 -- 172.21.15.16:0/2208483375 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643785bcec0 con 0x5643786e91f0 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0a937fe700 10 client.16230 handle_command_reply: tid=0 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.682+0000 7f0ac6c0db80 1 client.16230 shutdown 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 msgr2=0x564378685a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378685a80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009e00 tx=0x7f0aa802a670 comp rx=0 tx=0).stop 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 shutdown_connections 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5643786e91f0 0x5643786eb5e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56437867eb30 0x564378685fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564378678ca0 0x564378685540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0a937fe700 0 client.16230 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:22:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 --2- 172.21.15.16:0/2208483375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56437867e210 0x564378685a80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 >> 172.21.15.16:0/2208483375 conn(0x5643785d2210 msgr2=0x5643786707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 shutdown_connections 2024-05-08T23:22:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 1 -- 172.21.15.16:0/2208483375 wait complete. 2024-05-08T23:22:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0ac6c0db80 20 client.16230 trim_cache size 0 max 16384 2024-05-08T23:22:30.689 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a9c0a1310 msgr2=0x7f0a9c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a9c0a1310 0x7f0a9c0a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b2e0 tx=0x7f0aa4005c00 comp rx=0 tx=0).stop 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 msgr2=0x7f0ab419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 0x7f0ab419c050 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0aac007ee0 tx=0x7f0aac00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 shutdown_connections 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a9c0a1310 0x7f0a9c0a37d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ab419bbd0 0x7f0ab419c050 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ab4100060 0x7f0ab41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 --2- 172.21.15.16:0/2284323623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ab4105160 0x7f0ab4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.683+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 >> 172.21.15.16:0/2284323623 conn(0x7f0ab4076f40 msgr2=0x7f0ab4068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.684+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 shutdown_connections 2024-05-08T23:22:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.684+0000 7f0abaa53700 1 -- 172.21.15.16:0/2284323623 wait complete. 2024-05-08T23:22:30.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.836+0000 7f5c4536c700 1 -- 172.21.15.16:0/1088472200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c401062a0 msgr2=0x7f5c4010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.836+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1088472200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c401062a0 0x7f5c4010de20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800b0d0 tx=0x7f5c3800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.836+0000 7f5c4536c700 1 -- 172.21.15.16:0/1088472200 shutdown_connections 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.836+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1088472200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c401062a0 0x7f5c4010de20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.836+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1088472200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c400697e0 0x7f5c40069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.836+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1088472200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c4006a190 0x7f5c40105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.837+0000 7f5c4536c700 1 -- 172.21.15.16:0/1088472200 >> 172.21.15.16:0/1088472200 conn(0x7f5c40076f40 msgr2=0x7f5c40077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.837+0000 7f5c4536c700 1 -- 172.21.15.16:0/1088472200 shutdown_connections 2024-05-08T23:22:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.837+0000 7f5c4536c700 1 -- 172.21.15.16:0/1088472200 wait complete. 2024-05-08T23:22:30.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.837+0000 7f5c4536c700 1 Processor -- start 2024-05-08T23:22:30.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.837+0000 7f5c4536c700 1 -- start start 2024-05-08T23:22:30.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c4536c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c400697e0 0x7f5c40199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c4536c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c401062a0 0x7f5c40199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c4536c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 0x7f5c4019e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c4536c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4019e950 con 0x7f5c4019ded0 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c4536c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4019eac0 con 0x7f5c400697e0 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c4536c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4019ec80 con 0x7f5c401062a0 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 0x7f5c4019e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 0x7f5c4019e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49516/0 (socket says 172.21.15.16:49516) 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 -- 172.21.15.16:0/1295598440 learned_addr learned my addr 172.21.15.16:0/1295598440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3effd700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c400697e0 0x7f5c40199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c401062a0 0x7f5c40199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 -- 172.21.15.16:0/1295598440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c401062a0 msgr2=0x7f5c40199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c401062a0 0x7f5c40199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 -- 172.21.15.16:0/1295598440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c400697e0 msgr2=0x7f5c40199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c400697e0 0x7f5c40199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 -- 172.21.15.16:0/1295598440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38009d00 con 0x7f5c4019ded0 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c401062a0 0x7f5c40199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 0x7f5c4019e350 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800d010 tx=0x7f5c38007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.838+0000 7f5c27fff700 1 -- 172.21.15.16:0/1295598440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3800e040 con 0x7f5c4019ded0 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.839+0000 7f5c27fff700 1 -- 172.21.15.16:0/1295598440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c3801d910 con 0x7f5c4019ded0 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.839+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c401a6180 con 0x7f5c4019ded0 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.839+0000 7f5c27fff700 1 -- 172.21.15.16:0/1295598440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801b850 con 0x7f5c4019ded0 2024-05-08T23:22:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.839+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c401a6600 con 0x7f5c4019ded0 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.841+0000 7f5c27fff700 1 -- 172.21.15.16:0/1295598440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c3801d480 con 0x7f5c4019ded0 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.842+0000 7f5c27fff700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c280a1290 0x7f5c280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.842+0000 7f5c3effd700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c280a1290 0x7f5c280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.842+0000 7f5c27fff700 1 -- 172.21.15.16:0/1295598440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5c380ce870 con 0x7f5c4019ded0 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.842+0000 7f5c3effd700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c280a1290 0x7f5c280a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005950 tx=0x7f5c3000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 Processor -- start 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 -- start start 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da0e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da08cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d95f150 con 0x55798da0de00 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d95cb10 con 0x55798da0e720 2024-05-08T23:22:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c51526b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d958400 con 0x55798da08890 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da0e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da0e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49530/0 (socket says 172.21.15.16:49530) 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c3effd700 1 -- 172.21.15.16:0/1985445113 learned_addr learned my addr 172.21.15.16:0/1985445113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.843+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da08cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c3effd700 1 -- 172.21.15.16:0/1985445113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 msgr2=0x55798da08cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c3effd700 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da08cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c3effd700 1 -- 172.21.15.16:0/1985445113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 msgr2=0x55798da08350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c3effd700 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c3effd700 1 -- 172.21.15.16:0/1985445113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798d7d93b0 con 0x55798da0de00 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c3effd700 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da0e1e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5c30010fa0 tx=0x7f5c30013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cedb98b42e3f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c257fa700 1 -- 172.21.15.16:0/1985445113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30011db0 con 0x55798da0de00 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c257fa700 1 -- 172.21.15.16:0/1985445113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c30015070 con 0x55798da0de00 2024-05-08T23:22:30.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.844+0000 7f5c257fa700 1 -- 172.21.15.16:0/1985445113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3001c610 con 0x55798da0de00 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 -- 172.21.15.16:0/1985445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 msgr2=0x55798da0e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da0e1e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5c30010fa0 tx=0x7f5c30013000 comp rx=0 tx=0).stop 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 -- 172.21.15.16:0/1985445113 shutdown_connections 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da08cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da0e1e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 --2- 172.21.15.16:0/1985445113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 -- 172.21.15.16:0/1985445113 >> 172.21.15.16:0/1985445113 conn(0x55798d966210 msgr2=0x55798da02df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 -- 172.21.15.16:0/1985445113 shutdown_connections 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 -- 172.21.15.16:0/1985445113 wait complete. 2024-05-08T23:22:30.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 -- start start 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.845+0000 7f5c51526b80 1 Processor -- start 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da08890 0x55798da11a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da11f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0e720 0x55798da124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d8f4560 con 0x55798da0de00 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d8f4200 con 0x55798da08890 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d7db6a0 con 0x55798da0e720 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 10 client.0 _ll_register_callbacks cb 0x55798d8fdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da11f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da11f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49542/0 (socket says 172.21.15.16:49542) 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/3764250932 learned_addr learned my addr 172.21.15.16:0/3764250932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0e720 0x55798da124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da08890 0x55798da11a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/3764250932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0e720 msgr2=0x55798da124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0e720 0x55798da124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da08890 msgr2=0x55798da11a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da08890 0x55798da11a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/3764250932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c30010c50 con 0x55798da0de00 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da08890 0x55798da11a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da11f70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009f20 tx=0x7f5c340116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170395cd7cb2d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3764250932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34004220 con 0x55798da0de00 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3764250932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5c34004380 con 0x55798da0de00 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.846+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55798d8b1270 con 0x55798da0de00 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3764250932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c34004540 con 0x55798da0de00 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 10 client.16242 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55798d785b60 con 0x55798da0de00 2024-05-08T23:22:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3764250932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55798d8b1270 con 0x55798da0de00 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3764250932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c3400c070 con 0x55798da0de00 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 20 client.16242 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da22230 0x55798da24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 4 client.16242 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55798d9d7810 con 0x55798da22230 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.847+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da22230 0x55798da24650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.848+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da22230 0x55798da24650 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffa5afc1af15b32 server_cookie=518671acdd09bd3b in_seq=0 out_seq=0 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.848+0000 7f5c22ff5700 10 client.16242 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3764250932 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55798d9d7810 con 0x55798da22230 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c22ff5700 10 client.16242 handle_command_reply: tid=0 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 client.16242 shutdown 2024-05-08T23:22:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 msgr2=0x55798da11f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da11f70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009f20 tx=0x7f5c340116c0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 shutdown_connections 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0e720 0x55798da124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da22230 0x55798da24650 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da08890 0x55798da11a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3764250932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da11f70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c22ff5700 0 client.16242 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 >> 172.21.15.16:0/3764250932 conn(0x55798d966210 msgr2=0x55798da025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 shutdown_connections 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 1 -- 172.21.15.16:0/3764250932 wait complete. 2024-05-08T23:22:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.849+0000 7f5c51526b80 20 client.16242 trim_cache size 0 max 16384 2024-05-08T23:22:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.863+0000 7f5c51526b80 1 Processor -- start 2024-05-08T23:22:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.863+0000 7f5c51526b80 1 -- start start 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da24650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da1dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c51526b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d9fd800 con 0x55798da0de00 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c51526b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798da03a20 con 0x55798da0e720 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c51526b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798d9fb040 con 0x55798da08890 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da1dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da1dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49554/0 (socket says 172.21.15.16:49554) 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/541104753 learned_addr learned my addr 172.21.15.16:0/541104753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3effd700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da24650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/541104753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 msgr2=0x55798da24650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da24650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/541104753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 msgr2=0x55798da08c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 -- 172.21.15.16:0/541104753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798d91e560 con 0x55798da0de00 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3effd700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da24650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da1dc20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5c34012850 tx=0x7f5c34005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b504a2e6b2b612a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.864+0000 7f5c25ffb700 1 -- 172.21.15.16:0/541104753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34004020 con 0x55798da0de00 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c25ffb700 1 -- 172.21.15.16:0/541104753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c34005030 con 0x55798da0de00 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c25ffb700 1 -- 172.21.15.16:0/541104753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34004990 con 0x55798da0de00 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 -- 172.21.15.16:0/541104753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 msgr2=0x55798da1dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da1dc20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5c34012850 tx=0x7f5c34005cc0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 -- 172.21.15.16:0/541104753 shutdown_connections 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da08c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da08890 0x55798da24650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 --2- 172.21.15.16:0/541104753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da0de00 0x55798da1dc20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 -- 172.21.15.16:0/541104753 >> 172.21.15.16:0/541104753 conn(0x55798d966210 msgr2=0x55798da10300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 -- 172.21.15.16:0/541104753 shutdown_connections 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 -- 172.21.15.16:0/541104753 wait complete. 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.865+0000 7f5c51526b80 1 -- start start 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 Processor -- start 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 0x55798da19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0de00 0x55798da15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da15880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798da15fc0 con 0x55798da08890 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798da77740 con 0x55798da0e720 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798da77880 con 0x55798da0de00 2024-05-08T23:22:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c51526b80 10 client.0 _ll_register_callbacks cb 0x55798d8fdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 0x55798da19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 0x55798da19680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49568/0 (socket says 172.21.15.16:49568) 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 -- 172.21.15.16:0/3122016588 learned_addr learned my addr 172.21.15.16:0/3122016588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3f7fe700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da15880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0de00 0x55798da15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 -- 172.21.15.16:0/3122016588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0de00 msgr2=0x55798da15140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0de00 0x55798da15140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 -- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 msgr2=0x55798da15880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da15880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.866+0000 7f5c3effd700 1 -- 172.21.15.16:0/3122016588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798d785b60 con 0x55798da08890 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c3effd700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 0x55798da19680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5c30010fa0 tx=0x7f5c30010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2f46b3f4dc63ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3122016588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3002c040 con 0x55798da08890 2024-05-08T23:22:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3122016588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5c30017070 con 0x55798da08890 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c30015070 con 0x55798da08890 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3122016588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c30012e60 con 0x55798da08890 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 10 client.16248 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55798d7ddcd0 con 0x55798da08890 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3122016588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c30015070 con 0x55798da08890 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3122016588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c300047a0 con 0x55798da08890 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 20 client.16248 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da78ec0 0x55798da7b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 4 client.16248 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.867+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55798d950ec0 con 0x55798da78ec0 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da78ec0 0x55798da7b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c22ff5700 10 client.16248 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c3e7fc700 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da78ec0 0x55798da7b2e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed28386f2284e27c server_cookie=6b77ea986db79010 in_seq=0 out_seq=0 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c22ff5700 1 -- 172.21.15.16:0/3122016588 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55798d950ec0 con 0x55798da78ec0 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c22ff5700 10 client.16248 handle_command_reply: tid=0 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 client.16248 shutdown 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 msgr2=0x55798da19680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 0x55798da19680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5c30010fa0 tx=0x7f5c30010a40 comp rx=0 tx=0).stop 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 shutdown_connections 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55798da78ec0 0x55798da7b2e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55798da0e720 0x55798da15880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798da08890 0x55798da19680 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c22ff5700 0 client.16248 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 --2- 172.21.15.16:0/3122016588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55798da0de00 0x55798da15140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.868+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 >> 172.21.15.16:0/3122016588 conn(0x55798d966210 msgr2=0x55798da0cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 shutdown_connections 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c51526b80 1 -- 172.21.15.16:0/3122016588 wait complete. 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c51526b80 20 client.16248 trim_cache size 0 max 16384 2024-05-08T23:22:30.875 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c280a1290 msgr2=0x7f5c280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c280a1290 0x7f5c280a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c30005950 tx=0x7f5c3000b410 comp rx=0 tx=0).stop 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 msgr2=0x7f5c4019e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 0x7f5c4019e350 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800d010 tx=0x7f5c38007870 comp rx=0 tx=0).stop 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 shutdown_connections 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c280a1290 0x7f5c280a3750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c4019ded0 0x7f5c4019e350 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c400697e0 0x7f5c40199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 --2- 172.21.15.16:0/1295598440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c401062a0 0x7f5c40199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 >> 172.21.15.16:0/1295598440 conn(0x7f5c40076f40 msgr2=0x7f5c4010c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 shutdown_connections 2024-05-08T23:22:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:30.869+0000 7f5c4536c700 1 -- 172.21.15.16:0/1295598440 wait complete. 2024-05-08T23:22:31.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:31 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4186987818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:31.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:31 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4186987818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:31.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:31 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4186987818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:22:31.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:22:31.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:22:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 -- 172.21.15.16:0/1996463628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c8068c90 msgr2=0x7f19c8105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 --2- 172.21.15.16:0/1996463628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c8068c90 0x7f19c8105b60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f19b80099c0 tx=0x7f19b8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 -- 172.21.15.16:0/1996463628 shutdown_connections 2024-05-08T23:22:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 --2- 172.21.15.16:0/1996463628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c8106290 0x7f19c810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 --2- 172.21.15.16:0/1996463628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c80682e0 0x7f19c80686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 --2- 172.21.15.16:0/1996463628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c8068c90 0x7f19c8105b60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 -- 172.21.15.16:0/1996463628 >> 172.21.15.16:0/1996463628 conn(0x7f19c8076f40 msgr2=0x7f19c8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 -- 172.21.15.16:0/1996463628 shutdown_connections 2024-05-08T23:22:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 -- 172.21.15.16:0/1996463628 wait complete. 2024-05-08T23:22:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 Processor -- start 2024-05-08T23:22:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.035+0000 7f19d005a700 1 -- start start 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19d005a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c80682e0 0x7f19c81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19d005a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c8106290 0x7f19c8197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19d005a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 0x7f19c819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19d005a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c819c640 con 0x7f19c819bbc0 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19d005a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c819c7b0 con 0x7f19c8106290 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19d005a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c819c970 con 0x7f19c80682e0 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 0x7f19c819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 0x7f19c819c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49598/0 (socket says 172.21.15.16:49598) 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1184391197 learned_addr learned my addr 172.21.15.16:0/1184391197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c8106290 0x7f19c8197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19cddf6700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c80682e0 0x7f19c81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1184391197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c80682e0 msgr2=0x7f19c81972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c80682e0 0x7f19c81972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1184391197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c8106290 msgr2=0x7f19c8197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c8106290 0x7f19c8197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1184391197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80095f0 con 0x7f19c819bbc0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.036+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 0x7f19c819c040 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f19c000e940 tx=0x7f19c000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.037+0000 7f19beffd700 1 -- 172.21.15.16:0/1184391197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c00097b0 con 0x7f19c819bbc0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.037+0000 7f19beffd700 1 -- 172.21.15.16:0/1184391197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19c000ee90 con 0x7f19c819bbc0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.037+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c81a3ed0 con 0x7f19c819bbc0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.037+0000 7f19beffd700 1 -- 172.21.15.16:0/1184391197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0005450 con 0x7f19c819bbc0 2024-05-08T23:22:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.037+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c81a43a0 con 0x7f19c819bbc0 2024-05-08T23:22:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.039+0000 7f19beffd700 1 -- 172.21.15.16:0/1184391197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f19c00055f0 con 0x7f19c819bbc0 2024-05-08T23:22:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.040+0000 7f19beffd700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f19b40a1310 0x7f19b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.040+0000 7f19cddf6700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f19b40a1310 0x7f19b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.040+0000 7f19beffd700 1 -- 172.21.15.16:0/1184391197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f19c0014070 con 0x7f19c819bbc0 2024-05-08T23:22:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.041+0000 7f19cddf6700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f19b40a1310 0x7f19b40a37d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009d10 tx=0x7f19c4009040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.041+0000 7f19dc214b80 1 Processor -- start 2024-05-08T23:22:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.041+0000 7f19dc214b80 1 -- start start 2024-05-08T23:22:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.041+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7ee9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.041+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee97a0 0x5574f7ee33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.041+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19dc214b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7e3a1d0 con 0x5574f7ee3910 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19dc214b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7e37b90 con 0x5574f7ee97a0 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19dc214b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7e33480 con 0x5574f7ee8e80 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49608/0 (socket says 172.21.15.16:49608) 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1903640679 learned_addr learned my addr 172.21.15.16:0/1903640679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19cddf6700 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7ee9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1903640679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 msgr2=0x5574f7ee9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7ee9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1903640679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee97a0 msgr2=0x5574f7ee33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee97a0 0x5574f7ee33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 -- 172.21.15.16:0/1903640679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f7cb4430 con 0x5574f7ee3910 2024-05-08T23:22:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee3d50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f19c00968a0 tx=0x7f19c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd5753adacb5ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19b3fff700 1 -- 172.21.15.16:0/1903640679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c009dc60 con 0x5574f7ee3910 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19b3fff700 1 -- 172.21.15.16:0/1903640679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19c00a22b0 con 0x5574f7ee3910 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.042+0000 7f19b3fff700 1 -- 172.21.15.16:0/1903640679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c009a290 con 0x5574f7ee3910 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 -- 172.21.15.16:0/1903640679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 msgr2=0x5574f7ee3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee3d50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f19c00968a0 tx=0x7f19c000ccf0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 -- 172.21.15.16:0/1903640679 shutdown_connections 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee3d50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7ee9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 --2- 172.21.15.16:0/1903640679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee97a0 0x5574f7ee33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 -- 172.21.15.16:0/1903640679 >> 172.21.15.16:0/1903640679 conn(0x5574f7e41290 msgr2=0x5574f7edde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 -- 172.21.15.16:0/1903640679 shutdown_connections 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.043+0000 7f19dc214b80 1 -- 172.21.15.16:0/1903640679 wait complete. 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 -- start start 2024-05-08T23:22:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 Processor -- start 2024-05-08T23:22:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7eecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7eed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 0x5574f7eed660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7dcf5e0 con 0x5574f7ee97a0 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7dcf280 con 0x5574f7ee3910 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7cb6720 con 0x5574f7ee8e80 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19dc214b80 10 client.0 _ll_register_callbacks cb 0x5574f7dd8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19ce5f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 0x5574f7eed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19ce5f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 0x5574f7eed660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49614/0 (socket says 172.21.15.16:49614) 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.044+0000 7f19ce5f7700 1 -- 172.21.15.16:0/2813481825 learned_addr learned my addr 172.21.15.16:0/2813481825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19cddf6700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7eecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7eed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19ce5f7700 1 -- 172.21.15.16:0/2813481825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 msgr2=0x5574f7eed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7eed120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19ce5f7700 1 -- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 msgr2=0x5574f7eecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7eecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19ce5f7700 1 -- 172.21.15.16:0/2813481825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c81a43a0 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7eed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 0x5574f7eed660 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f19c00ca9f0 tx=0x7f19c0016610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3dfb4635817f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19b17fa700 1 -- 172.21.15.16:0/2813481825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c009dce0 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19b17fa700 1 -- 172.21.15.16:0/2813481825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f19c009ae40 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574f7d8c2f0 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19b17fa700 1 -- 172.21.15.16:0/2813481825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19c0099530 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19dc214b80 10 client.16260 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5574f7c60be0 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.045+0000 7f19b17fa700 1 -- 172.21.15.16:0/2813481825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5574f7d8c2f0 con 0x5574f7ee97a0 2024-05-08T23:22:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19b17fa700 1 -- 172.21.15.16:0/2813481825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19c00a1e20 con 0x5574f7ee97a0 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19dc214b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19dc214b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19dc214b80 20 client.16260 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19dc214b80 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7efced0 0x5574f7eff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19dc214b80 4 client.16260 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574f7eb2890 con 0x5574f7efced0 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.046+0000 7f19cddf6700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7efced0 0x5574f7eff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19b17fa700 10 client.16260 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19cddf6700 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7efced0 0x5574f7eff2c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0efd3b1feaa9dd9 server_cookie=d04727e21dfd9375 in_seq=0 out_seq=0 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19b17fa700 1 -- 172.21.15.16:0/2813481825 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574f7eb2890 con 0x5574f7efced0 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19b17fa700 10 client.16260 handle_command_reply: tid=0 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 client.16260 shutdown 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 msgr2=0x5574f7eed660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 0x5574f7eed660 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f19c00ca9f0 tx=0x7f19c0016610 comp rx=0 tx=0).stop 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 shutdown_connections 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee97a0 0x5574f7eed660 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7efced0 0x5574f7eff2c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7eecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 --2- 172.21.15.16:0/2813481825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee8e80 0x5574f7eed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19b17fa700 0 client.16260 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.047+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 >> 172.21.15.16:0/2813481825 conn(0x5574f7e41290 msgr2=0x5574f7edd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.048+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 shutdown_connections 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.048+0000 7f19dc214b80 1 -- 172.21.15.16:0/2813481825 wait complete. 2024-05-08T23:22:32.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.048+0000 7f19dc214b80 20 client.16260 trim_cache size 0 max 16384 2024-05-08T23:22:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.066+0000 7f19dc214b80 1 Processor -- start 2024-05-08T23:22:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.066+0000 7f19dc214b80 1 -- start start 2024-05-08T23:22:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee8e80 0x5574f7ee8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7eddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19dc214b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7eef700 con 0x5574f7ee3910 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19dc214b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7eee070 con 0x5574f7ee8e80 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19dc214b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7f044d0 con 0x5574f7ee97a0 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49616/0 (socket says 172.21.15.16:49616) 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 -- 172.21.15.16:0/3440653482 learned_addr learned my addr 172.21.15.16:0/3440653482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee8e80 0x5574f7ee8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7eddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 -- 172.21.15.16:0/3440653482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 msgr2=0x5574f7eddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7eddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 -- 172.21.15.16:0/3440653482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee8e80 msgr2=0x5574f7ee8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee8e80 0x5574f7ee8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 -- 172.21.15.16:0/3440653482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f7eda560 con 0x5574f7ee3910 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19cddf6700 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee33d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009260 tx=0x7f19c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12dcf10483f195 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.067+0000 7f19bcff9700 1 -- 172.21.15.16:0/3440653482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c402a070 con 0x5574f7ee3910 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19bcff9700 1 -- 172.21.15.16:0/3440653482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19c4003e90 con 0x5574f7ee3910 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19bcff9700 1 -- 172.21.15.16:0/3440653482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c40169a0 con 0x5574f7ee3910 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 -- 172.21.15.16:0/3440653482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 msgr2=0x5574f7ee33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee33d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009260 tx=0x7f19c4015000 comp rx=0 tx=0).stop 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 -- 172.21.15.16:0/3440653482 shutdown_connections 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7eddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee3910 0x5574f7ee33d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3440653482 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee8e80 0x5574f7ee8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 -- 172.21.15.16:0/3440653482 >> 172.21.15.16:0/3440653482 conn(0x5574f7e41290 msgr2=0x5574f7ee7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 -- 172.21.15.16:0/3440653482 shutdown_connections 2024-05-08T23:22:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.068+0000 7f19dc214b80 1 -- 172.21.15.16:0/3440653482 wait complete. 2024-05-08T23:22:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 -- start start 2024-05-08T23:22:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 Processor -- start 2024-05-08T23:22:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7ef33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 0x5574f7ef38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7f577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7f57d00 con 0x5574f7ee8e80 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7f57e40 con 0x5574f7ee3910 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f7f58000 con 0x5574f7ee97a0 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19dc214b80 10 client.0 _ll_register_callbacks cb 0x5574f7dd8bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19cd5f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 0x5574f7ef38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19cd5f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 0x5574f7ef38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49618/0 (socket says 172.21.15.16:49618) 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19cddf6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7ef33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19cd5f5700 1 -- 172.21.15.16:0/3208325813 learned_addr learned my addr 172.21.15.16:0/3208325813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.069+0000 7f19ce5f7700 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7f577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19cd5f5700 1 -- 172.21.15.16:0/3208325813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 msgr2=0x5574f7f577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7f577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19cd5f5700 1 -- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 msgr2=0x5574f7ef33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7ef33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19cd5f5700 1 -- 172.21.15.16:0/3208325813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574f7df95e0 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19cd5f5700 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 0x5574f7ef38e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f19b80099c0 tx=0x7f19b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f2e1f319d54f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.070+0000 7f19b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19b17fa700 1 -- 172.21.15.16:0/3208325813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b801c030 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19c00a22b0 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19b17fa700 1 -- 172.21.15.16:0/3208325813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f19b800bc20 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19b17fa700 1 -- 172.21.15.16:0/3208325813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19b800bde0 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 10 client.16266 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5574f7cb8d50 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19b17fa700 1 -- 172.21.15.16:0/3208325813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19c00a22b0 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19b17fa700 1 -- 172.21.15.16:0/3208325813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19b8022040 con 0x5574f7ee8e80 2024-05-08T23:22:32.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 10 client.16266 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 20 client.16266 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7f59840 0x5574f7f5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 4 client.16266 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.072+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5574f7e2bf40 con 0x5574f7f59840 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.073+0000 7f19cddf6700 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7f59840 0x5574f7f5bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.073+0000 7f19cddf6700 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7f59840 0x5574f7f5bc60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b608ee053428de server_cookie=5ea143b660455deb in_seq=0 out_seq=0 2024-05-08T23:22:32.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.073+0000 7f19b17fa700 10 client.16266 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.074+0000 7f19b17fa700 1 -- 172.21.15.16:0/3208325813 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5574f7e2bf40 con 0x5574f7f59840 2024-05-08T23:22:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.074+0000 7f19b17fa700 10 client.16266 handle_command_reply: tid=0 2024-05-08T23:22:32.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.074+0000 7f19dc214b80 1 client.16266 shutdown 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.074+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 msgr2=0x5574f7ef38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.074+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 0x5574f7ef38e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f19b80099c0 tx=0x7f19b8005800 comp rx=0 tx=0).stop 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 shutdown_connections 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f7ee97a0 0x5574f7f577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f7f59840 0x5574f7f5bc60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f7ee3910 0x5574f7ef33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 --2- 172.21.15.16:0/3208325813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f7ee8e80 0x5574f7ef38e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19b17fa700 0 client.16266 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 >> 172.21.15.16:0/3208325813 conn(0x5574f7e41290 msgr2=0x5574f7eee410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 shutdown_connections 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 1 -- 172.21.15.16:0/3208325813 wait complete. 2024-05-08T23:22:32.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19dc214b80 20 client.16266 trim_cache size 0 max 16384 2024-05-08T23:22:32.079 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:22:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:22:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "b8d53bf5-9dcf-4145-9278-2d0427cb08ec", 2024-05-08T23:22:32.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:22:32.079 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f19b40a1310 msgr2=0x7f19b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19d005a700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f19b40a1310 0x7f19b40a37d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009d10 tx=0x7f19c4009040 comp rx=0 tx=0).stop 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 msgr2=0x7f19c819c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.075+0000 7f19d005a700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 0x7f19c819c040 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f19c000e940 tx=0x7f19c000ec50 comp rx=0 tx=0).stop 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 shutdown_connections 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f19b40a1310 0x7f19b40a37d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c819bbc0 0x7f19c819c040 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19c80682e0 0x7f19c81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 --2- 172.21.15.16:0/1184391197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19c8106290 0x7f19c8197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 >> 172.21.15.16:0/1184391197 conn(0x7f19c8076f40 msgr2=0x7f19c80fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 shutdown_connections 2024-05-08T23:22:32.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.076+0000 7f19d005a700 1 -- 172.21.15.16:0/1184391197 wait complete. 2024-05-08T23:22:32.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:22:32.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 -- 172.21.15.16:0/3322270572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c100ab0 msgr2=0x7fb50c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 --2- 172.21.15.16:0/3322270572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c100ab0 0x7fb50c10ccc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb50400b0d0 tx=0x7fb50400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 -- 172.21.15.16:0/3322270572 shutdown_connections 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 --2- 172.21.15.16:0/3322270572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c100ab0 0x7fb50c10ccc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 --2- 172.21.15.16:0/3322270572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c105f90 0x7fb50c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 --2- 172.21.15.16:0/3322270572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb50c1000f0 0x7fb50c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 -- 172.21.15.16:0/3322270572 >> 172.21.15.16:0/3322270572 conn(0x7fb50c076ba0 msgr2=0x7fb50c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 -- 172.21.15.16:0/3322270572 shutdown_connections 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.229+0000 7fb511e56700 1 -- 172.21.15.16:0/3322270572 wait complete. 2024-05-08T23:22:32.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 Processor -- start 2024-05-08T23:22:32.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 -- start start 2024-05-08T23:22:32.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 0x7fb50c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c100ab0 0x7fb50c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb50c105f90 0x7fb50c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c19c250 con 0x7fb50c1000f0 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c19c3c0 con 0x7fb50c100ab0 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb511e56700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c19c580 con 0x7fb50c105f90 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 0x7fb50c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 0x7fb50c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49640/0 (socket says 172.21.15.16:49640) 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.230+0000 7fb50b7fe700 1 -- 172.21.15.16:0/1472959216 learned_addr learned my addr 172.21.15.16:0/1472959216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50bfff700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb50c105f90 0x7fb50c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50affd700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c100ab0 0x7fb50c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50b7fe700 1 -- 172.21.15.16:0/1472959216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb50c105f90 msgr2=0x7fb50c19bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb50c105f90 0x7fb50c19bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50b7fe700 1 -- 172.21.15.16:0/1472959216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c100ab0 msgr2=0x7fb50c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c100ab0 0x7fb50c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50b7fe700 1 -- 172.21.15.16:0/1472959216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504009d00 con 0x7fb50c1000f0 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 0x7fb50c197330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d980 tx=0x7fb4fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb508ff9700 1 -- 172.21.15.16:0/1472959216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004020 con 0x7fb50c1000f0 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb508ff9700 1 -- 172.21.15.16:0/1472959216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc009730 con 0x7fb50c1000f0 2024-05-08T23:22:32.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c19c860 con 0x7fb50c1000f0 2024-05-08T23:22:32.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb508ff9700 1 -- 172.21.15.16:0/1472959216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc003a00 con 0x7fb50c1000f0 2024-05-08T23:22:32.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.231+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c1a40a0 con 0x7fb50c1000f0 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.234+0000 7fb508ff9700 1 -- 172.21.15.16:0/1472959216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4fc0098a0 con 0x7fb50c1000f0 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.235+0000 7fb508ff9700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb4f80a1290 0x7fb4f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.235+0000 7fb50affd700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb4f80a1290 0x7fb4f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.235+0000 7fb508ff9700 1 -- 172.21.15.16:0/1472959216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb4fc0ccce0 con 0x7fb50c1000f0 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.235+0000 7fb50affd700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb4f80a1290 0x7fb4f80a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005950 tx=0x7fb4f40058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 Processor -- start 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 -- start start 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5bd890 0x55d87c5bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c514150 con 0x55d87c5c2e00 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c511b10 con 0x55d87c5bd890 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb51e010b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c50d400 con 0x55d87c5c3720 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49654/0 (socket says 172.21.15.16:49654) 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 -- 172.21.15.16:0/2712306727 learned_addr learned my addr 172.21.15.16:0/2712306727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 -- 172.21.15.16:0/2712306727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 msgr2=0x55d87c5bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 -- 172.21.15.16:0/2712306727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5bd890 msgr2=0x55d87c5bdcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:22:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5bd890 0x55d87c5bdcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.236+0000 7fb50b7fe700 1 -- 172.21.15.16:0/2712306727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d87c38e3b0 con 0x55d87c5c2e00 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.237+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c31e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc095fb0 tx=0x7fb4fc099410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4eae7d920b66b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.237+0000 7fb501ffb700 1 -- 172.21.15.16:0/2712306727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc099ab0 con 0x55d87c5c2e00 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.237+0000 7fb501ffb700 1 -- 172.21.15.16:0/2712306727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc09d8e0 con 0x55d87c5c2e00 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.237+0000 7fb501ffb700 1 -- 172.21.15.16:0/2712306727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc098fa0 con 0x55d87c5c2e00 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.237+0000 7fb51e010b80 1 -- 172.21.15.16:0/2712306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 msgr2=0x55d87c5c31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.237+0000 7fb51e010b80 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c31e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc095fb0 tx=0x7fb4fc099410 comp rx=0 tx=0).stop 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 -- 172.21.15.16:0/2712306727 shutdown_connections 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5bd890 0x55d87c5bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c31e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 --2- 172.21.15.16:0/2712306727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 -- 172.21.15.16:0/2712306727 >> 172.21.15.16:0/2712306727 conn(0x55d87c51b210 msgr2=0x55d87c5b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 -- 172.21.15.16:0/2712306727 shutdown_connections 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 -- 172.21.15.16:0/2712306727 wait complete. 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 -- start start 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 Processor -- start 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.238+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb51e010b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c4a9560 con 0x55d87c5c2e00 2024-05-08T23:22:32.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb51e010b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c4a9200 con 0x55d87c5c3720 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb51e010b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c3906a0 con 0x55d87c5bd890 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb51e010b80 10 client.0 _ll_register_callbacks cb 0x55d87c4b2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49656/0 (socket says 172.21.15.16:49656) 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 -- 172.21.15.16:0/3783069954 learned_addr learned my addr 172.21.15.16:0/3783069954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50bfff700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 -- 172.21.15.16:0/3783069954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 msgr2=0x55d87c5c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 -- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 msgr2=0x55d87c5c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 -- 172.21.15.16:0/3783069954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c1a40a0 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb50affd700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c70f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400fd50 tx=0x7fb4f4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8701c8c8f1b640 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb4eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb4eaffd700 1 -- 172.21.15.16:0/3783069954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011e50 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d87c466270 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.239+0000 7fb4eaffd700 1 -- 172.21.15.16:0/3783069954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb4f401e450 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb4eaffd700 1 -- 172.21.15.16:0/3783069954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f401e610 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 10 client.16278 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d87c33ab60 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb4eaffd700 1 -- 172.21.15.16:0/3783069954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d87c466270 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb4eaffd700 1 -- 172.21.15.16:0/3783069954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4f4010830 con 0x55d87c5c2e00 2024-05-08T23:22:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 20 client.16278 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c5d7340 0x55d87c5d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 4 client.16278 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d87c58c810 con 0x55d87c5d7340 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.240+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c5d7340 0x55d87c5d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.241+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c5d7340 0x55d87c5d9760 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d60c856e4b497c server_cookie=70969a841c9bf1e3 in_seq=0 out_seq=0 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.241+0000 7fb4eaffd700 10 client.16278 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.241+0000 7fb4eaffd700 1 -- 172.21.15.16:0/3783069954 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d87c58c810 con 0x55d87c5d7340 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.241+0000 7fb4eaffd700 10 client.16278 handle_command_reply: tid=0 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 client.16278 shutdown 2024-05-08T23:22:32.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 msgr2=0x55d87c5c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c70f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400fd50 tx=0x7fb4f4011720 comp rx=0 tx=0).stop 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 shutdown_connections 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c5d7340 0x55d87c5d9760 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb4eaffd700 0 client.16278 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 --2- 172.21.15.16:0/3783069954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5c70f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 >> 172.21.15.16:0/3783069954 conn(0x55d87c51b210 msgr2=0x55d87c5b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 shutdown_connections 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 1 -- 172.21.15.16:0/3783069954 wait complete. 2024-05-08T23:22:32.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.242+0000 7fb51e010b80 20 client.16278 trim_cache size 0 max 16384 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 Processor -- start 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 -- start start 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 0x55d87c5bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c2e00 0x55d87c5b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c5c8220 con 0x55d87c5bd890 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c5b88e0 con 0x55d87c5c2e00 2024-05-08T23:22:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.260+0000 7fb51e010b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c5b4dc0 con 0x55d87c5c3720 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 0x55d87c5bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 0x55d87c5bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49660/0 (socket says 172.21.15.16:49660) 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 -- 172.21.15.16:0/454629941 learned_addr learned my addr 172.21.15.16:0/454629941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50affd700 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c2e00 0x55d87c5b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50bfff700 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 -- 172.21.15.16:0/454629941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 msgr2=0x55d87c5c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5c1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 -- 172.21.15.16:0/454629941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c2e00 msgr2=0x55d87c5b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c2e00 0x55d87c5b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 -- 172.21.15.16:0/454629941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d87c4d3560 con 0x55d87c5bd890 2024-05-08T23:22:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50affd700 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c2e00 0x55d87c5b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 0x55d87c5bd350 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0ca1a0 tx=0x7fb4fc095f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a401a68040c8f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb5027fc700 1 -- 172.21.15.16:0/454629941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0b2280 con 0x55d87c5bd890 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb5027fc700 1 -- 172.21.15.16:0/454629941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4fc0b23e0 con 0x55d87c5bd890 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.261+0000 7fb5027fc700 1 -- 172.21.15.16:0/454629941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc099f70 con 0x55d87c5bd890 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 -- 172.21.15.16:0/454629941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 msgr2=0x55d87c5bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 0x55d87c5bd350 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0ca1a0 tx=0x7fb4fc095f20 comp rx=0 tx=0).stop 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 -- 172.21.15.16:0/454629941 shutdown_connections 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5c3720 0x55d87c5c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5bd890 0x55d87c5bd350 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 --2- 172.21.15.16:0/454629941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c2e00 0x55d87c5b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 -- 172.21.15.16:0/454629941 >> 172.21.15.16:0/454629941 conn(0x55d87c51b210 msgr2=0x55d87c5c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 -- 172.21.15.16:0/454629941 shutdown_connections 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 -- 172.21.15.16:0/454629941 wait complete. 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.262+0000 7fb51e010b80 1 -- start start 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 Processor -- start 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c62c800 con 0x55d87c5c2e00 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c62c940 con 0x55d87c5c3720 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d87c62cab0 con 0x55d87c5bd890 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb51e010b80 10 client.0 _ll_register_callbacks cb 0x55d87c4b2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:22:32.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49670/0 (socket says 172.21.15.16:49670) 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 -- 172.21.15.16:0/629660728 learned_addr learned my addr 172.21.15.16:0/629660728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50bfff700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 -- 172.21.15.16:0/629660728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 msgr2=0x55d87c5cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 -- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 msgr2=0x55d87c5cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.263+0000 7fb50affd700 1 -- 172.21.15.16:0/629660728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d87c38e3b0 con 0x55d87c5c2e00 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb50affd700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5cd8d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f4010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c714d951b73ee2c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb4eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:22:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb4eaffd700 1 -- 172.21.15.16:0/629660728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011da0 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb4eaffd700 1 -- 172.21.15.16:0/629660728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb4f40103d0 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4fc09d8e0 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb4eaffd700 1 -- 172.21.15.16:0/629660728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f4010590 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 10 client.16284 fetch_fsmap learned FSMap version 25 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d87c392cd0 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb4eaffd700 1 -- 172.21.15.16:0/629660728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb4f4044020 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb4eaffd700 1 -- 172.21.15.16:0/629660728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb4f40047c0 con 0x55d87c5c2e00 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 20 client.16284 populate_metadata read hostname 'smithi016' 2024-05-08T23:22:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.264+0000 7fb51e010b80 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c62e210 0x55d87c630630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.265+0000 7fb51e010b80 4 client.16284 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.265+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d87c505ec0 con 0x55d87c62e210 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.265+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c62e210 0x55d87c630630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.265+0000 7fb50b7fe700 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c62e210 0x55d87c630630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447ffd34cb925b89 server_cookie=1ab35d418091d679 in_seq=0 out_seq=0 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.265+0000 7fb4eaffd700 10 client.16284 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb4eaffd700 1 -- 172.21.15.16:0/629660728 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d87c505ec0 con 0x55d87c62e210 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb4eaffd700 10 client.16284 handle_command_reply: tid=0 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 client.16284 shutdown 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 msgr2=0x55d87c5cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5cd8d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f4010fc0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 shutdown_connections 2024-05-08T23:22:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d87c62e210 0x55d87c630630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d87c5c3720 0x55d87c5cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d87c5bd890 0x55d87c5cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 --2- 172.21.15.16:0/629660728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d87c5c2e00 0x55d87c5cd8d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb4eaffd700 0 client.16284 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 >> 172.21.15.16:0/629660728 conn(0x55d87c51b210 msgr2=0x55d87c5c1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 shutdown_connections 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 1 -- 172.21.15.16:0/629660728 wait complete. 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb51e010b80 20 client.16284 trim_cache size 0 max 16384 2024-05-08T23:22:32.275 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "b8d53bf5-9dcf-4145-9278-2d0427cb08ec": { 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "b8d53bf5-9dcf-4145-9278-2d0427cb08ec", 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:22:32.276 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:22:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb4f80a1290 msgr2=0x7fb4f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb511e56700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb4f80a1290 0x7fb4f80a3750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005950 tx=0x7fb4f40058e0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 msgr2=0x7fb50c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:22:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.266+0000 7fb511e56700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 0x7fb50c197330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d980 tx=0x7fb4fc00dc90 comp rx=0 tx=0).stop 2024-05-08T23:22:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 shutdown_connections 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb4f80a1290 0x7fb4f80a3750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb50c105f90 0x7fb50c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb50c1000f0 0x7fb50c197330 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 --2- 172.21.15.16:0/1472959216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb50c100ab0 0x7fb50c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 >> 172.21.15.16:0/1472959216 conn(0x7fb50c076ba0 msgr2=0x7fb50c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 shutdown_connections 2024-05-08T23:22:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:22:32.267+0000 7fb511e56700 1 -- 172.21.15.16:0/1472959216 wait complete. 2024-05-08T23:22:32.279 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:32 smithi016 ceph-mon[42541]: pgmap v92: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:32.280 INFO:tasks.cephfs.filesystem:scrub status for tag:b8d53bf5-9dcf-4145-9278-2d0427cb08ec - {'path': '/', 'tag': 'b8d53bf5-9dcf-4145-9278-2d0427cb08ec', 'options': 'recursive,force'} 2024-05-08T23:22:32.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:32 smithi189 ceph-mon[46919]: pgmap v92: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:32.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:32 smithi155 ceph-mon[47105]: pgmap v92: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:33 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:22:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:33 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:33 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:22:33.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:33 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:33.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:33 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:22:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:33 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:33 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:22:33.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:33 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:33 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:22:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:33 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:33 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:22:33.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:33 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:22:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:34 smithi016 ceph-mon[42541]: pgmap v93: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:34 smithi189 ceph-mon[46919]: pgmap v93: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:34 smithi155 ceph-mon[47105]: pgmap v93: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:35.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:35 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:22:35.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:35 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:22:35.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:35 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:22:36.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:36 smithi189 ceph-mon[46919]: pgmap v94: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:36.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:36 smithi155 ceph-mon[47105]: pgmap v94: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:36.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:36 smithi016 ceph-mon[42541]: pgmap v94: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:38 smithi016 ceph-mon[42541]: pgmap v95: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:38.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:38 smithi189 ceph-mon[46919]: pgmap v95: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:38 smithi155 ceph-mon[47105]: pgmap v95: 129 pgs: 129 active+clean; 2.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:1/600: open fd = 3 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:2/600: open fd = 3 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:3/600: open fd = 3 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.158 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:4/600: open fd = 3 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:5/600: open fd = 3 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:6/600: open fd = 3 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:7/600: open fd = 3 2024-05-08T23:22:40.159 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:8/600: open fd = 3 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:9/600: open fd = 3 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:10/600: open fd = 3 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.160 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:11/600: open fd = 3 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:12/600: open fd = 3 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:13/600: open fd = 3 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:14/600: open fd = 3 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.161 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:15/600: open fd = 3 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:16/600: open fd = 3 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:17/600: open fd = 3 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.162 INFO:tasks.workunit.client.0.smithi016.stdout:18/600: open fd = 3 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:19/600: open fd = 3 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:20/600: open fd = 3 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:21/600: open fd = 3 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.163 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:22/600: open fd = 3 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:23/600: open fd = 3 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:24/600: open fd = 3 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:25/600: open fd = 3 2024-05-08T23:22:40.164 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:26/600: open fd = 3 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:27/600: open fd = 3 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:28/600: open fd = 3 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.165 INFO:tasks.workunit.client.0.smithi016.stdout:29/600: open fd = 3 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:30/600: open fd = 3 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:31/600: open fd = 3 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:32/600: open fd = 3 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.166 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:33/600: open fd = 3 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:34/600: open fd = 3 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:35/600: open fd = 3 2024-05-08T23:22:40.167 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:36/600: open fd = 3 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:37/600: open fd = 3 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:38/600: open fd = 3 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.168 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:39/600: open fd = 3 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:40/600: open fd = 3 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:41/600: open fd = 3 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:42/600: open fd = 3 2024-05-08T23:22:40.169 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:43/600: open fd = 3 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:44/600: open fd = 3 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:45/600: open fd = 3 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.170 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:46/600: open fd = 3 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:47/600: open fd = 3 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:48/600: open fd = 3 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:49/600: open fd = 3 2024-05-08T23:22:40.171 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:50/600: open fd = 3 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:51/600: open fd = 3 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:52/600: open fd = 3 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.172 INFO:tasks.workunit.client.0.smithi016.stdout:53/600: open fd = 3 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:54/600: open fd = 3 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:55/600: open fd = 3 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:56/600: open fd = 3 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.173 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:57/600: open fd = 3 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:58/600: open fd = 3 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:59/600: open fd = 3 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:60/600: open fd = 3 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.174 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:61/600: open fd = 3 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:62/600: open fd = 3 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:63/600: open fd = 3 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:22:40.175 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:22:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:40 smithi016 ceph-mon[42541]: pgmap v96: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-08T23:22:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:40 smithi189 ceph-mon[46919]: pgmap v96: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-08T23:22:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:40 smithi155 ceph-mon[47105]: pgmap v96: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-08T23:22:42.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:42 smithi016 ceph-mon[42541]: pgmap v97: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:42.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:42 smithi189 ceph-mon[46919]: pgmap v97: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:42 smithi155 ceph-mon[47105]: pgmap v97: 129 pgs: 129 active+clean; 2.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:43.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:43 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:22:43.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:43 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:22:43.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:43 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:22:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:44 smithi016 ceph-mon[42541]: pgmap v98: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:22:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:22:45.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:44 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:44 smithi189 ceph-mon[46919]: pgmap v98: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:22:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:22:45.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:44 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:44 smithi155 ceph-mon[47105]: pgmap v98: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:22:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:22:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:44 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:22:46.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:46 smithi016 ceph-mon[42541]: pgmap v99: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:46.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:46 smithi189 ceph-mon[46919]: pgmap v99: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:46 smithi155 ceph-mon[47105]: pgmap v99: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:48.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:48 smithi016 ceph-mon[42541]: pgmap v100: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:48.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:48 smithi189 ceph-mon[46919]: pgmap v100: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:48.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:48 smithi155 ceph-mon[47105]: pgmap v100: 129 pgs: 129 active+clean; 2.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:50 smithi016 ceph-mon[42541]: pgmap v101: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:50 smithi189 ceph-mon[46919]: pgmap v101: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:50.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:50 smithi155 ceph-mon[47105]: pgmap v101: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:52.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:52 smithi016 ceph-mon[42541]: pgmap v102: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:52 smithi189 ceph-mon[46919]: pgmap v102: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:52 smithi155 ceph-mon[47105]: pgmap v102: 129 pgs: 129 active+clean; 2.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:54.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:54 smithi016 ceph-mon[42541]: pgmap v103: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:54 smithi189 ceph-mon[46919]: pgmap v103: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:54 smithi155 ceph-mon[47105]: pgmap v103: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-05-08T23:22:56.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:56 smithi016 ceph-mon[42541]: pgmap v104: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:56.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:56 smithi189 ceph-mon[46919]: pgmap v104: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:56.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:56 smithi155 ceph-mon[47105]: pgmap v104: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:58.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:22:58 smithi016 ceph-mon[42541]: pgmap v105: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:58.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:22:58 smithi189 ceph-mon[46919]: pgmap v105: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:22:58.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:22:58 smithi155 ceph-mon[47105]: pgmap v105: 129 pgs: 129 active+clean; 2.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:23:00.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:00 smithi016 ceph-mon[42541]: pgmap v106: 129 pgs: 129 active+clean; 2.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-08T23:23:00.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:00 smithi189 ceph-mon[46919]: pgmap v106: 129 pgs: 129 active+clean; 2.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-08T23:23:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:00 smithi155 ceph-mon[47105]: pgmap v106: 129 pgs: 129 active+clean; 2.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-08T23:23:02.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:02 smithi016 ceph-mon[42541]: pgmap v107: 129 pgs: 129 active+clean; 2.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:23:02.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:02 smithi189 ceph-mon[46919]: pgmap v107: 129 pgs: 129 active+clean; 2.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 -- 172.21.15.16:0/4230685606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c068c90 msgr2=0x7f472c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 --2- 172.21.15.16:0/4230685606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c068c90 0x7f472c105b60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f471c0099c0 tx=0x7f471c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 -- 172.21.15.16:0/4230685606 shutdown_connections 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 --2- 172.21.15.16:0/4230685606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472c106290 0x7f472c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 --2- 172.21.15.16:0/4230685606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c0682e0 0x7f472c0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 --2- 172.21.15.16:0/4230685606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c068c90 0x7f472c105b60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 -- 172.21.15.16:0/4230685606 >> 172.21.15.16:0/4230685606 conn(0x7f472c076f40 msgr2=0x7f472c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 -- 172.21.15.16:0/4230685606 shutdown_connections 2024-05-08T23:23:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.431+0000 7f4732f68700 1 -- 172.21.15.16:0/4230685606 wait complete. 2024-05-08T23:23:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 Processor -- start 2024-05-08T23:23:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 -- start start 2024-05-08T23:23:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 0x7f472c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472c106290 0x7f472c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c19bba0 0x7f472c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c19c620 con 0x7f472c0682e0 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c19c790 con 0x7f472c19bba0 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4732f68700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c19c950 con 0x7f472c106290 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4730d04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 0x7f472c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4730d04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 0x7f472c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47382/0 (socket says 172.21.15.16:47382) 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.432+0000 7f4730d04700 1 -- 172.21.15.16:0/590219061 learned_addr learned my addr 172.21.15.16:0/590219061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4723fff700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472c106290 0x7f472c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4731505700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c19bba0 0x7f472c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4730d04700 1 -- 172.21.15.16:0/590219061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472c106290 msgr2=0x7f472c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4730d04700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472c106290 0x7f472c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4730d04700 1 -- 172.21.15.16:0/590219061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c19bba0 msgr2=0x7f472c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4730d04700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c19bba0 0x7f472c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4730d04700 1 -- 172.21.15.16:0/590219061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c0095f0 con 0x7f472c0682e0 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4731505700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c19bba0 0x7f472c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4730d04700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 0x7f472c1972c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9a0 tx=0x7f472800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4721ffb700 1 -- 172.21.15.16:0/590219061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728009790 con 0x7f472c0682e0 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4721ffb700 1 -- 172.21.15.16:0/590219061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f472800cbd0 con 0x7f472c0682e0 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c1a3eb0 con 0x7f472c0682e0 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.433+0000 7f4721ffb700 1 -- 172.21.15.16:0/590219061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728018680 con 0x7f472c0682e0 2024-05-08T23:23:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.434+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c1a42e0 con 0x7f472c0682e0 2024-05-08T23:23:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.436+0000 7f4721ffb700 1 -- 172.21.15.16:0/590219061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f472800cd40 con 0x7f472c0682e0 2024-05-08T23:23:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.437+0000 7f4721ffb700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47140a1310 0x7f47140a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.437+0000 7f4723fff700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47140a1310 0x7f47140a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.437+0000 7f4721ffb700 1 -- 172.21.15.16:0/590219061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4728014070 con 0x7f472c0682e0 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f4723fff700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47140a1310 0x7f47140a37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f471c009fd0 tx=0x7f471c019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 Processor -- start 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 -- start start 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d17695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d1763760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 0x5563d17640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d16b6150 con 0x5563d1763ca0 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d16b3b10 con 0x5563d1769b30 2024-05-08T23:23:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f473f122b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d16af400 con 0x5563d1769210 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f4731505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 0x5563d17640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f4731505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 0x5563d17640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47398/0 (socket says 172.21.15.16:47398) 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f4731505700 1 -- 172.21.15.16:0/247301795 learned_addr learned my addr 172.21.15.16:0/247301795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f4730d04700 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d17695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.438+0000 7f4723fff700 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d1763760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4731505700 1 -- 172.21.15.16:0/247301795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 msgr2=0x5563d17695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4731505700 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d17695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4731505700 1 -- 172.21.15.16:0/247301795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 msgr2=0x5563d1763760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4731505700 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d1763760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4731505700 1 -- 172.21.15.16:0/247301795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563d15303b0 con 0x5563d1763ca0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4731505700 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 0x5563d17640e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4724009440 tx=0x7f4724009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adc97a8455f7dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4712ffd700 1 -- 172.21.15.16:0/247301795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400e070 con 0x5563d1763ca0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4712ffd700 1 -- 172.21.15.16:0/247301795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f472400f040 con 0x5563d1763ca0 2024-05-08T23:23:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.439+0000 7f4712ffd700 1 -- 172.21.15.16:0/247301795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724015ac0 con 0x5563d1763ca0 2024-05-08T23:23:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 -- 172.21.15.16:0/247301795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 msgr2=0x5563d17640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 0x5563d17640e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4724009440 tx=0x7f4724009920 comp rx=0 tx=0).stop 2024-05-08T23:23:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 -- 172.21.15.16:0/247301795 shutdown_connections 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1763ca0 0x5563d17640e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d17695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 --2- 172.21.15.16:0/247301795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d1763760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 -- 172.21.15.16:0/247301795 >> 172.21.15.16:0/247301795 conn(0x5563d16bd210 msgr2=0x5563d1759df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 -- 172.21.15.16:0/247301795 shutdown_connections 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 -- 172.21.15.16:0/247301795 wait complete. 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 -- start start 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.440+0000 7f473f122b80 1 Processor -- start 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d176fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d176ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d176d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d164b560 con 0x5563d1769b30 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d164b200 con 0x5563d1763ca0 2024-05-08T23:23:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d15326a0 con 0x5563d1769210 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f473f122b80 10 client.0 _ll_register_callbacks cb 0x5563d1654b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d176d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d176d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47406/0 (socket says 172.21.15.16:47406) 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 -- 172.21.15.16:0/2121464543 learned_addr learned my addr 172.21.15.16:0/2121464543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4723fff700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d176ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4730d04700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d176fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 -- 172.21.15.16:0/2121464543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 msgr2=0x5563d176ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d176ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 -- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 msgr2=0x5563d176fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d176fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.441+0000 7f4731505700 1 -- 172.21.15.16:0/2121464543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47240090f0 con 0x5563d1769b30 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f4731505700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d176d570 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f472400b010 tx=0x7f4724016940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760c42e3e5c7c33d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f470bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f470bfff700 1 -- 172.21.15.16:0/2121464543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400e040 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f470bfff700 1 -- 172.21.15.16:0/2121464543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f4724003c00 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563d1608270 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f470bfff700 1 -- 172.21.15.16:0/2121464543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47240095e0 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 10 client.16296 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5563d14dcb60 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f470bfff700 1 -- 172.21.15.16:0/2121464543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5563d1608270 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f470bfff700 1 -- 172.21.15.16:0/2121464543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f472401c050 con 0x5563d1769b30 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 20 client.16296 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d177d3d0 0x5563d177f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 4 client.16296 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.442+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563d172e810 con 0x5563d177d3d0 2024-05-08T23:23:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.443+0000 7f4730d04700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d177d3d0 0x5563d177f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.443+0000 7f470bfff700 10 client.16296 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.443+0000 7f4730d04700 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d177d3d0 0x5563d177f7f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a566bd436619fe server_cookie=a30d3b5c6735f108 in_seq=0 out_seq=0 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f470bfff700 1 -- 172.21.15.16:0/2121464543 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563d172e810 con 0x5563d177d3d0 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f470bfff700 10 client.16296 handle_command_reply: tid=0 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 client.16296 shutdown 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 msgr2=0x5563d176d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d176d570 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f472400b010 tx=0x7f4724016940 comp rx=0 tx=0).stop 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 shutdown_connections 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d176d570 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d177d3d0 0x5563d177f7f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d176fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 --2- 172.21.15.16:0/2121464543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d176ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f470bfff700 0 client.16296 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 >> 172.21.15.16:0/2121464543 conn(0x5563d16bd210 msgr2=0x5563d17595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 shutdown_connections 2024-05-08T23:23:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 1 -- 172.21.15.16:0/2121464543 wait complete. 2024-05-08T23:23:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.444+0000 7f473f122b80 20 client.16296 trim_cache size 0 max 16384 2024-05-08T23:23:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.462+0000 7f473f122b80 1 Processor -- start 2024-05-08T23:23:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 -- start start 2024-05-08T23:23:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d1756210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d1764080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d1759d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d1756750 con 0x5563d1769b30 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d176b380 con 0x5563d1763ca0 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f473f122b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d17645c0 con 0x5563d1769210 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d1759d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d1759d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47420/0 (socket says 172.21.15.16:47420) 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 -- 172.21.15.16:0/11214142 learned_addr learned my addr 172.21.15.16:0/11214142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4723fff700 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d1764080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4730d04700 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d1756210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 -- 172.21.15.16:0/11214142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 msgr2=0x5563d1764080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d1764080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 -- 172.21.15.16:0/11214142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 msgr2=0x5563d1756210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d1756210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 -- 172.21.15.16:0/11214142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563d1675560 con 0x5563d1769b30 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4730d04700 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d1756210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.463+0000 7f4731505700 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d1759d20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4724000f80 tx=0x7f4724009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d873483391e21e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f47137fe700 1 -- 172.21.15.16:0/11214142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724016ea0 con 0x5563d1769b30 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f47137fe700 1 -- 172.21.15.16:0/11214142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f472401f790 con 0x5563d1769b30 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f47137fe700 1 -- 172.21.15.16:0/11214142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472401e3a0 con 0x5563d1769b30 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 -- 172.21.15.16:0/11214142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 msgr2=0x5563d1759d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d1759d20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4724000f80 tx=0x7f4724009920 comp rx=0 tx=0).stop 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 -- 172.21.15.16:0/11214142 shutdown_connections 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769b30 0x5563d1759d20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1763ca0 0x5563d1756210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 --2- 172.21.15.16:0/11214142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1769210 0x5563d1764080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 -- 172.21.15.16:0/11214142 >> 172.21.15.16:0/11214142 conn(0x5563d16bd210 msgr2=0x5563d176c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 -- 172.21.15.16:0/11214142 shutdown_connections 2024-05-08T23:23:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.464+0000 7f473f122b80 1 -- 172.21.15.16:0/11214142 wait complete. 2024-05-08T23:23:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 -- start start 2024-05-08T23:23:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 Processor -- start 2024-05-08T23:23:02.468 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:02 smithi155 ceph-mon[47105]: pgmap v107: 129 pgs: 129 active+clean; 2.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-05-08T23:23:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1763ca0 0x5563d1770760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 0x5563d1770ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d17d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d17d2e00 con 0x5563d1769210 2024-05-08T23:23:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d17d2f40 con 0x5563d1769b30 2024-05-08T23:23:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563d17d30d0 con 0x5563d1763ca0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f473f122b80 10 client.0 _ll_register_callbacks cb 0x5563d1654b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f4723fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 0x5563d1770ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f4723fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 0x5563d1770ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47436/0 (socket says 172.21.15.16:47436) 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f4723fff700 1 -- 172.21.15.16:0/3963006959 learned_addr learned my addr 172.21.15.16:0/3963006959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.465+0000 7f4730d04700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1763ca0 0x5563d1770760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4731505700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d17d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4723fff700 1 -- 172.21.15.16:0/3963006959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1763ca0 msgr2=0x5563d1770760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4723fff700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1763ca0 0x5563d1770760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4723fff700 1 -- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 msgr2=0x5563d17d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4723fff700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d17d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4723fff700 1 -- 172.21.15.16:0/3963006959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563d14dcb60 con 0x5563d1769210 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f4723fff700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 0x5563d1770ca0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f471c015000 tx=0x7f471c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6099ae5090dc3b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f470bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f470bfff700 1 -- 172.21.15.16:0/3963006959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c031640 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f470bfff700 1 -- 172.21.15.16:0/3963006959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f471c0317a0 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f472400f040 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f470bfff700 1 -- 172.21.15.16:0/3963006959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471c031960 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f473f122b80 10 client.16302 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.466+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5563d1534cd0 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f470bfff700 1 -- 172.21.15.16:0/3963006959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f472400f040 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f470bfff700 1 -- 172.21.15.16:0/3963006959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f471c037020 con 0x5563d1769210 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f473f122b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f473f122b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f473f122b80 20 client.16302 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f473f122b80 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d17d4960 0x5563d17d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f473f122b80 4 client.16302 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5563d16a7ec0 con 0x5563d17d4960 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.467+0000 7f4730d04700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d17d4960 0x5563d17d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f4730d04700 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d17d4960 0x5563d17d6d80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b8b17a6835ba64 server_cookie=b66823198412fa2a in_seq=0 out_seq=0 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f470bfff700 10 client.16302 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f470bfff700 1 -- 172.21.15.16:0/3963006959 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5563d16a7ec0 con 0x5563d17d4960 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f470bfff700 10 client.16302 handle_command_reply: tid=0 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 client.16302 shutdown 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 msgr2=0x5563d1770ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 0x5563d1770ca0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f471c015000 tx=0x7f471c009310 comp rx=0 tx=0).stop 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 shutdown_connections 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5563d17d4960 0x5563d17d6d80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5563d1769b30 0x5563d17d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563d1763ca0 0x5563d1770760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f470bfff700 0 client.16302 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 --2- 172.21.15.16:0/3963006959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563d1769210 0x5563d1770ca0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 >> 172.21.15.16:0/3963006959 conn(0x5563d16bd210 msgr2=0x5563d1766f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 shutdown_connections 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 1 -- 172.21.15.16:0/3963006959 wait complete. 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.468+0000 7f473f122b80 20 client.16302 trim_cache size 0 max 16384 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:23:02.476 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:23:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47140a1310 msgr2=0x7f47140a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47140a1310 0x7f47140a37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f471c009fd0 tx=0x7f471c019040 comp rx=0 tx=0).stop 2024-05-08T23:23:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 msgr2=0x7f472c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 0x7f472c1972c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9a0 tx=0x7f472800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 shutdown_connections 2024-05-08T23:23:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47140a1310 0x7f47140a37d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472c19bba0 0x7f472c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c0682e0 0x7f472c1972c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 --2- 172.21.15.16:0/590219061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472c106290 0x7f472c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 >> 172.21.15.16:0/590219061 conn(0x7f472c076f40 msgr2=0x7f472c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 shutdown_connections 2024-05-08T23:23:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.469+0000 7f4732f68700 1 -- 172.21.15.16:0/590219061 wait complete. 2024-05-08T23:23:02.483 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:23:02.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 -- 172.21.15.16:0/4182846272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb501008f0 msgr2=0x7fcb50104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/4182846272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb501008f0 0x7fcb50104930 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009990 tx=0x7fcb40009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 -- 172.21.15.16:0/4182846272 shutdown_connections 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/4182846272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb50105060 0x7fcb5010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/4182846272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb500fff40 0x7fcb50100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/4182846272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb501008f0 0x7fcb50104930 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 -- 172.21.15.16:0/4182846272 >> 172.21.15.16:0/4182846272 conn(0x7fcb500fb7e0 msgr2=0x7fcb500fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 -- 172.21.15.16:0/4182846272 shutdown_connections 2024-05-08T23:23:02.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.620+0000 7fcb57cb8700 1 -- 172.21.15.16:0/4182846272 wait complete. 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 Processor -- start 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 -- start start 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb500fff40 0x7fcb50197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 0x7fcb50197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5019bbe0 0x7fcb5019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019c660 con 0x7fcb50105060 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019c7d0 con 0x7fcb5019bbe0 2024-05-08T23:23:02.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb57cb8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019c990 con 0x7fcb500fff40 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb55253700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 0x7fcb50197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb55253700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 0x7fcb50197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47450/0 (socket says 172.21.15.16:47450) 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.621+0000 7fcb55253700 1 -- 172.21.15.16:0/2151802269 learned_addr learned my addr 172.21.15.16:0/2151802269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55a54700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb500fff40 0x7fcb50197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb56255700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5019bbe0 0x7fcb5019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55253700 1 -- 172.21.15.16:0/2151802269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb500fff40 msgr2=0x7fcb50197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55253700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb500fff40 0x7fcb50197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55253700 1 -- 172.21.15.16:0/2151802269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5019bbe0 msgr2=0x7fcb5019c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55253700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5019bbe0 0x7fcb5019c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55253700 1 -- 172.21.15.16:0/2151802269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400095f0 con 0x7fcb50105060 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb56255700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5019bbe0 0x7fcb5019c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb55253700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 0x7fcb50197840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcb40004730 tx=0x7fcb40009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4001d070 con 0x7fcb50105060 2024-05-08T23:23:02.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb40004ca0 con 0x7fcb50105060 2024-05-08T23:23:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000f730 con 0x7fcb50105060 2024-05-08T23:23:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb501a3e90 con 0x7fcb50105060 2024-05-08T23:23:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.622+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501a4200 con 0x7fcb50105060 2024-05-08T23:23:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.623+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb50109340 con 0x7fcb50105060 2024-05-08T23:23:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.625+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb40004810 con 0x7fcb50105060 2024-05-08T23:23:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.625+0000 7fcb46ffd700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb3c0a1290 0x7fcb3c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.625+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcb400ce2f0 con 0x7fcb50105060 2024-05-08T23:23:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.625+0000 7fcb55a54700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb3c0a1290 0x7fcb3c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.626+0000 7fcb55a54700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb3c0a1290 0x7fcb3c0a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005950 tx=0x7fcb4c00b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.629+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb40097490 con 0x7fcb50105060 2024-05-08T23:23:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.896+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb50198600 con 0x7fcb50105060 2024-05-08T23:23:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.897+0000 7fcb46ffd700 1 -- 172.21.15.16:0/2151802269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fcb4009aab0 con 0x7fcb50105060 2024-05-08T23:23:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:23:02.897 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb3c0a1290 msgr2=0x7fcb3c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb3c0a1290 0x7fcb3c0a3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c005950 tx=0x7fcb4c00b410 comp rx=0 tx=0).stop 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 msgr2=0x7fcb50197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 0x7fcb50197840 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcb40004730 tx=0x7fcb40009580 comp rx=0 tx=0).stop 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 shutdown_connections 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcb3c0a1290 0x7fcb3c0a3750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcb5019bbe0 0x7fcb5019c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb500fff40 0x7fcb50197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 --2- 172.21.15.16:0/2151802269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb50105060 0x7fcb50197840 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 >> 172.21.15.16:0/2151802269 conn(0x7fcb500fb7e0 msgr2=0x7fcb50068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 shutdown_connections 2024-05-08T23:23:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:02.899+0000 7fcb57cb8700 1 -- 172.21.15.16:0/2151802269 wait complete. 2024-05-08T23:23:02.901 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:23:02.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:23:03.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 -- 172.21.15.16:0/1015674134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c100ab0 msgr2=0x7fdc6c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 --2- 172.21.15.16:0/1015674134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c10ccc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400b0d0 tx=0x7fdc6400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 -- 172.21.15.16:0/1015674134 shutdown_connections 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 --2- 172.21.15.16:0/1015674134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c10ccc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 --2- 172.21.15.16:0/1015674134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c105f90 0x7fdc6c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 --2- 172.21.15.16:0/1015674134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 -- 172.21.15.16:0/1015674134 >> 172.21.15.16:0/1015674134 conn(0x7fdc6c076ba0 msgr2=0x7fdc6c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 -- 172.21.15.16:0/1015674134 shutdown_connections 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.053+0000 7fdc733a5700 1 -- 172.21.15.16:0/1015674134 wait complete. 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 Processor -- start 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 -- start start 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc6c105f90 0x7fdc6c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c19c270 con 0x7fdc6c1000f0 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c19c3e0 con 0x7fdc6c100ab0 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc733a5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c19c5a0 con 0x7fdc6c105f90 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.054+0000 7fdc71141700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47472/0 (socket says 172.21.15.16:47472) 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 -- 172.21.15.16:0/3768878277 learned_addr learned my addr 172.21.15.16:0/3768878277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc70940700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71942700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc6c105f90 0x7fdc6c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 -- 172.21.15.16:0/3768878277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc6c105f90 msgr2=0x7fdc6c19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc6c105f90 0x7fdc6c19bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 -- 172.21.15.16:0/3768878277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c100ab0 msgr2=0x7fdc6c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 -- 172.21.15.16:0/3768878277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc64009d00 con 0x7fdc6c1000f0 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc70940700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc71141700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c1973a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00d980 tx=0x7fdc5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc627fc700 1 -- 172.21.15.16:0/3768878277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004020 con 0x7fdc6c1000f0 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c19c880 con 0x7fdc6c1000f0 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc627fc700 1 -- 172.21.15.16:0/3768878277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc5c009730 con 0x7fdc6c1000f0 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc627fc700 1 -- 172.21.15.16:0/3768878277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c003a00 con 0x7fdc6c1000f0 2024-05-08T23:23:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.055+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c1a40c0 con 0x7fdc6c1000f0 2024-05-08T23:23:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.058+0000 7fdc627fc700 1 -- 172.21.15.16:0/3768878277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdc5c0098a0 con 0x7fdc6c1000f0 2024-05-08T23:23:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.059+0000 7fdc627fc700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdc580a1290 0x7fdc580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.059+0000 7fdc627fc700 1 -- 172.21.15.16:0/3768878277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fdc5c0ccce0 con 0x7fdc6c1000f0 2024-05-08T23:23:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.059+0000 7fdc70940700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdc580a1290 0x7fdc580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc70940700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdc580a1290 0x7fdc580a3750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c1988a0 tx=0x7fdc68009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 Processor -- start 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 -- start start 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679ec890 0x55f9679eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f967943150 con 0x55f9679f2720 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f967940b10 con 0x55f9679ec890 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f96793c400 con 0x55f9679f1e00 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc70940700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc70940700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47488/0 (socket says 172.21.15.16:47488) 2024-05-08T23:23:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc70940700 1 -- 172.21.15.16:0/2473117827 learned_addr learned my addr 172.21.15.16:0/2473117827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc71942700 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679ec890 0x55f9679eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.060+0000 7fdc71141700 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc70940700 1 -- 172.21.15.16:0/2473117827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 msgr2=0x55f9679f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc70940700 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc70940700 1 -- 172.21.15.16:0/2473117827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679ec890 msgr2=0x55f9679eccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc70940700 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679ec890 0x55f9679eccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc70940700 1 -- 172.21.15.16:0/2473117827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9677bd3b0 con 0x55f9679f2720 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc71141700 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc70940700 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679ec350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdc68011f60 tx=0x7fdc68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da13980d107c0af0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc577fe700 1 -- 172.21.15.16:0/2473117827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68003680 con 0x55f9679f2720 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc577fe700 1 -- 172.21.15.16:0/2473117827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc68003c70 con 0x55f9679f2720 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.061+0000 7fdc577fe700 1 -- 172.21.15.16:0/2473117827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68012bf0 con 0x55f9679f2720 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2473117827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 msgr2=0x55f9679ec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679ec350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdc68011f60 tx=0x7fdc68014000 comp rx=0 tx=0).stop 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2473117827 shutdown_connections 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679ec890 0x55f9679eccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2473117827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679ec350 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2473117827 >> 172.21.15.16:0/2473117827 conn(0x55f96794a210 msgr2=0x55f9679e6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2473117827 shutdown_connections 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2473117827 wait complete. 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 -- start start 2024-05-08T23:23:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.062+0000 7fdc7f55fb80 1 Processor -- start 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f9679f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f2720 0x55f9679f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9678d8560 con 0x55f9679ec890 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9678d8200 con 0x55f9679f1e00 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9677bf6a0 con 0x55f9679f2720 2024-05-08T23:23:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc7f55fb80 10 client.0 _ll_register_callbacks cb 0x55f9678e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47494/0 (socket says 172.21.15.16:47494) 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 -- 172.21.15.16:0/4167325651 learned_addr learned my addr 172.21.15.16:0/4167325651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc70940700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f9679f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71942700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f2720 0x55f9679f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 -- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f2720 msgr2=0x55f9679f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f2720 0x55f9679f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 -- 172.21.15.16:0/4167325651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 msgr2=0x55f9679f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f9679f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.063+0000 7fdc71141700 1 -- 172.21.15.16:0/4167325651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc68011c60 con 0x55f9679ec890 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc71141700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679f5a50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0997b0 tx=0x7fdc5c09a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33e31f1038c7de5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc54ff9700 1 -- 172.21.15.16:0/4167325651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0b2410 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc54ff9700 1 -- 172.21.15.16:0/4167325651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fdc5c023500 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f967895270 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc54ff9700 1 -- 172.21.15.16:0/4167325651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5c0a1f30 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 10 client.16320 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f967769b60 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc54ff9700 1 -- 172.21.15.16:0/4167325651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f967895270 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc54ff9700 1 -- 172.21.15.16:0/4167325651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc5c084050 con 0x55f9679ec890 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 10 client.16320 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 10 client.16320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 20 client.16320 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a06230 0x55f967a08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 4 client.16320 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.064+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9679bb810 con 0x55f967a06230 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.065+0000 7fdc70940700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a06230 0x55f967a08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.065+0000 7fdc54ff9700 10 client.16320 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.065+0000 7fdc70940700 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a06230 0x55f967a08620 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46fa18faff14f8f5 server_cookie=d8c3ba76d1023a3c in_seq=0 out_seq=0 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc54ff9700 1 -- 172.21.15.16:0/4167325651 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9679bb810 con 0x55f967a06230 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc54ff9700 10 client.16320 handle_command_reply: tid=0 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 client.16320 shutdown 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 msgr2=0x55f9679f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679f5a50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0997b0 tx=0x7fdc5c09a1b0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 shutdown_connections 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f2720 0x55f9679f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679f5a50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a06230 0x55f967a08620 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/4167325651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f9679f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 >> 172.21.15.16:0/4167325651 conn(0x55f96794a210 msgr2=0x55f9679e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc54ff9700 0 client.16320 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 shutdown_connections 2024-05-08T23:23:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/4167325651 wait complete. 2024-05-08T23:23:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.066+0000 7fdc7f55fb80 20 client.16320 trim_cache size 0 max 16384 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 Processor -- start 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 -- start start 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679ec890 0x55f9679ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f967a08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9679e1800 con 0x55f9679f2720 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9679e8a00 con 0x55f9679f1e00 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f967a05fc0 con 0x55f9679ec890 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc71942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc71942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47506/0 (socket says 172.21.15.16:47506) 2024-05-08T23:23:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc71942700 1 -- 172.21.15.16:0/1940945223 learned_addr learned my addr 172.21.15.16:0/1940945223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc70940700 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f967a08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.081+0000 7fdc71141700 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679ec890 0x55f9679ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc71942700 1 -- 172.21.15.16:0/1940945223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679ec890 msgr2=0x55f9679ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc71942700 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679ec890 0x55f9679ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc71942700 1 -- 172.21.15.16:0/1940945223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 msgr2=0x55f967a08620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc71942700 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f967a08620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc71942700 1 -- 172.21.15.16:0/1940945223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f967902560 con 0x55f9679f2720 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc71942700 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679e6d20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009500 tx=0x7fdc64009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c8f7e66cf50257 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc57fff700 1 -- 172.21.15.16:0/1940945223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6400e040 con 0x55f9679f2720 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc57fff700 1 -- 172.21.15.16:0/1940945223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc6400bda0 con 0x55f9679f2720 2024-05-08T23:23:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.082+0000 7fdc57fff700 1 -- 172.21.15.16:0/1940945223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64003de0 con 0x55f9679f2720 2024-05-08T23:23:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/1940945223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 msgr2=0x55f9679e6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679e6d20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009500 tx=0x7fdc64009530 comp rx=0 tx=0).stop 2024-05-08T23:23:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/1940945223 shutdown_connections 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679f2720 0x55f9679e6d20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679ec890 0x55f9679ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/1940945223 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f1e00 0x55f967a08620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/1940945223 >> 172.21.15.16:0/1940945223 conn(0x55f96794a210 msgr2=0x55f9679ecc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/1940945223 shutdown_connections 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/1940945223 wait complete. 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 -- start start 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.083+0000 7fdc7f55fb80 1 Processor -- start 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f2720 0x55f9679ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9679ef620 con 0x55f9679ec890 2024-05-08T23:23:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9679edee0 con 0x55f9679f2720 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9679ee0a0 con 0x55f9679f1e00 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc7f55fb80 10 client.0 _ll_register_callbacks cb 0x55f9678e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679fd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47514/0 (socket says 172.21.15.16:47514) 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 -- 172.21.15.16:0/2788697284 learned_addr learned my addr 172.21.15.16:0/2788697284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc70940700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71942700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f2720 0x55f9679ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 -- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 msgr2=0x55f9679eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 -- 172.21.15.16:0/2788697284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f2720 msgr2=0x55f9679ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f2720 0x55f9679ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.084+0000 7fdc71141700 1 -- 172.21.15.16:0/2788697284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c10d550 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc71141700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679fd640 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c099a40 tx=0x7fdc5c0b2440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acc5a8f916a5b97 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc54ff9700 1 -- 172.21.15.16:0/2788697284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c023500 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc54ff9700 1 -- 172.21.15.16:0/2788697284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fdc5c0a1f30 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc68003c70 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc54ff9700 1 -- 172.21.15.16:0/2788697284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5c099d10 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 10 client.16326 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f9677c1cd0 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc54ff9700 1 -- 172.21.15.16:0/2788697284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc68003c70 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc54ff9700 1 -- 172.21.15.16:0/2788697284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc5c0ca370 con 0x55f9679ec890 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 10 client.16326 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 10 client.16326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 20 client.16326 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a5cec0 0x55f967a5f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 4 client.16326 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.085+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f967934ec0 con 0x55f967a5cec0 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.086+0000 7fdc70940700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a5cec0 0x55f967a5f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.086+0000 7fdc54ff9700 10 client.16326 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.086+0000 7fdc70940700 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a5cec0 0x55f967a5f2b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4885405ba674bdef server_cookie=f1a23b7ddd5a766a in_seq=0 out_seq=0 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.086+0000 7fdc54ff9700 1 -- 172.21.15.16:0/2788697284 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f967934ec0 con 0x55f967a5cec0 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.086+0000 7fdc54ff9700 10 client.16326 handle_command_reply: tid=0 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.086+0000 7fdc7f55fb80 1 client.16326 shutdown 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 msgr2=0x55f9679fd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679fd640 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c099a40 tx=0x7fdc5c0b2440 comp rx=0 tx=0).stop 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 shutdown_connections 2024-05-08T23:23:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55f967a5cec0 0x55f967a5f2b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55f9679f2720 0x55f9679ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9679ec890 0x55f9679fd640 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc54ff9700 0 client.16326 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 --2- 172.21.15.16:0/2788697284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9679f1e00 0x55f9679eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 >> 172.21.15.16:0/2788697284 conn(0x55f96794a210 msgr2=0x55f9679f4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 shutdown_connections 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 1 -- 172.21.15.16:0/2788697284 wait complete. 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc7f55fb80 20 client.16326 trim_cache size 0 max 16384 2024-05-08T23:23:03.093 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:23:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdc580a1290 msgr2=0x7fdc580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdc580a1290 0x7fdc580a3750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c1988a0 tx=0x7fdc68009260 comp rx=0 tx=0).stop 2024-05-08T23:23:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 msgr2=0x7fdc6c1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c1973a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00d980 tx=0x7fdc5c00dc90 comp rx=0 tx=0).stop 2024-05-08T23:23:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 shutdown_connections 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdc580a1290 0x7fdc580a3750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc6c105f90 0x7fdc6c19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc6c1000f0 0x7fdc6c1973a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.087+0000 7fdc733a5700 1 --2- 172.21.15.16:0/3768878277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc6c100ab0 0x7fdc6c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.088+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 >> 172.21.15.16:0/3768878277 conn(0x7fdc6c076ba0 msgr2=0x7fdc6c069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.088+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 shutdown_connections 2024-05-08T23:23:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.088+0000 7fdc733a5700 1 -- 172.21.15.16:0/3768878277 wait complete. 2024-05-08T23:23:03.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 -- 172.21.15.16:0/2307244716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ffe00 msgr2=0x7f75e4100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 --2- 172.21.15.16:0/2307244716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ffe00 0x7f75e4100280 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009990 tx=0x7f75d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 -- 172.21.15.16:0/2307244716 shutdown_connections 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 --2- 172.21.15.16:0/2307244716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e41007c0 0x7f75e410cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 --2- 172.21.15.16:0/2307244716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75e4105ec0 0x7f75e41062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 --2- 172.21.15.16:0/2307244716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ffe00 0x7f75e4100280 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 -- 172.21.15.16:0/2307244716 >> 172.21.15.16:0/2307244716 conn(0x7f75e4076f10 msgr2=0x7f75e4077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 -- 172.21.15.16:0/2307244716 shutdown_connections 2024-05-08T23:23:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.240+0000 7f75e9334700 1 -- 172.21.15.16:0/2307244716 wait complete. 2024-05-08T23:23:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 Processor -- start 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 -- start start 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e40ffe00 0x7f75e41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75e41007c0 0x7f75e4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 0x7f75e41a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4198e70 con 0x7f75e4105ec0 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e41a3680 con 0x7f75e40ffe00 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e9334700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e41a37c0 con 0x7f75e41007c0 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 0x7f75e41a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 0x7f75e41a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47540/0 (socket says 172.21.15.16:47540) 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e37fe700 1 -- 172.21.15.16:0/1464343554 learned_addr learned my addr 172.21.15.16:0/1464343554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.241+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e40ffe00 0x7f75e41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e27fc700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75e41007c0 0x7f75e4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e37fe700 1 -- 172.21.15.16:0/1464343554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75e41007c0 msgr2=0x7f75e4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e37fe700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75e41007c0 0x7f75e4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e37fe700 1 -- 172.21.15.16:0/1464343554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e40ffe00 msgr2=0x7f75e41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e37fe700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e40ffe00 0x7f75e41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e37fe700 1 -- 172.21.15.16:0/1464343554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80095f0 con 0x7f75e4105ec0 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e37fe700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 0x7f75e41a3140 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f75dc007ee0 tx=0x7f75dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75cbfff700 1 -- 172.21.15.16:0/1464343554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc017070 con 0x7f75e4105ec0 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75cbfff700 1 -- 172.21.15.16:0/1464343554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75dc00f040 con 0x7f75e4105ec0 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e41a3a50 con 0x7f75e4105ec0 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75cbfff700 1 -- 172.21.15.16:0/1464343554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc005710 con 0x7f75e4105ec0 2024-05-08T23:23:03.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.242+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e41a3f10 con 0x7f75e4105ec0 2024-05-08T23:23:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.245+0000 7f75cbfff700 1 -- 172.21.15.16:0/1464343554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f75dc005870 con 0x7f75e4105ec0 2024-05-08T23:23:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.246+0000 7f75cbfff700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f75cc0a1290 0x7f75cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.246+0000 7f75cbfff700 1 -- 172.21.15.16:0/1464343554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f75dc0cc890 con 0x7f75e4105ec0 2024-05-08T23:23:03.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.246+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f75cc0a1290 0x7f75cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f75cc0a1290 0x7f75cc0a3750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005950 tx=0x7f75d400b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 Processor -- start 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 -- start start 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x55559237f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592379350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555592379890 0x555592379cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555922d0150 con 0x55559237f720 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555922cdb10 con 0x55559237ee00 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555922c9400 con 0x555592379890 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592379350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592379350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47552/0 (socket says 172.21.15.16:47552) 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 -- 172.21.15.16:0/2450756984 learned_addr learned my addr 172.21.15.16:0/2450756984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x55559237f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e37fe700 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555592379890 0x555592379cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 -- 172.21.15.16:0/2450756984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555592379890 msgr2=0x555592379cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555592379890 0x555592379cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 -- 172.21.15.16:0/2450756984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 msgr2=0x55559237f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x55559237f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e27fc700 1 -- 172.21.15.16:0/2450756984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559214a3b0 con 0x55559237f720 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.247+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x55559237f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.248+0000 7f75e27fc700 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592379350 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f75d800b5c0 tx=0x7f75d8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4b29d61a24a238 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.248+0000 7f75c97fa700 1 -- 172.21.15.16:0/2450756984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8005240 con 0x55559237f720 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.248+0000 7f75c97fa700 1 -- 172.21.15.16:0/2450756984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d801e040 con 0x55559237f720 2024-05-08T23:23:03.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.248+0000 7f75c97fa700 1 -- 172.21.15.16:0/2450756984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8004700 con 0x55559237f720 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.248+0000 7f75f54eeb80 1 -- 172.21.15.16:0/2450756984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 msgr2=0x555592379350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.248+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592379350 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f75d800b5c0 tx=0x7f75d8005e80 comp rx=0 tx=0).stop 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 -- 172.21.15.16:0/2450756984 shutdown_connections 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555592379890 0x555592379cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x55559237f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/2450756984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592379350 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 -- 172.21.15.16:0/2450756984 >> 172.21.15.16:0/2450756984 conn(0x5555922d7210 msgr2=0x555592373df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 -- 172.21.15.16:0/2450756984 shutdown_connections 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 -- 172.21.15.16:0/2450756984 wait complete. 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 -- start start 2024-05-08T23:23:03.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.249+0000 7f75f54eeb80 1 Processor -- start 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 0x555592382ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x555592383010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237f720 0x555592383550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555592265560 con 0x555592379890 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555592265200 con 0x55559237ee00 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559214c6a0 con 0x55559237f720 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75f54eeb80 10 client.0 _ll_register_callbacks cb 0x55559226eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 0x555592382ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e27fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x555592383010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237f720 0x555592383550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 0x555592382ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47566/0 (socket says 172.21.15.16:47566) 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 -- 172.21.15.16:0/4211204000 learned_addr learned my addr 172.21.15.16:0/4211204000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 -- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237f720 msgr2=0x555592383550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237f720 0x555592383550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 -- 172.21.15.16:0/4211204000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 msgr2=0x555592383010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x555592383010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 -- 172.21.15.16:0/4211204000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e410d550 con 0x555592379890 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 0x555592382ad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f75d4011770 tx=0x7f75d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9645bff1a16475fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.250+0000 7f75c6ff5700 1 -- 172.21.15.16:0/4211204000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d4004a30 con 0x555592379890 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75c6ff5700 1 -- 172.21.15.16:0/4211204000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f75d4017070 con 0x555592379890 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555592222270 con 0x555592379890 2024-05-08T23:23:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75c6ff5700 1 -- 172.21.15.16:0/4211204000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75d400be60 con 0x555592379890 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 10 client.16338 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5555920f6b60 con 0x555592379890 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75c6ff5700 1 -- 172.21.15.16:0/4211204000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555592222270 con 0x555592379890 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75c6ff5700 1 -- 172.21.15.16:0/4211204000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75d401f640 con 0x555592379890 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 10 client.16338 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 10 client.16338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 20 client.16338 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923931c0 0x5555923955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 4 client.16338 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.251+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555592348810 con 0x5555923931c0 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.252+0000 7f75e27fc700 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923931c0 0x5555923955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.252+0000 7f75c6ff5700 10 client.16338 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.252+0000 7f75e27fc700 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923931c0 0x5555923955e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2384d79ebebd170c server_cookie=5b7499d81a6fa67e in_seq=0 out_seq=0 2024-05-08T23:23:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75c6ff5700 1 -- 172.21.15.16:0/4211204000 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555592348810 con 0x5555923931c0 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75c6ff5700 10 client.16338 handle_command_reply: tid=0 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 client.16338 shutdown 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 msgr2=0x555592382ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 0x555592382ad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f75d4011770 tx=0x7f75d40047c0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 shutdown_connections 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237f720 0x555592383550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555592379890 0x555592382ad0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923931c0 0x5555923955e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4211204000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55559237ee00 0x555592383010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75c6ff5700 0 client.16338 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 >> 172.21.15.16:0/4211204000 conn(0x5555922d7210 msgr2=0x5555923735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 shutdown_connections 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4211204000 wait complete. 2024-05-08T23:23:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.253+0000 7f75f54eeb80 20 client.16338 trim_cache size 0 max 16384 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 Processor -- start 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 -- start start 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592373d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x55559237e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592383370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559236e800 con 0x55559237f720 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555592375020 con 0x555592379890 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.267+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555592381020 con 0x55559237ee00 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592383370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592383370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47570/0 (socket says 172.21.15.16:47570) 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 -- 172.21.15.16:0/4112287038 learned_addr learned my addr 172.21.15.16:0/4112287038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e27fc700 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x55559237e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592373d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 -- 172.21.15.16:0/4112287038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 msgr2=0x55559237e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x55559237e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 -- 172.21.15.16:0/4112287038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 msgr2=0x555592373d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592373d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 -- 172.21.15.16:0/4112287038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559228f560 con 0x55559237f720 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592373d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75e37fe700 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592383370 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0c9ca0 tx=0x7f75dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adac39f7558f0de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75c9ffb700 1 -- 172.21.15.16:0/4112287038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b90b0 con 0x55559237f720 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75c9ffb700 1 -- 172.21.15.16:0/4112287038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75dc00dc60 con 0x55559237f720 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.268+0000 7f75c9ffb700 1 -- 172.21.15.16:0/4112287038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc09d120 con 0x55559237f720 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4112287038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 msgr2=0x555592383370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592383370 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0c9ca0 tx=0x7f75dc00b040 comp rx=0 tx=0).stop 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4112287038 shutdown_connections 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x555592383370 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592373d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/4112287038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x55559237e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4112287038 >> 172.21.15.16:0/4112287038 conn(0x5555922d7210 msgr2=0x55559237dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4112287038 shutdown_connections 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 -- 172.21.15.16:0/4112287038 wait complete. 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 -- start start 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.269+0000 7f75f54eeb80 1 Processor -- start 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592386110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x555592386650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x5555923e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555592386de0 con 0x55559237f720 2024-05-08T23:23:03.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555592386f50 con 0x555592379890 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555923e8bf0 con 0x55559237ee00 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75f54eeb80 10 client.0 _ll_register_callbacks cb 0x55559226eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x5555923e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e2ffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592386110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x5555923e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47574/0 (socket says 172.21.15.16:47574) 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 -- 172.21.15.16:0/36961628 learned_addr learned my addr 172.21.15.16:0/36961628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e27fc700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x555592386650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 -- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 msgr2=0x555592386650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x555592386650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 -- 172.21.15.16:0/36961628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 msgr2=0x555592386110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592386110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 -- 172.21.15.16:0/36961628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e41a3f10 con 0x55559237f720 2024-05-08T23:23:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592386110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.270+0000 7f75e37fe700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x5555923e86b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0b9010 tx=0x7f75dc0b91b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e333998114820a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75c6ff5700 1 -- 172.21.15.16:0/36961628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc099d00 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75c6ff5700 1 -- 172.21.15.16:0/36961628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f75dc0a1a80 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75d801e040 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75c6ff5700 1 -- 172.21.15.16:0/36961628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75dc099860 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 10 client.16344 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55559214ecd0 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75c6ff5700 1 -- 172.21.15.16:0/36961628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75d801e040 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75c6ff5700 1 -- 172.21.15.16:0/36961628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75dc098bf0 con 0x55559237f720 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 10 client.16344 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 10 client.16344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 20 client.16344 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923ea0c0 0x5555923ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 4 client.16344 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.271+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5555922c1ec0 con 0x5555923ea0c0 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923ea0c0 0x5555923ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75e2ffd700 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923ea0c0 0x5555923ec4e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7899e7809df5b23e server_cookie=4d8167cc1f86bb11 in_seq=0 out_seq=0 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75c6ff5700 10 client.16344 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75c6ff5700 1 -- 172.21.15.16:0/36961628 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5555922c1ec0 con 0x5555923ea0c0 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75c6ff5700 10 client.16344 handle_command_reply: tid=0 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 client.16344 shutdown 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 msgr2=0x5555923e86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x5555923e86b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0b9010 tx=0x7f75dc0b91b0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 shutdown_connections 2024-05-08T23:23:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55559237f720 0x5555923e86b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5555923ea0c0 0x5555923ec4e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555592379890 0x555592386110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 --2- 172.21.15.16:0/36961628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55559237ee00 0x555592386650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75c6ff5700 0 client.16344 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 >> 172.21.15.16:0/36961628 conn(0x5555922d7210 msgr2=0x555592380440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.272+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 shutdown_connections 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75f54eeb80 1 -- 172.21.15.16:0/36961628 wait complete. 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75f54eeb80 20 client.16344 trim_cache size 0 max 16384 2024-05-08T23:23:03.280 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:23:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f75cc0a1290 msgr2=0x7f75cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f75cc0a1290 0x7f75cc0a3750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005950 tx=0x7f75d400b410 comp rx=0 tx=0).stop 2024-05-08T23:23:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 msgr2=0x7f75e41a3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 0x7f75e41a3140 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f75dc007ee0 tx=0x7f75dc00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 shutdown_connections 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f75cc0a1290 0x7f75cc0a3750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4105ec0 0x7f75e41a3140 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75e40ffe00 0x7f75e41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 --2- 172.21.15.16:0/1464343554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75e41007c0 0x7f75e4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 >> 172.21.15.16:0/1464343554 conn(0x7f75e4076f10 msgr2=0x7f75e40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 shutdown_connections 2024-05-08T23:23:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.273+0000 7f75e9334700 1 -- 172.21.15.16:0/1464343554 wait complete. 2024-05-08T23:23:03.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:03 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2151802269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:03.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:23:03.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:03 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2151802269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 -- 172.21.15.16:0/1338959426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c0ffdd0 msgr2=0x7f250c10dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 --2- 172.21.15.16:0/1338959426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c0ffdd0 0x7f250c10dda0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f250400b0a0 tx=0x7f250400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 -- 172.21.15.16:0/1338959426 shutdown_connections 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 --2- 172.21.15.16:0/1338959426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c0ffdd0 0x7f250c10dda0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 --2- 172.21.15.16:0/1338959426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c105ef0 0x7f250c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 --2- 172.21.15.16:0/1338959426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ff410 0x7f250c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 -- 172.21.15.16:0/1338959426 >> 172.21.15.16:0/1338959426 conn(0x7f250c0763a0 msgr2=0x7f250c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 -- 172.21.15.16:0/1338959426 shutdown_connections 2024-05-08T23:23:03.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.425+0000 7f25143b4700 1 -- 172.21.15.16:0/1338959426 wait complete. 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 Processor -- start 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 -- start start 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c0ff410 0x7f250c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ffdd0 0x7f250c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 0x7f250c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c19c370 con 0x7f250c105ef0 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c19c4e0 con 0x7f250c0ffdd0 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.426+0000 7f25143b4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c19c6a0 con 0x7f250c0ff410 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 0x7f250c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 0x7f250c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47598/0 (socket says 172.21.15.16:47598) 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 -- 172.21.15.16:0/2576055876 learned_addr learned my addr 172.21.15.16:0/2576055876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512150700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c0ff410 0x7f250c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f251194f700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ffdd0 0x7f250c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 -- 172.21.15.16:0/2576055876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c0ff410 msgr2=0x7f250c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c0ff410 0x7f250c197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 -- 172.21.15.16:0/2576055876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ffdd0 msgr2=0x7f250c197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ffdd0 0x7f250c197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 -- 172.21.15.16:0/2576055876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504009d00 con 0x7f250c105ef0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f251194f700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ffdd0 0x7f250c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f2512951700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 0x7f250c19bd30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f250400b070 tx=0x7f2504007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f25037fe700 1 -- 172.21.15.16:0/2576055876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400e040 con 0x7f250c105ef0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f25037fe700 1 -- 172.21.15.16:0/2576055876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250401d480 con 0x7f250c105ef0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250c19c920 con 0x7f250c105ef0 2024-05-08T23:23:03.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f25037fe700 1 -- 172.21.15.16:0/2576055876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250401b6b0 con 0x7f250c105ef0 2024-05-08T23:23:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.427+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c1a41c0 con 0x7f250c105ef0 2024-05-08T23:23:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.430+0000 7f25037fe700 1 -- 172.21.15.16:0/2576055876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f250401d8a0 con 0x7f250c105ef0 2024-05-08T23:23:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.430+0000 7f25037fe700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.431+0000 7f2512150700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.431+0000 7f25037fe700 1 -- 172.21.15.16:0/2576055876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f25040ce750 con 0x7f250c105ef0 2024-05-08T23:23:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.431+0000 7f252056eb80 1 Processor -- start 2024-05-08T23:23:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.431+0000 7f2512150700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2508009c60 tx=0x7f2508009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.431+0000 7f252056eb80 1 -- start start 2024-05-08T23:23:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f252056eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d65866a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f252056eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d658664760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f252056eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658665100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f252056eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6585b7150 con 0x55d65866a1f0 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f252056eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6585b4b10 con 0x55d65866ab10 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f252056eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6585b0400 con 0x55d658664ca0 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d65866a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d65866a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47606/0 (socket says 172.21.15.16:47606) 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 -- 172.21.15.16:0/1063776815 learned_addr learned my addr 172.21.15.16:0/1063776815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512951700 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658665100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f251194f700 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d658664760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 -- 172.21.15.16:0/1063776815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 msgr2=0x55d658665100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658665100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 -- 172.21.15.16:0/1063776815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 msgr2=0x55d658664760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d658664760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 -- 172.21.15.16:0/1063776815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6584313b0 con 0x55d65866a1f0 2024-05-08T23:23:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.432+0000 7f2512150700 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d65866a5d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f250800e4e0 tx=0x7f2508014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff434927b7e67589 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f2500da4700 1 -- 172.21.15.16:0/1063776815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250800ee00 con 0x55d65866a1f0 2024-05-08T23:23:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f2500da4700 1 -- 172.21.15.16:0/1063776815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2508021070 con 0x55d65866a1f0 2024-05-08T23:23:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f2500da4700 1 -- 172.21.15.16:0/1063776815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250801d740 con 0x55d65866a1f0 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 -- 172.21.15.16:0/1063776815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 msgr2=0x55d65866a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d65866a5d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f250800e4e0 tx=0x7f2508014000 comp rx=0 tx=0).stop 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 -- 172.21.15.16:0/1063776815 shutdown_connections 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658665100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d65866a5d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 --2- 172.21.15.16:0/1063776815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d658664760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.433+0000 7f252056eb80 1 -- 172.21.15.16:0/1063776815 >> 172.21.15.16:0/1063776815 conn(0x55d6585be210 msgr2=0x55d65865adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 -- 172.21.15.16:0/1063776815 shutdown_connections 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 -- 172.21.15.16:0/1063776815 wait complete. 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 -- start start 2024-05-08T23:23:03.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 Processor -- start 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d658664ca0 0x55d65866de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 0x55d65866e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65854c560 con 0x55d65866ab10 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65854c200 con 0x55d658664ca0 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6584336a0 con 0x55d65866a1f0 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f252056eb80 10 client.0 _ll_register_callbacks cb 0x55d658555b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f251194f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 0x55d65866e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.434+0000 7f251194f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 0x55d65866e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:58236/0 (socket says 172.21.15.16:58236) 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 -- 172.21.15.16:0/3715195496 learned_addr learned my addr 172.21.15.16:0/3715195496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f2512150700 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d658664ca0 0x55d65866de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 -- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d658664ca0 msgr2=0x55d65866de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d658664ca0 0x55d65866de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 -- 172.21.15.16:0/3715195496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 msgr2=0x55d65866e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 -- 172.21.15.16:0/3715195496 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2508017070 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f251194f700 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 0x55d65866e3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f24fc013790 tx=0x7f24fc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=837b272506a3921b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f24f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.435+0000 7f24f5ffb700 1 -- 172.21.15.16:0/3715195496 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc009810 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d658509270 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f24f5ffb700 1 -- 172.21.15.16:0/3715195496 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f24fc011970 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f24f5ffb700 1 -- 172.21.15.16:0/3715195496 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24fc011b30 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 10 client.26140 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d6583ddb60 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f24f5ffb700 1 -- 172.21.15.16:0/3715195496 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d658509270 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f24f5ffb700 1 -- 172.21.15.16:0/3715195496 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24fc01d3f0 con 0x55d65866a1f0 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 10 client.26140 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 20 client.26140 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d65867e6d0 0x55d658680af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 4 client.26140 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.436+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d65862f810 con 0x55d65867e6d0 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.437+0000 7f2512150700 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d65867e6d0 0x55d658680af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.437+0000 7f2512150700 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d65867e6d0 0x55d658680af0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d74e007528389a server_cookie=89c7ef87a35fec8a in_seq=0 out_seq=0 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.437+0000 7f24f5ffb700 10 client.26140 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f24f5ffb700 1 -- 172.21.15.16:0/3715195496 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d65862f810 con 0x55d65867e6d0 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f24f5ffb700 10 client.26140 handle_command_reply: tid=0 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 client.26140 shutdown 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 msgr2=0x55d65866e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 0x55d65866e3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f24fc013790 tx=0x7f24fc009f90 comp rx=0 tx=0).stop 2024-05-08T23:23:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 shutdown_connections 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d65867e6d0 0x55d658680af0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d658664ca0 0x55d65866de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 --2- 172.21.15.16:0/3715195496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d65866a1f0 0x55d65866e3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 >> 172.21.15.16:0/3715195496 conn(0x55d6585be210 msgr2=0x55d65865a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f24f5ffb700 0 client.26140 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 shutdown_connections 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 1 -- 172.21.15.16:0/3715195496 wait complete. 2024-05-08T23:23:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.438+0000 7f252056eb80 20 client.26140 trim_cache size 0 max 16384 2024-05-08T23:23:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 Processor -- start 2024-05-08T23:23:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 -- start start 2024-05-08T23:23:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d65867a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866a1f0 0x55d65867a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65867bdf0 con 0x55d65866ab10 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65866dca0 con 0x55d65866a1f0 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f252056eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65865b960 con 0x55d658664ca0 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f2512951700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f2512951700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47626/0 (socket says 172.21.15.16:47626) 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f2512951700 1 -- 172.21.15.16:0/3089233504 learned_addr learned my addr 172.21.15.16:0/3089233504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.452+0000 7f251194f700 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866a1f0 0x55d65867a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512150700 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d65867a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512951700 1 -- 172.21.15.16:0/3089233504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 msgr2=0x55d65867a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512951700 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d65867a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512951700 1 -- 172.21.15.16:0/3089233504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866a1f0 msgr2=0x55d65867a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512951700 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866a1f0 0x55d65867a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512951700 1 -- 172.21.15.16:0/3089233504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d658576560 con 0x55d65866ab10 2024-05-08T23:23:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f2512951700 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866d760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2504097a50 tx=0x7f250409b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3818d2b6eb014abd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f25015a5700 1 -- 172.21.15.16:0/3089233504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250409bc50 con 0x55d65866ab10 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f25015a5700 1 -- 172.21.15.16:0/3089233504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f250409b550 con 0x55d65866ab10 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.453+0000 7f25015a5700 1 -- 172.21.15.16:0/3089233504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25040999a0 con 0x55d65866ab10 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- 172.21.15.16:0/3089233504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 msgr2=0x55d65866d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866d760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2504097a50 tx=0x7f250409b420 comp rx=0 tx=0).stop 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- 172.21.15.16:0/3089233504 shutdown_connections 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866ab10 0x55d65866d760 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d65867a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- 172.21.15.16:0/3089233504 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866a1f0 0x55d65867a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- 172.21.15.16:0/3089233504 >> 172.21.15.16:0/3089233504 conn(0x55d6585be210 msgr2=0x55d658668f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- 172.21.15.16:0/3089233504 shutdown_connections 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- 172.21.15.16:0/3089233504 wait complete. 2024-05-08T23:23:03.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- start start 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 Processor -- start 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658675a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d658666ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d6586675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d658665cd0 con 0x55d65866a1f0 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d658665e40 con 0x55d65866ab10 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d658666000 con 0x55d658664ca0 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.454+0000 7f252056eb80 10 client.0 _ll_register_callbacks cb 0x55d658555b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f251194f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d658666ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658675a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658675a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:58260/0 (socket says 172.21.15.16:58260) 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f251194f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d658666ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47632/0 (socket says 172.21.15.16:47632) 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 -- 172.21.15.16:0/1223117030 learned_addr learned my addr 172.21.15.16:0/1223117030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512951700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d6586675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 -- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 msgr2=0x55d6586675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d6586675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 -- 172.21.15.16:0/1223117030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 msgr2=0x55d658666ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d658666ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 -- 172.21.15.16:0/1223117030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250401de70 con 0x55d658664ca0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f251194f700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d658666ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f2512150700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658675a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2508009ec0 tx=0x7f2508011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b81e695248b27366 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f24f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f24f5ffb700 1 -- 172.21.15.16:0/1223117030 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080050c0 con 0x55d658664ca0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f24f5ffb700 1 -- 172.21.15.16:0/1223117030 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2508005220 con 0x55d658664ca0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.455+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2508021070 con 0x55d658664ca0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f24f5ffb700 1 -- 172.21.15.16:0/1223117030 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25080053e0 con 0x55d658664ca0 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 10 client.26146 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d658435cd0 con 0x55d658664ca0 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f24f5ffb700 1 -- 172.21.15.16:0/1223117030 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2508021070 con 0x55d658664ca0 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f24f5ffb700 1 -- 172.21.15.16:0/1223117030 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f250803a060 con 0x55d658664ca0 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 10 client.26146 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 20 client.26146 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d6586d5540 0x55d6586d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 4 client.26146 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d6585a8ec0 con 0x55d6586d5540 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.456+0000 7f251194f700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d6586d5540 0x55d6586d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f24f5ffb700 10 client.26146 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f251194f700 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d6586d5540 0x55d6586d7960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252db9c0a545dc94 server_cookie=23e41a6199702ccc in_seq=0 out_seq=0 2024-05-08T23:23:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f24f5ffb700 1 -- 172.21.15.16:0/1223117030 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d6585a8ec0 con 0x55d6586d5540 2024-05-08T23:23:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f24f5ffb700 10 client.26146 handle_command_reply: tid=0 2024-05-08T23:23:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 client.26146 shutdown 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 msgr2=0x55d658675a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658675a60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2508009ec0 tx=0x7f2508011ae0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 shutdown_connections 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55d6586d5540 0x55d6586d7960 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d65866ab10 0x55d6586675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d658664ca0 0x55d658675a60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f24f5ffb700 0 client.26146 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 --2- 172.21.15.16:0/1223117030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d65866a1f0 0x55d658666ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 >> 172.21.15.16:0/1223117030 conn(0x55d6585be210 msgr2=0x55d6586697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 shutdown_connections 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 1 -- 172.21.15.16:0/1223117030 wait complete. 2024-05-08T23:23:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.457+0000 7f252056eb80 20 client.26146 trim_cache size 0 max 16384 2024-05-08T23:23:03.465 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:23:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 msgr2=0x7f24f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2508009c60 tx=0x7f2508009260 comp rx=0 tx=0).stop 2024-05-08T23:23:03.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 msgr2=0x7f250c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 0x7f250c19bd30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f250400b070 tx=0x7f2504007e00 comp rx=0 tx=0).stop 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 shutdown_connections 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105ef0 0x7f250c19bd30 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c0ff410 0x7f250c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 --2- 172.21.15.16:0/2576055876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c0ffdd0 0x7f250c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 >> 172.21.15.16:0/2576055876 conn(0x7f250c0763a0 msgr2=0x7f250c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 shutdown_connections 2024-05-08T23:23:03.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:03.458+0000 7f25143b4700 1 -- 172.21.15.16:0/2576055876 wait complete. 2024-05-08T23:23:03.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:03 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2151802269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:04.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:04 smithi016 ceph-mon[42541]: pgmap v108: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-05-08T23:23:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:04 smithi189 ceph-mon[46919]: pgmap v108: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-05-08T23:23:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:04 smithi155 ceph-mon[47105]: pgmap v108: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-05-08T23:23:04.472 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:23:04.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:23:04.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 -- 172.21.15.16:0/1552937842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc100a10 msgr2=0x7fa1fc104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 --2- 172.21.15.16:0/1552937842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc100a10 0x7fa1fc104a30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009990 tx=0x7fa1ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 -- 172.21.15.16:0/1552937842 shutdown_connections 2024-05-08T23:23:04.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 --2- 172.21.15.16:0/1552937842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1fc105160 0x7fa1fc10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 --2- 172.21.15.16:0/1552937842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 0x7fa1fc100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 --2- 172.21.15.16:0/1552937842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc100a10 0x7fa1fc104a30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 -- 172.21.15.16:0/1552937842 >> 172.21.15.16:0/1552937842 conn(0x7fa1fc076f40 msgr2=0x7fa1fc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 -- 172.21.15.16:0/1552937842 shutdown_connections 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.623+0000 7fa2033d8700 1 -- 172.21.15.16:0/1552937842 wait complete. 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 Processor -- start 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 -- start start 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 0x7fa1fc199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1fc105160 0x7fa1fc199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 0x7fa1fc19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc19e870 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc19e9e0 con 0x7fa1fc105160 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa2033d8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc19eba0 con 0x7fa1fc100060 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa201975700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 0x7fa1fc19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa201975700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 0x7fa1fc19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47640/0 (socket says 172.21.15.16:47640) 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa201975700 1 -- 172.21.15.16:0/2499740445 learned_addr learned my addr 172.21.15.16:0/2499740445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa201174700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 0x7fa1fc199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.624+0000 7fa200973700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1fc105160 0x7fa1fc199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201975700 1 -- 172.21.15.16:0/2499740445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 msgr2=0x7fa1fc199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201975700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 0x7fa1fc199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201975700 1 -- 172.21.15.16:0/2499740445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1fc105160 msgr2=0x7fa1fc199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201975700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1fc105160 0x7fa1fc199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201975700 1 -- 172.21.15.16:0/2499740445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0095f0 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201174700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 0x7fa1fc199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa201975700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 0x7fa1fc19e270 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400ef30 tx=0x7fa1f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa1f27fc700 1 -- 172.21.15.16:0/2499740445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4018070 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa1f27fc700 1 -- 172.21.15.16:0/2499740445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1f400f040 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc1a60a0 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa1f27fc700 1 -- 172.21.15.16:0/2499740445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400cbc0 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.625+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc1a6530 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.628+0000 7fa1f27fc700 1 -- 172.21.15.16:0/2499740445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa1f4007500 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.629+0000 7fa1f27fc700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1e80a1290 0x7fa1e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.629+0000 7fa201174700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1e80a1290 0x7fa1e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.629+0000 7fa1f27fc700 1 -- 172.21.15.16:0/2499740445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa1f40ccaf0 con 0x7fa1fc19ddf0 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.629+0000 7fa201174700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1e80a1290 0x7fa1e80a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8011490 tx=0x7fa1f8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 Processor -- start 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 -- start start 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c492e80 0x55dc6c493260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c48d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c48d910 0x55dc6c48dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c3e41d0 con 0x55dc6c4937a0 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c3e1b90 con 0x55dc6c48d910 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa20f592b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c3dd480 con 0x55dc6c492e80 2024-05-08T23:23:04.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c48d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c48d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47646/0 (socket says 172.21.15.16:47646) 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 -- 172.21.15.16:0/3498901855 learned_addr learned my addr 172.21.15.16:0/3498901855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa201174700 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c492e80 0x55dc6c493260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa201975700 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c48d910 0x55dc6c48dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 -- 172.21.15.16:0/3498901855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c492e80 msgr2=0x55dc6c493260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c492e80 0x55dc6c493260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 -- 172.21.15.16:0/3498901855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c48d910 msgr2=0x55dc6c48dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c48d910 0x55dc6c48dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.630+0000 7fa200973700 1 -- 172.21.15.16:0/3498901855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc6c25e430 con 0x55dc6c4937a0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.631+0000 7fa200973700 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c48d3d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06a6d0 tx=0x7fa1ec005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a17d1437b26efa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.631+0000 7fa1e77fe700 1 -- 172.21.15.16:0/3498901855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec01d070 con 0x55dc6c4937a0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.631+0000 7fa1e77fe700 1 -- 172.21.15.16:0/3498901855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1ec01e040 con 0x55dc6c4937a0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.631+0000 7fa1e77fe700 1 -- 172.21.15.16:0/3498901855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec004700 con 0x55dc6c4937a0 2024-05-08T23:23:04.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 -- 172.21.15.16:0/3498901855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 msgr2=0x55dc6c48d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c48d3d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06a6d0 tx=0x7fa1ec005c40 comp rx=0 tx=0).stop 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 -- 172.21.15.16:0/3498901855 shutdown_connections 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c48d910 0x55dc6c48dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c492e80 0x55dc6c493260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 --2- 172.21.15.16:0/3498901855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c48d3d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 -- 172.21.15.16:0/3498901855 >> 172.21.15.16:0/3498901855 conn(0x55dc6c3eb290 msgr2=0x55dc6c487e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 -- 172.21.15.16:0/3498901855 shutdown_connections 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 -- 172.21.15.16:0/3498901855 wait complete. 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 -- start start 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.632+0000 7fa20f592b80 1 Processor -- start 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c4997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c496b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c497270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c3795e0 con 0x55dc6c4937a0 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c379280 con 0x55dc6c492e80 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c260720 con 0x55dc6c48d910 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa20f592b80 10 client.0 _ll_register_callbacks cb 0x55dc6c382bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c497270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c497270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47650/0 (socket says 172.21.15.16:47650) 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201174700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c4997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 -- 172.21.15.16:0/4075064811 learned_addr learned my addr 172.21.15.16:0/4075064811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa200973700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c496b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 -- 172.21.15.16:0/4075064811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 msgr2=0x55dc6c4997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c4997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 -- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 msgr2=0x55dc6c496b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c496b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 -- 172.21.15.16:0/4075064811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc067ff0 con 0x55dc6c4937a0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201174700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c4997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa201975700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c497270 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa1f401b000 tx=0x7fa1f40992c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87edd760002d4ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.633+0000 7fa1e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/4075064811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f40b2210 con 0x55dc6c4937a0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc6c3362f0 con 0x55dc6c4937a0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/4075064811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa1f4099ff0 con 0x55dc6c4937a0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/4075064811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f40998f0 con 0x55dc6c4937a0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 10 client.16371 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dc6c20abe0 con 0x55dc6c4937a0 2024-05-08T23:23:04.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/4075064811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1f4063020 con 0x55dc6c4937a0 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/4075064811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1f4099c70 con 0x55dc6c4937a0 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 20 client.16371 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4a7170 0x55dc6c4a9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 4 client.16371 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc6c45c890 con 0x55dc6c4a7170 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.634+0000 7fa201174700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4a7170 0x55dc6c4a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.635+0000 7fa201174700 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4a7170 0x55dc6c4a9590 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a21360e8c64be8 server_cookie=d59c9fd6e514a4c0 in_seq=0 out_seq=0 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.635+0000 7fa1e4ff9700 10 client.16371 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/4075064811 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc6c45c890 con 0x55dc6c4a7170 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa1e4ff9700 10 client.16371 handle_command_reply: tid=0 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 client.16371 shutdown 2024-05-08T23:23:04.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 msgr2=0x55dc6c497270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c497270 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa1f401b000 tx=0x7fa1f40992c0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 shutdown_connections 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c4937a0 0x55dc6c497270 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c4997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4a7170 0x55dc6c4a9590 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 --2- 172.21.15.16:0/4075064811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c496b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa1e4ff9700 0 client.16371 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 >> 172.21.15.16:0/4075064811 conn(0x55dc6c3eb290 msgr2=0x55dc6c487620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 shutdown_connections 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 1 -- 172.21.15.16:0/4075064811 wait complete. 2024-05-08T23:23:04.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.636+0000 7fa20f592b80 20 client.16371 trim_cache size 0 max 16384 2024-05-08T23:23:04.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 Processor -- start 2024-05-08T23:23:04.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 -- start start 2024-05-08T23:23:04.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c487da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 0x55dc6c491c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c4937a0 0x55dc6c492160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c484e90 con 0x55dc6c492e80 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c489f00 con 0x55dc6c4937a0 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa20f592b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c489740 con 0x55dc6c48d910 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa200973700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 0x55dc6c491c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa200973700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 0x55dc6c491c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47654/0 (socket says 172.21.15.16:47654) 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.655+0000 7fa200973700 1 -- 172.21.15.16:0/2463630505 learned_addr learned my addr 172.21.15.16:0/2463630505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa201174700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c487da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa201975700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c4937a0 0x55dc6c492160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa200973700 1 -- 172.21.15.16:0/2463630505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 msgr2=0x55dc6c487da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa200973700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c487da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa200973700 1 -- 172.21.15.16:0/2463630505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c4937a0 msgr2=0x55dc6c492160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa200973700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c4937a0 0x55dc6c492160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa200973700 1 -- 172.21.15.16:0/2463630505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc6c484560 con 0x55dc6c492e80 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa201174700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c487da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa201975700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c4937a0 0x55dc6c492160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa200973700 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 0x55dc6c491c20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006010 tx=0x7fa1ec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b699e3ea881fa9c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa1e7fff700 1 -- 172.21.15.16:0/2463630505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec01d040 con 0x55dc6c492e80 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa1e7fff700 1 -- 172.21.15.16:0/2463630505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1ec00ba40 con 0x55dc6c492e80 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.656+0000 7fa1e7fff700 1 -- 172.21.15.16:0/2463630505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec004ac0 con 0x55dc6c492e80 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 -- 172.21.15.16:0/2463630505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 msgr2=0x55dc6c491c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 0x55dc6c491c20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006010 tx=0x7fa1ec005f10 comp rx=0 tx=0).stop 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 -- 172.21.15.16:0/2463630505 shutdown_connections 2024-05-08T23:23:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c4937a0 0x55dc6c492160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c48d910 0x55dc6c487da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2463630505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c492e80 0x55dc6c491c20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 -- 172.21.15.16:0/2463630505 >> 172.21.15.16:0/2463630505 conn(0x55dc6c3eb290 msgr2=0x55dc6c495420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 -- 172.21.15.16:0/2463630505 shutdown_connections 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 -- 172.21.15.16:0/2463630505 wait complete. 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 -- start start 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.657+0000 7fa20f592b80 1 Processor -- start 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 0x55dc6c48f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c48f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c4937a0 0x55dc6c48fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c49ea60 con 0x55dc6c48d910 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c4fd670 con 0x55dc6c492e80 2024-05-08T23:23:04.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6c4fd7e0 con 0x55dc6c4937a0 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa20f592b80 10 client.0 _ll_register_callbacks cb 0x55dc6c382bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa201174700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 0x55dc6c48f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa201174700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 0x55dc6c48f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47656/0 (socket says 172.21.15.16:47656) 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa201174700 1 -- 172.21.15.16:0/2054269822 learned_addr learned my addr 172.21.15.16:0/2054269822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa200973700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c48f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa201975700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c4937a0 0x55dc6c48fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.658+0000 7fa201174700 1 -- 172.21.15.16:0/2054269822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c4937a0 msgr2=0x55dc6c48fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa201174700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c4937a0 0x55dc6c48fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa201174700 1 -- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 msgr2=0x55dc6c48f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa201174700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c48f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa201174700 1 -- 172.21.15.16:0/2054269822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc6c25e430 con 0x55dc6c48d910 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa200973700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c48f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:04.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa201174700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 0x55dc6c48f260 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55dc6c4a9e70 tx=0x7fa1f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f29c542620ca32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa1e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/2054269822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8024040 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/2054269822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa1f8003e10 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1ec01e040 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/2054269822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f80297b0 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa20f592b80 10 client.16377 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dc6c3a35e0 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/2054269822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1ec01e040 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.659+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/2054269822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1f80327c0 con 0x55dc6c48d910 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa20f592b80 10 client.16377 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa20f592b80 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa20f592b80 20 client.16377 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4ff080 0x55dc6c5014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa20f592b80 4 client.16377 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:23:04.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dc6c3d5f40 con 0x55dc6c4ff080 2024-05-08T23:23:04.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa200973700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4ff080 0x55dc6c5014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa200973700 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4ff080 0x55dc6c5014a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=570d64b4046897ec server_cookie=74d74fbff40e0d98 in_seq=0 out_seq=0 2024-05-08T23:23:04.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.660+0000 7fa1e4ff9700 10 client.16377 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.661+0000 7fa1e4ff9700 1 -- 172.21.15.16:0/2054269822 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dc6c3d5f40 con 0x55dc6c4ff080 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.661+0000 7fa1e4ff9700 10 client.16377 handle_command_reply: tid=0 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.661+0000 7fa20f592b80 1 client.16377 shutdown 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.661+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 msgr2=0x55dc6c48f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.661+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 0x55dc6c48f260 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55dc6c4a9e70 tx=0x7fa1f80047c0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 shutdown_connections 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc6c4937a0 0x55dc6c48fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc6c48d910 0x55dc6c48f260 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc6c4ff080 0x55dc6c5014a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 --2- 172.21.15.16:0/2054269822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc6c492e80 0x55dc6c48f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa1e4ff9700 0 client.16377 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 >> 172.21.15.16:0/2054269822 conn(0x55dc6c3eb290 msgr2=0x55dc6c491c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 shutdown_connections 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 1 -- 172.21.15.16:0/2054269822 wait complete. 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa20f592b80 20 client.16377 trim_cache size 0 max 16384 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "fe237874-8f70-44b9-b57c-80ce141ce5f8", 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:23:04.670 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:23:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1e80a1290 msgr2=0x7fa1e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1e80a1290 0x7fa1e80a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8011490 tx=0x7fa1f8009260 comp rx=0 tx=0).stop 2024-05-08T23:23:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 msgr2=0x7fa1fc19e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 0x7fa1fc19e270 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400ef30 tx=0x7fa1f400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 shutdown_connections 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1e80a1290 0x7fa1e80a3750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1fc19ddf0 0x7fa1fc19e270 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.662+0000 7fa2033d8700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc100060 0x7fa1fc199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.663+0000 7fa2033d8700 1 --2- 172.21.15.16:0/2499740445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1fc105160 0x7fa1fc199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.663+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 >> 172.21.15.16:0/2499740445 conn(0x7fa1fc076f40 msgr2=0x7fa1fc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.663+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 shutdown_connections 2024-05-08T23:23:04.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.663+0000 7fa2033d8700 1 -- 172.21.15.16:0/2499740445 wait complete. 2024-05-08T23:23:04.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 -- 172.21.15.16:0/495313939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c0682e0 msgr2=0x7f1f0c0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 --2- 172.21.15.16:0/495313939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c0682e0 0x7f1f0c0686c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1f080099c0 tx=0x7f1f08009cd0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 -- 172.21.15.16:0/495313939 shutdown_connections 2024-05-08T23:23:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 --2- 172.21.15.16:0/495313939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f0c106290 0x7f1f0c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 --2- 172.21.15.16:0/495313939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c0682e0 0x7f1f0c0686c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 --2- 172.21.15.16:0/495313939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f0c068c90 0x7f1f0c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 -- 172.21.15.16:0/495313939 >> 172.21.15.16:0/495313939 conn(0x7f1f0c076f40 msgr2=0x7f1f0c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 -- 172.21.15.16:0/495313939 shutdown_connections 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.815+0000 7f1f13c68700 1 -- 172.21.15.16:0/495313939 wait complete. 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 Processor -- start 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 -- start start 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f0c0682e0 0x7f1f0c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 0x7f1f0c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f0c19bc20 0x7f1f0c19c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c19c6a0 con 0x7f1f0c106290 2024-05-08T23:23:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c19c810 con 0x7f1f0c19bc20 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f13c68700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c19c9d0 con 0x7f1f0c0682e0 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f11203700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 0x7f1f0c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f11203700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 0x7f1f0c197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47676/0 (socket says 172.21.15.16:47676) 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f11203700 1 -- 172.21.15.16:0/436796124 learned_addr learned my addr 172.21.15.16:0/436796124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.816+0000 7f1f11a04700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f0c0682e0 0x7f1f0c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f12205700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f0c19bc20 0x7f1f0c19c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f11203700 1 -- 172.21.15.16:0/436796124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f0c0682e0 msgr2=0x7f1f0c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f11203700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f0c0682e0 0x7f1f0c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f11203700 1 -- 172.21.15.16:0/436796124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f0c19bc20 msgr2=0x7f1f0c19c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f11203700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f0c19bc20 0x7f1f0c19c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f11203700 1 -- 172.21.15.16:0/436796124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080095f0 con 0x7f1f0c106290 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f11203700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 0x7f1f0c197880 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e970 tx=0x7f1efc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f02ffd700 1 -- 172.21.15.16:0/436796124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc009790 con 0x7f1f0c106290 2024-05-08T23:23:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f02ffd700 1 -- 172.21.15.16:0/436796124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1efc00c7e0 con 0x7f1f0c106290 2024-05-08T23:23:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c1a3f30 con 0x7f1f0c106290 2024-05-08T23:23:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f02ffd700 1 -- 172.21.15.16:0/436796124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc005440 con 0x7f1f0c106290 2024-05-08T23:23:04.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.817+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c1a43b0 con 0x7f1f0c106290 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.819+0000 7f1f02ffd700 1 -- 172.21.15.16:0/436796124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1efc00c950 con 0x7f1f0c106290 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.820+0000 7f1f02ffd700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ef80a1290 0x7f1ef80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.820+0000 7f1f02ffd700 1 -- 172.21.15.16:0/436796124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1efc014070 con 0x7f1f0c106290 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.820+0000 7f1f11a04700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ef80a1290 0x7f1ef80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.821+0000 7f1f11a04700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ef80a1290 0x7f1ef80a3750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f08006010 tx=0x7f1f08019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.821+0000 7f1f1fe22b80 1 Processor -- start 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.821+0000 7f1f1fe22b80 1 -- start start 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cda1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875cd4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875cd4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875c2b150 con 0x556875cd9e00 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875c28b10 con 0x556875cd4890 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875c24400 con 0x556875cda720 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cda1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cda1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47678/0 (socket says 172.21.15.16:47678) 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 -- 172.21.15.16:0/3666132618 learned_addr learned my addr 172.21.15.16:0/3666132618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11203700 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875cd4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f12205700 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875cd4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 -- 172.21.15.16:0/3666132618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 msgr2=0x556875cd4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875cd4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 -- 172.21.15.16:0/3666132618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 msgr2=0x556875cd4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875cd4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 -- 172.21.15.16:0/3666132618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556875aa53b0 con 0x556875cd9e00 2024-05-08T23:23:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f12205700 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875cd4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.822+0000 7f1f11a04700 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cda1e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1f08015000 tx=0x7f1f0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cfb07e3f5cdad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1ef7fff700 1 -- 172.21.15.16:0/3666132618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800bca0 con 0x556875cd9e00 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1ef7fff700 1 -- 172.21.15.16:0/3666132618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f0801d040 con 0x556875cd9e00 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1ef7fff700 1 -- 172.21.15.16:0/3666132618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800f7b0 con 0x556875cd9e00 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/3666132618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 msgr2=0x556875cda1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cda1e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1f08015000 tx=0x7f1f0801f000 comp rx=0 tx=0).stop 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/3666132618 shutdown_connections 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875cd4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cda1e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/3666132618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875cd4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.823+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/3666132618 >> 172.21.15.16:0/3666132618 conn(0x556875c32210 msgr2=0x556875ccedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.824+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/3666132618 shutdown_connections 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.824+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/3666132618 wait complete. 2024-05-08T23:23:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.824+0000 7f1f1fe22b80 1 -- start start 2024-05-08T23:23:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.824+0000 7f1f1fe22b80 1 Processor -- start 2024-05-08T23:23:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd4890 0x556875cddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cde100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cde640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875bc0560 con 0x556875cd9e00 2024-05-08T23:23:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875bc0200 con 0x556875cda720 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875aa76a0 con 0x556875cd4890 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 10 client.0 _ll_register_callbacks cb 0x556875bc9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cde100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cde100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47680/0 (socket says 172.21.15.16:47680) 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 -- 172.21.15.16:0/1598529749 learned_addr learned my addr 172.21.15.16:0/1598529749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11a04700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd4890 0x556875cddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f12205700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cde640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 -- 172.21.15.16:0/1598529749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd4890 msgr2=0x556875cddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd4890 0x556875cddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 -- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 msgr2=0x556875cde640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cde640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 -- 172.21.15.16:0/1598529749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c10e650 con 0x556875cd9e00 2024-05-08T23:23:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f11203700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cde100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0964d0 tx=0x7f1efc00e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ac2a198b52aee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1ef57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1ef57fa700 1 -- 172.21.15.16:0/1598529749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0adea0 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1ef57fa700 1 -- 172.21.15.16:0/1598529749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1efc016530 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.825+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556875b7d270 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1ef57fa700 1 -- 172.21.15.16:0/1598529749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1efc099fd0 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 10 client.16389 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556875a51b60 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1ef57fa700 1 -- 172.21.15.16:0/1598529749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556875b7d270 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1ef57fa700 1 -- 172.21.15.16:0/1598529749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1efc094be0 con 0x556875cd9e00 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 20 client.16389 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875cee300 0x556875cf0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 4 client.16389 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556875ca3810 con 0x556875cee300 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.826+0000 7f1f11a04700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875cee300 0x556875cf0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.827+0000 7f1f11a04700 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875cee300 0x556875cf0720 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12b887abd6fcb9a server_cookie=c06bef465aa75219 in_seq=0 out_seq=0 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.827+0000 7f1ef57fa700 10 client.16389 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.827+0000 7f1ef57fa700 1 -- 172.21.15.16:0/1598529749 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556875ca3810 con 0x556875cee300 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.827+0000 7f1ef57fa700 10 client.16389 handle_command_reply: tid=0 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 client.16389 shutdown 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 msgr2=0x556875cde100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cde100 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0964d0 tx=0x7f1efc00e8f0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 shutdown_connections 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875cee300 0x556875cf0720 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cde640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1ef57fa700 0 client.16389 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd4890 0x556875cddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1598529749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875cde100 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 >> 172.21.15.16:0/1598529749 conn(0x556875c32210 msgr2=0x556875cce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 shutdown_connections 2024-05-08T23:23:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1598529749 wait complete. 2024-05-08T23:23:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.828+0000 7f1f1fe22b80 20 client.16389 trim_cache size 0 max 16384 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 Processor -- start 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 -- start start 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 0x556875cd93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd9e00 0x556875ce9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875cdf1c0 con 0x556875cd4890 2024-05-08T23:23:04.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875cf5930 con 0x556875cda720 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.846+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875cd0a20 con 0x556875cd9e00 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 0x556875cd93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 0x556875cd93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47686/0 (socket says 172.21.15.16:47686) 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 -- 172.21.15.16:0/1020993991 learned_addr learned my addr 172.21.15.16:0/1020993991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f12205700 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11203700 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd9e00 0x556875ce9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 -- 172.21.15.16:0/1020993991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd9e00 msgr2=0x556875ce9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd9e00 0x556875ce9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 -- 172.21.15.16:0/1020993991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 msgr2=0x556875cced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 -- 172.21.15.16:0/1020993991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556875bea560 con 0x556875cd4890 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f11a04700 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 0x556875cd93e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1f08004fb0 tx=0x7f1f08009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbc8e31f5470d62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f00da4700 1 -- 172.21.15.16:0/1020993991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0803dea0 con 0x556875cd4890 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f00da4700 1 -- 172.21.15.16:0/1020993991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f08004b20 con 0x556875cd4890 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.847+0000 7f1f00da4700 1 -- 172.21.15.16:0/1020993991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0803d780 con 0x556875cd4890 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1020993991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 msgr2=0x556875cd93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 0x556875cd93e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1f08004fb0 tx=0x7f1f08009cd0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1020993991 shutdown_connections 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cda720 0x556875cced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd4890 0x556875cd93e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/1020993991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cd9e00 0x556875ce9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1020993991 >> 172.21.15.16:0/1020993991 conn(0x556875c32210 msgr2=0x556875cd8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1020993991 shutdown_connections 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/1020993991 wait complete. 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 -- start start 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.848+0000 7f1f1fe22b80 1 Processor -- start 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875ce2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875ce3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875ce35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875ce5a20 con 0x556875cd9e00 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875d43800 con 0x556875cd4890 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556875d43940 con 0x556875cda720 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f1fe22b80 10 client.0 _ll_register_callbacks cb 0x556875bc9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f11203700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875ce3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f11203700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875ce3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47690/0 (socket says 172.21.15.16:47690) 2024-05-08T23:23:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f11203700 1 -- 172.21.15.16:0/2133407631 learned_addr learned my addr 172.21.15.16:0/2133407631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f11a04700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875ce2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.849+0000 7f1f12205700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875ce35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f11203700 1 -- 172.21.15.16:0/2133407631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 msgr2=0x556875ce35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f11203700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875ce35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f11203700 1 -- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 msgr2=0x556875ce2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f11203700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875ce2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f11203700 1 -- 172.21.15.16:0/2133407631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556875aa53b0 con 0x556875cd9e00 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f11203700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875ce3090 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1efc099cd0 tx=0x7f1efc0cd190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6479216b07747f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1ef57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1ef57fa700 1 -- 172.21.15.16:0/2133407631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0adea0 con 0x556875cd9e00 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1ef57fa700 1 -- 172.21.15.16:0/2133407631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1efc09a490 con 0x556875cd9e00 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f0801d040 con 0x556875cd9e00 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1ef57fa700 1 -- 172.21.15.16:0/2133407631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1efc099fd0 con 0x556875cd9e00 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f1fe22b80 10 client.16395 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556875aa9cd0 con 0x556875cd9e00 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1ef57fa700 1 -- 172.21.15.16:0/2133407631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f0801d040 con 0x556875cd9e00 2024-05-08T23:23:04.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1ef57fa700 1 -- 172.21.15.16:0/2133407631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1efc0a1fc0 con 0x556875cd9e00 2024-05-08T23:23:04.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.850+0000 7f1f1fe22b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:04.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:04.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 20 client.16395 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875d44bf0 0x556875d46fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 4 client.16395 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556875c1cec0 con 0x556875d44bf0 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f11a04700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875d44bf0 0x556875d46fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1ef57fa700 10 client.16395 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f11a04700 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875d44bf0 0x556875d46fe0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113086a6cceeb48f server_cookie=e0464c2ccd50828c in_seq=0 out_seq=0 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1ef57fa700 1 -- 172.21.15.16:0/2133407631 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556875c1cec0 con 0x556875d44bf0 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1ef57fa700 10 client.16395 handle_command_reply: tid=0 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 1 client.16395 shutdown 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 msgr2=0x556875ce3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.851+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875ce3090 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1efc099cd0 tx=0x7f1efc0cd190 comp rx=0 tx=0).stop 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 shutdown_connections 2024-05-08T23:23:04.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556875cda720 0x556875ce35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556875d44bf0 0x556875d46fe0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556875cd4890 0x556875ce2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 --2- 172.21.15.16:0/2133407631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556875cd9e00 0x556875ce3090 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1ef57fa700 0 client.16395 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 >> 172.21.15.16:0/2133407631 conn(0x556875c32210 msgr2=0x556875cde4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 shutdown_connections 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 1 -- 172.21.15.16:0/2133407631 wait complete. 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f1fe22b80 20 client.16395 trim_cache size 0 max 16384 2024-05-08T23:23:04.866 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "fe237874-8f70-44b9-b57c-80ce141ce5f8": { 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "fe237874-8f70-44b9-b57c-80ce141ce5f8", 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:23:04.867 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:23:04.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ef80a1290 msgr2=0x7f1ef80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ef80a1290 0x7f1ef80a3750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f08006010 tx=0x7f1f08019040 comp rx=0 tx=0).stop 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 msgr2=0x7f1f0c197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 0x7f1f0c197880 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e970 tx=0x7f1efc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 shutdown_connections 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1ef80a1290 0x7f1ef80a3750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f0c19bc20 0x7f1f0c19c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f0c0682e0 0x7f1f0c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 --2- 172.21.15.16:0/436796124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c106290 0x7f1f0c197880 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.852+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 >> 172.21.15.16:0/436796124 conn(0x7f1f0c076f40 msgr2=0x7f1f0c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.853+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 shutdown_connections 2024-05-08T23:23:04.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:04.853+0000 7f1f13c68700 1 -- 172.21.15.16:0/436796124 wait complete. 2024-05-08T23:23:04.869 INFO:tasks.cephfs.filesystem:scrub status for tag:fe237874-8f70-44b9-b57c-80ce141ce5f8 - {'path': '/', 'tag': 'fe237874-8f70-44b9-b57c-80ce141ce5f8', 'options': 'recursive,force'} 2024-05-08T23:23:05.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:05 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:23:05.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:05 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:05.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:05 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:23:05.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:05 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:05.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:05 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:23:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:05 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:23:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:05 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:05 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:23:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:05 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:05 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:23:05.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:05 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:23:05.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:05 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:05.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:05 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:23:05.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:05 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:05.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:05 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:23:06.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:06 smithi189 ceph-mon[46919]: pgmap v109: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-05-08T23:23:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:06 smithi155 ceph-mon[47105]: pgmap v109: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-05-08T23:23:06.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:06 smithi016 ceph-mon[42541]: pgmap v109: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-05-08T23:23:08.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:08 smithi016 ceph-mon[42541]: pgmap v110: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-05-08T23:23:08.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:08 smithi189 ceph-mon[46919]: pgmap v110: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-05-08T23:23:08.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:08 smithi155 ceph-mon[47105]: pgmap v110: 129 pgs: 129 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 25 op/s 2024-05-08T23:23:10.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:10 smithi016 ceph-mon[42541]: pgmap v111: 129 pgs: 129 active+clean; 2.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-05-08T23:23:10.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:10 smithi189 ceph-mon[46919]: pgmap v111: 129 pgs: 129 active+clean; 2.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-05-08T23:23:10.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:10 smithi155 ceph-mon[47105]: pgmap v111: 129 pgs: 129 active+clean; 2.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-05-08T23:23:12.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:12 smithi016 ceph-mon[42541]: pgmap v112: 129 pgs: 129 active+clean; 2.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 25 op/s 2024-05-08T23:23:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:12 smithi189 ceph-mon[46919]: pgmap v112: 129 pgs: 129 active+clean; 2.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 25 op/s 2024-05-08T23:23:12.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:12 smithi155 ceph-mon[47105]: pgmap v112: 129 pgs: 129 active+clean; 2.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 43 KiB/s wr, 25 op/s 2024-05-08T23:23:14.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:14 smithi016 ceph-mon[42541]: pgmap v113: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-08T23:23:14.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:14 smithi189 ceph-mon[46919]: pgmap v113: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-08T23:23:14.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:14 smithi155 ceph-mon[47105]: pgmap v113: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-08T23:23:16.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:16 smithi016 ceph-mon[42541]: pgmap v114: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:16.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:16 smithi189 ceph-mon[46919]: pgmap v114: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:16.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:16 smithi155 ceph-mon[47105]: pgmap v114: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:18 smithi016 ceph-mon[42541]: pgmap v115: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:18.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:18 smithi189 ceph-mon[46919]: pgmap v115: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:18.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:18 smithi155 ceph-mon[47105]: pgmap v115: 129 pgs: 129 active+clean; 3.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:20.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:20 smithi016 ceph-mon[42541]: pgmap v116: 129 pgs: 129 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:20.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:20 smithi189 ceph-mon[46919]: pgmap v116: 129 pgs: 129 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:20.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:20 smithi155 ceph-mon[47105]: pgmap v116: 129 pgs: 129 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:22 smithi016 ceph-mon[42541]: pgmap v117: 129 pgs: 129 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:22.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:22 smithi189 ceph-mon[46919]: pgmap v117: 129 pgs: 129 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:22.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:22 smithi155 ceph-mon[47105]: pgmap v117: 129 pgs: 129 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:24.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:24 smithi016 ceph-mon[42541]: pgmap v118: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:24 smithi189 ceph-mon[46919]: pgmap v118: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:24.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:24 smithi155 ceph-mon[47105]: pgmap v118: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:26.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:26 smithi016 ceph-mon[42541]: pgmap v119: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:26 smithi189 ceph-mon[46919]: pgmap v119: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:26.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:26 smithi155 ceph-mon[47105]: pgmap v119: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:28.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:28 smithi016 ceph-mon[42541]: pgmap v120: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:28.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:28 smithi189 ceph-mon[46919]: pgmap v120: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:28.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:28 smithi155 ceph-mon[47105]: pgmap v120: 129 pgs: 129 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:30.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:30 smithi016 ceph-mon[42541]: pgmap v121: 129 pgs: 129 active+clean; 3.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:30.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:30 smithi189 ceph-mon[46919]: pgmap v121: 129 pgs: 129 active+clean; 3.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:30.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:30 smithi155 ceph-mon[47105]: pgmap v121: 129 pgs: 129 active+clean; 3.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:32 smithi016 ceph-mon[42541]: pgmap v122: 129 pgs: 129 active+clean; 3.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:32.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:32 smithi189 ceph-mon[46919]: pgmap v122: 129 pgs: 129 active+clean; 3.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:32 smithi155 ceph-mon[47105]: pgmap v122: 129 pgs: 129 active+clean; 3.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:34.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:34 smithi016 ceph-mon[42541]: pgmap v123: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:34 smithi189 ceph-mon[46919]: pgmap v123: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:34 smithi155 ceph-mon[47105]: pgmap v123: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:34.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 -- 172.21.15.16:0/1531062692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c101160 msgr2=0x7f7a1c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 --2- 172.21.15.16:0/1531062692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c101160 0x7f7a1c101540 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009990 tx=0x7f7a0c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 -- 172.21.15.16:0/1531062692 shutdown_connections 2024-05-08T23:23:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 --2- 172.21.15.16:0/1531062692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c106280 0x7f7a1c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 --2- 172.21.15.16:0/1531062692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c101160 0x7f7a1c101540 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 --2- 172.21.15.16:0/1531062692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a1c101b10 0x7f7a1c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 -- 172.21.15.16:0/1531062692 >> 172.21.15.16:0/1531062692 conn(0x7f7a1c0fc9e0 msgr2=0x7f7a1c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 -- 172.21.15.16:0/1531062692 shutdown_connections 2024-05-08T23:23:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 -- 172.21.15.16:0/1531062692 wait complete. 2024-05-08T23:23:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 Processor -- start 2024-05-08T23:23:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.020+0000 7f7a226b0700 1 -- start start 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a226b0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a1c101160 0x7f7a1c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a226b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 0x7f7a1c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a226b0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c19bbe0 0x7f7a1c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a226b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c19c660 con 0x7f7a1c106280 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a226b0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c19c7d0 con 0x7f7a1c19bbe0 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a226b0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c19c990 con 0x7f7a1c101160 2024-05-08T23:23:35.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 0x7f7a1c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 0x7f7a1c197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35464/0 (socket says 172.21.15.16:35464) 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/238328624 learned_addr learned my addr 172.21.15.16:0/238328624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a1c101160 0x7f7a1c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a20c4d700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c19bbe0 0x7f7a1c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/238328624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a1c101160 msgr2=0x7f7a1c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a1c101160 0x7f7a1c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/238328624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c19bbe0 msgr2=0x7f7a1c19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c19bbe0 0x7f7a1c19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.021+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/238328624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0095f0 con 0x7f7a1c106280 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.022+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 0x7f7a1c197890 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000e970 tx=0x7f7a1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.022+0000 7f7a197fa700 1 -- 172.21.15.16:0/238328624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10009790 con 0x7f7a1c106280 2024-05-08T23:23:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.022+0000 7f7a197fa700 1 -- 172.21.15.16:0/238328624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1000c7e0 con 0x7f7a1c106280 2024-05-08T23:23:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.022+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1c1a3ef0 con 0x7f7a1c106280 2024-05-08T23:23:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.022+0000 7f7a197fa700 1 -- 172.21.15.16:0/238328624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10005440 con 0x7f7a1c106280 2024-05-08T23:23:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.022+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c1a4370 con 0x7f7a1c106280 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.024+0000 7f7a197fa700 1 -- 172.21.15.16:0/238328624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7a1000c950 con 0x7f7a1c106280 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.025+0000 7f7a197fa700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7a040a1290 0x7f7a040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.025+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7a040a1290 0x7f7a040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.025+0000 7f7a197fa700 1 -- 172.21.15.16:0/238328624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f7a10014070 con 0x7f7a1c106280 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.025+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7a040a1290 0x7f7a040a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00b2e0 tx=0x7f7a0c005040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 Processor -- start 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 -- start start 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x5602344581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 0x560234452350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234452890 0x560234452cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602343a9150 con 0x560234458720 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602343a6b10 con 0x560234452890 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602343a2400 con 0x560234457e00 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.026+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 0x560234452350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x5602344581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 0x560234452350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35476/0 (socket says 172.21.15.16:35476) 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/2553516565 learned_addr learned my addr 172.21.15.16:0/2553516565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a20c4d700 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234452890 0x560234452cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/2553516565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 msgr2=0x5602344581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x5602344581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/2553516565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234452890 msgr2=0x560234452cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234452890 0x560234452cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/2553516565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602342233b0 con 0x560234458720 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 0x560234452350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7a10096a40 tx=0x7f7a1000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b719c4251f8430 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a027fc700 1 -- 172.21.15.16:0/2553516565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100adea0 con 0x560234458720 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a027fc700 1 -- 172.21.15.16:0/2553516565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1009de00 con 0x560234458720 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.027+0000 7f7a027fc700 1 -- 172.21.15.16:0/2553516565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10016330 con 0x560234458720 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/2553516565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 msgr2=0x560234452350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 0x560234452350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7a10096a40 tx=0x7f7a1000ccd0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/2553516565 shutdown_connections 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234452890 0x560234452cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x5602344581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/2553516565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234458720 0x560234452350 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/2553516565 >> 172.21.15.16:0/2553516565 conn(0x5602343b0210 msgr2=0x56023444cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/2553516565 shutdown_connections 2024-05-08T23:23:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/2553516565 wait complete. 2024-05-08T23:23:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.028+0000 7f7a2e86ab80 1 -- start start 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 Processor -- start 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 0x56023445bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x56023445c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023433e560 con 0x560234452890 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023433e200 con 0x560234458720 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602342256a0 con 0x560234457e00 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a2e86ab80 10 client.0 _ll_register_callbacks cb 0x560234347b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 0x56023445bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 0x56023445bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35478/0 (socket says 172.21.15.16:35478) 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 -- 172.21.15.16:0/3350855199 learned_addr learned my addr 172.21.15.16:0/3350855199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a20c4d700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x56023445c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 -- 172.21.15.16:0/3350855199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 msgr2=0x56023445c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x56023445c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 -- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 msgr2=0x56023445c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.029+0000 7f7a1bfff700 1 -- 172.21.15.16:0/3350855199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c1a4370 con 0x560234452890 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 0x56023445bae0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009990 tx=0x7f7a0c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5492b23583d7080 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f79fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f79fb7fe700 1 -- 172.21.15.16:0/3350855199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c004cf0 con 0x560234452890 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f79fb7fe700 1 -- 172.21.15.16:0/3350855199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f7a0c004e50 con 0x560234452890 2024-05-08T23:23:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602342fb270 con 0x560234452890 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f79fb7fe700 1 -- 172.21.15.16:0/3350855199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a0c00f460 con 0x560234452890 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a2e86ab80 10 client.16407 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5602341cfb60 con 0x560234452890 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f79fb7fe700 1 -- 172.21.15.16:0/3350855199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5602342fb270 con 0x560234452890 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f79fb7fe700 1 -- 172.21.15.16:0/3350855199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a0c02f070 con 0x560234452890 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a2e86ab80 10 client.16407 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a2e86ab80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.030+0000 7f7a2e86ab80 20 client.16407 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.031+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56023446c240 0x56023446e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.031+0000 7f7a2e86ab80 4 client.16407 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.031+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560234421810 con 0x56023446c240 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.031+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56023446c240 0x56023446e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.031+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56023446c240 0x56023446e660 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750a21a28425a0c5 server_cookie=f938e44927476b86 in_seq=0 out_seq=0 2024-05-08T23:23:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.031+0000 7f79fb7fe700 10 client.16407 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f79fb7fe700 1 -- 172.21.15.16:0/3350855199 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560234421810 con 0x56023446c240 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f79fb7fe700 10 client.16407 handle_command_reply: tid=0 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 client.16407 shutdown 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 msgr2=0x56023445bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 0x56023445bae0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009990 tx=0x7f7a0c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 shutdown_connections 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56023446c240 0x56023446e660 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234452890 0x56023445bae0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f79fb7fe700 0 client.16407 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3350855199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234457e00 0x56023445c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 >> 172.21.15.16:0/3350855199 conn(0x5602343b0210 msgr2=0x56023444c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 shutdown_connections 2024-05-08T23:23:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3350855199 wait complete. 2024-05-08T23:23:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.032+0000 7f7a2e86ab80 20 client.16407 trim_cache size 0 max 16384 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 Processor -- start 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 -- start start 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x560234452350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023444cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x5602344572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560234449dc0 con 0x560234457e00 2024-05-08T23:23:35.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560234458560 con 0x560234458720 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560234473830 con 0x560234452890 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023444cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023444cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35488/0 (socket says 172.21.15.16:35488) 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/3631014039 learned_addr learned my addr 172.21.15.16:0/3631014039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x560234452350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a20c4d700 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x5602344572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/3631014039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 msgr2=0x560234452350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x560234452350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/3631014039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 msgr2=0x5602344572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x5602344572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/3631014039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560234368560 con 0x560234457e00 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.051+0000 7f7a20c4d700 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x5602344572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:35.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023444cd20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000cc00 tx=0x7f7a1000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1496a8e10c2de36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a02ffd700 1 -- 172.21.15.16:0/3631014039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a10063070 con 0x560234457e00 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a02ffd700 1 -- 172.21.15.16:0/3631014039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a100983f0 con 0x560234457e00 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a02ffd700 1 -- 172.21.15.16:0/3631014039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100adea0 con 0x560234457e00 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3631014039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 msgr2=0x56023444cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023444cd20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000cc00 tx=0x7f7a1000cf50 comp rx=0 tx=0).stop 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3631014039 shutdown_connections 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x5602344572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x560234452350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/3631014039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023444cd20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3631014039 >> 172.21.15.16:0/3631014039 conn(0x5602343b0210 msgr2=0x560234456bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.052+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3631014039 shutdown_connections 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/3631014039 wait complete. 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 -- start start 2024-05-08T23:23:35.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 Processor -- start 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x56023445f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023445f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602344c1800 con 0x560234457e00 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602344c1970 con 0x560234458720 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.053+0000 7f7a2e86ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602344c1b30 con 0x560234452890 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a2e86ab80 10 client.0 _ll_register_callbacks cb 0x560234347b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023445f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023445f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35498/0 (socket says 172.21.15.16:35498) 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/1741314298 learned_addr learned my addr 172.21.15.16:0/1741314298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x56023445f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a20c4d700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/1741314298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 msgr2=0x56023445f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x56023445f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 msgr2=0x56023445fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 -- 172.21.15.16:0/1741314298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602342233b0 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a1b7fe700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023445f680 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7a100cc010 tx=0x7f7a100cd160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8761b54d220725ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f79fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f79fb7fe700 1 -- 172.21.15.16:0/1741314298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a100b7070 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f79fb7fe700 1 -- 172.21.15.16:0/1741314298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f7a10064070 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a1009de00 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f79fb7fe700 1 -- 172.21.15.16:0/1741314298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a10063030 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a2e86ab80 10 client.16413 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.054+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560234227cd0 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f79fb7fe700 1 -- 172.21.15.16:0/1741314298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a1009de00 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f79fb7fe700 1 -- 172.21.15.16:0/1741314298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a10094be0 con 0x560234457e00 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a2e86ab80 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a2e86ab80 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a2e86ab80 20 client.16413 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5602344c2dd0 0x5602344c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a2e86ab80 4 client.16413 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56023439aec0 con 0x5602344c2dd0 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.055+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5602344c2dd0 0x5602344c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f79fb7fe700 10 client.16413 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a1bfff700 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5602344c2dd0 0x5602344c51c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2495c62637b161cf server_cookie=cb67e60e76c54cd3 in_seq=0 out_seq=0 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f79fb7fe700 1 -- 172.21.15.16:0/1741314298 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56023439aec0 con 0x5602344c2dd0 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f79fb7fe700 10 client.16413 handle_command_reply: tid=0 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 client.16413 shutdown 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 msgr2=0x56023445f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023445f680 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f7a100cc010 tx=0x7f7a100cd160 comp rx=0 tx=0).stop 2024-05-08T23:23:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 shutdown_connections 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5602344c2dd0 0x5602344c51c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560234458720 0x56023445fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560234452890 0x56023445f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 --2- 172.21.15.16:0/1741314298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560234457e00 0x56023445f680 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 >> 172.21.15.16:0/1741314298 conn(0x5602343b0210 msgr2=0x560234457460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f79fb7fe700 0 client.16413 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 shutdown_connections 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 1 -- 172.21.15.16:0/1741314298 wait complete. 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.056+0000 7f7a2e86ab80 20 client.16413 trim_cache size 0 max 16384 2024-05-08T23:23:35.063 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:23:35.064 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:23:35.064 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:23:35.064 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7a040a1290 msgr2=0x7f7a040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7a040a1290 0x7f7a040a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00b2e0 tx=0x7f7a0c005040 comp rx=0 tx=0).stop 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 msgr2=0x7f7a1c197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 0x7f7a1c197890 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000e970 tx=0x7f7a1000ec80 comp rx=0 tx=0).stop 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 shutdown_connections 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7a040a1290 0x7f7a040a3750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a1c19bbe0 0x7f7a1c19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a1c101160 0x7f7a1c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 --2- 172.21.15.16:0/238328624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a1c106280 0x7f7a1c197890 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 >> 172.21.15.16:0/238328624 conn(0x7f7a1c0fc9e0 msgr2=0x7f7a1c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 shutdown_connections 2024-05-08T23:23:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.057+0000 7f7a226b0700 1 -- 172.21.15.16:0/238328624 wait complete. 2024-05-08T23:23:35.070 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:23:35.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:23:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/1707568795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 msgr2=0x7fe2d0100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/1707568795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0100320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009990 tx=0x7fe2c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/1707568795 shutdown_connections 2024-05-08T23:23:35.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/1707568795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2d0105060 0x7fe2d010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/1707568795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0100320 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/1707568795 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d01008f0 0x7fe2d0104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/1707568795 >> 172.21.15.16:0/1707568795 conn(0x7fe2d00fb7e0 msgr2=0x7fe2d00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/1707568795 shutdown_connections 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.207+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/1707568795 wait complete. 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 Processor -- start 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 -- start start 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2d0105060 0x7fe2d0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d019de10 0x7fe2d019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d019e890 con 0x7fe2d00fff40 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d019ea00 con 0x7fe2d019de10 2024-05-08T23:23:35.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.208+0000 7fe2d4ea2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d019ebc0 con 0x7fe2d0105060 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35528/0 (socket says 172.21.15.16:35528) 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 -- 172.21.15.16:0/4257445840 learned_addr learned my addr 172.21.15.16:0/4257445840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2cdd9b700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2d0105060 0x7fe2d0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ced9d700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d019de10 0x7fe2d019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 -- 172.21.15.16:0/4257445840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2d0105060 msgr2=0x7fe2d0199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2d0105060 0x7fe2d0199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 -- 172.21.15.16:0/4257445840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d019de10 msgr2=0x7fe2d019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d019de10 0x7fe2d019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 -- 172.21.15.16:0/4257445840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00095f0 con 0x7fe2d00fff40 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2ce59c700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0199530 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0004730 tx=0x7fe2c0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c001d070 con 0x7fe2d00fff40 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2c000bea0 con 0x7fe2d00fff40 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d01a60c0 con 0x7fe2d00fff40 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00173f0 con 0x7fe2d00fff40 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.209+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01a6540 con 0x7fe2d00fff40 2024-05-08T23:23:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.211+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d01030a0 con 0x7fe2d00fff40 2024-05-08T23:23:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.212+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe2c00226e0 con 0x7fe2d00fff40 2024-05-08T23:23:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.213+0000 7fe2bf7fe700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe2b80a1310 0x7fe2b80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.213+0000 7fe2cdd9b700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe2b80a1310 0x7fe2b80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.213+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe2c00ce420 con 0x7fe2d00fff40 2024-05-08T23:23:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.213+0000 7fe2cdd9b700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe2b80a1310 0x7fe2b80a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2d019aa80 tx=0x7fe2c40073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.217+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe2c0097660 con 0x7fe2d00fff40 2024-05-08T23:23:35.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.487+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2d0101410 con 0x7fe2d00fff40 2024-05-08T23:23:35.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.488+0000 7fe2bf7fe700 1 -- 172.21.15.16:0/4257445840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fe2c009ac80 con 0x7fe2d00fff40 2024-05-08T23:23:35.488 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:23:35.488 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:23:35.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe2b80a1310 msgr2=0x7fe2b80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe2b80a1310 0x7fe2b80a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2d019aa80 tx=0x7fe2c40073d0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 msgr2=0x7fe2d0199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0199530 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0004730 tx=0x7fe2c0004760 comp rx=0 tx=0).stop 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 shutdown_connections 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe2b80a1310 0x7fe2b80a37d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe2d019de10 0x7fe2d019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2d00fff40 0x7fe2d0199530 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 --2- 172.21.15.16:0/4257445840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2d0105060 0x7fe2d0199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.490+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 >> 172.21.15.16:0/4257445840 conn(0x7fe2d00fb7e0 msgr2=0x7fe2d00fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.491+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 shutdown_connections 2024-05-08T23:23:35.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.491+0000 7fe2d4ea2700 1 -- 172.21.15.16:0/4257445840 wait complete. 2024-05-08T23:23:35.492 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:23:35.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 -- 172.21.15.16:0/2478493919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668100060 msgr2=0x7f2668100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 --2- 172.21.15.16:0/2478493919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668100060 0x7f2668100440 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2658009990 tx=0x7f2658009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 -- 172.21.15.16:0/2478493919 shutdown_connections 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 --2- 172.21.15.16:0/2478493919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2668105160 0x7f266810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 --2- 172.21.15.16:0/2478493919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668100060 0x7f2668100440 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 --2- 172.21.15.16:0/2478493919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2668100a10 0x7f2668104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 -- 172.21.15.16:0/2478493919 >> 172.21.15.16:0/2478493919 conn(0x7f2668076f40 msgr2=0x7f2668077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 -- 172.21.15.16:0/2478493919 shutdown_connections 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.644+0000 7f266e7f9700 1 -- 172.21.15.16:0/2478493919 wait complete. 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 Processor -- start 2024-05-08T23:23:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 -- start start 2024-05-08T23:23:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2668100060 0x7f2668197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 0x7f26681978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266819bc40 0x7f266819c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266819c6c0 con 0x7f2668105160 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266819c830 con 0x7f266819bc40 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.645+0000 7f266e7f9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266819c9f0 con 0x7f2668100060 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 0x7f26681978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 0x7f26681978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35542/0 (socket says 172.21.15.16:35542) 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 -- 172.21.15.16:0/1960152306 learned_addr learned my addr 172.21.15.16:0/1960152306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f2667fff700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2668100060 0x7f2668197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f266cd96700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266819bc40 0x7f266819c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 -- 172.21.15.16:0/1960152306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2668100060 msgr2=0x7f2668197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2668100060 0x7f2668197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 -- 172.21.15.16:0/1960152306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266819bc40 msgr2=0x7f266819c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266819bc40 0x7f266819c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 -- 172.21.15.16:0/1960152306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26580095f0 con 0x7f2668105160 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26677fe700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 0x7f26681978a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b550 tx=0x7f265c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26657fa700 1 -- 172.21.15.16:0/1960152306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c011830 con 0x7f2668105160 2024-05-08T23:23:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26657fa700 1 -- 172.21.15.16:0/1960152306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f265c011e20 con 0x7f2668105160 2024-05-08T23:23:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26681a3f50 con 0x7f2668105160 2024-05-08T23:23:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f26657fa700 1 -- 172.21.15.16:0/1960152306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c010dd0 con 0x7f2668105160 2024-05-08T23:23:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.646+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26681a4370 con 0x7f2668105160 2024-05-08T23:23:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.649+0000 7f26657fa700 1 -- 172.21.15.16:0/1960152306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f265c011990 con 0x7f2668105160 2024-05-08T23:23:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.649+0000 7f26657fa700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26500a1290 0x7f26500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.650+0000 7f2667fff700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26500a1290 0x7f26500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.650+0000 7f26657fa700 1 -- 172.21.15.16:0/1960152306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f265c0cc990 con 0x7f2668105160 2024-05-08T23:23:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.650+0000 7f2667fff700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26500a1290 0x7f26500a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2658006010 tx=0x7f265800b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 Processor -- start 2024-05-08T23:23:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 -- start start 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55566291ae00 0x55566291b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x555662915350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555662915890 0x555662915cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55566286c150 con 0x55566291b720 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662869b10 con 0x55566291ae00 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662865400 con 0x555662915890 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x555662915350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x555662915350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35552/0 (socket says 172.21.15.16:35552) 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 -- 172.21.15.16:0/3067868187 learned_addr learned my addr 172.21.15.16:0/3067868187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f2667fff700 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55566291ae00 0x55566291b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f266cd96700 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555662915890 0x555662915cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 -- 172.21.15.16:0/3067868187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555662915890 msgr2=0x555662915cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555662915890 0x555662915cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 -- 172.21.15.16:0/3067868187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55566291ae00 msgr2=0x55566291b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55566291ae00 0x55566291b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f26677fe700 1 -- 172.21.15.16:0/3067868187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556626e63b0 con 0x55566291b720 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.651+0000 7f266cd96700 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555662915890 0x555662915cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.652+0000 7f26677fe700 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x555662915350 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f265c0108c0 tx=0x7f265c095c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4964842ea6b352d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.652+0000 7f264e7fc700 1 -- 172.21.15.16:0/3067868187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0991b0 con 0x55566291b720 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.652+0000 7f264e7fc700 1 -- 172.21.15.16:0/3067868187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f265c010510 con 0x55566291b720 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.652+0000 7f264e7fc700 1 -- 172.21.15.16:0/3067868187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0b9310 con 0x55566291b720 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.652+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3067868187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 msgr2=0x555662915350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.652+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x555662915350 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f265c0108c0 tx=0x7f265c095c80 comp rx=0 tx=0).stop 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3067868187 shutdown_connections 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555662915890 0x555662915cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55566291ae00 0x55566291b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3067868187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x555662915350 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3067868187 >> 172.21.15.16:0/3067868187 conn(0x555662873210 msgr2=0x55566290fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3067868187 shutdown_connections 2024-05-08T23:23:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3067868187 wait complete. 2024-05-08T23:23:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 -- start start 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.653+0000 7f267a9b3b80 1 Processor -- start 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566291eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291ae00 0x55566291f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x55566291f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662801560 con 0x55566291b720 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662801200 con 0x555662915890 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556626e86a0 con 0x55566291ae00 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f267a9b3b80 10 client.0 _ll_register_callbacks cb 0x55566280ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x55566291f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x55566291f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35556/0 (socket says 172.21.15.16:35556) 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 -- 172.21.15.16:0/2816538497 learned_addr learned my addr 172.21.15.16:0/2816538497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f26677fe700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291ae00 0x55566291f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f2667fff700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566291eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 -- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291ae00 msgr2=0x55566291f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291ae00 0x55566291f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 -- 172.21.15.16:0/2816538497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 msgr2=0x55566291eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566291eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 -- 172.21.15.16:0/2816538497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26681a4370 con 0x55566291b720 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f26677fe700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291ae00 0x55566291f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f266cd96700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x55566291f5c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2660009e90 tx=0x7f2660007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cced20f694768a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f26477fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.654+0000 7f26477fe700 1 -- 172.21.15.16:0/2816538497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011930 con 0x55566291b720 2024-05-08T23:23:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556627be270 con 0x55566291b720 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f26477fe700 1 -- 172.21.15.16:0/2816538497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2660011a90 con 0x55566291b720 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f26477fe700 1 -- 172.21.15.16:0/2816538497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2660011c50 con 0x55566291b720 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 10 client.16431 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555662692b60 con 0x55566291b720 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f26477fe700 1 -- 172.21.15.16:0/2816538497 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5556627be270 con 0x55566291b720 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f26477fe700 1 -- 172.21.15.16:0/2816538497 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f266001f500 con 0x55566291b720 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 20 client.16431 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55566292f340 0x555662931760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 4 client.16431 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556628e4810 con 0x55566292f340 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.655+0000 7f2667fff700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55566292f340 0x555662931760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.656+0000 7f26477fe700 10 client.16431 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.656+0000 7f2667fff700 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55566292f340 0x555662931760 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d1a0c18fdb90c33 server_cookie=6415b5550a12987a in_seq=0 out_seq=0 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f26477fe700 1 -- 172.21.15.16:0/2816538497 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556628e4810 con 0x55566292f340 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f26477fe700 10 client.16431 handle_command_reply: tid=0 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 client.16431 shutdown 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 msgr2=0x55566291f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x55566291f5c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2660009e90 tx=0x7f2660007e80 comp rx=0 tx=0).stop 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 shutdown_connections 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291b720 0x55566291f5c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55566292f340 0x555662931760 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566291eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/2816538497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291ae00 0x55566291f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f26477fe700 0 client.16431 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 >> 172.21.15.16:0/2816538497 conn(0x555662873210 msgr2=0x55566290f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 shutdown_connections 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 1 -- 172.21.15.16:0/2816538497 wait complete. 2024-05-08T23:23:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.657+0000 7f267a9b3b80 20 client.16431 trim_cache size 0 max 16384 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 Processor -- start 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 -- start start 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566290fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662926160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x5556629266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662921b10 con 0x55566291ae00 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55566290cdc0 con 0x555662915890 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.671+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662920130 con 0x55566291b720 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662926160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x5556629266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662926160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35566/0 (socket says 172.21.15.16:35566) 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f26677fe700 1 -- 172.21.15.16:0/4127302773 learned_addr learned my addr 172.21.15.16:0/4127302773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f2667fff700 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566290fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 -- 172.21.15.16:0/4127302773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 msgr2=0x55566290fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566290fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 -- 172.21.15.16:0/4127302773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 msgr2=0x555662926160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662926160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 -- 172.21.15.16:0/4127302773 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55566282b560 con 0x55566291b720 2024-05-08T23:23:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f266cd96700 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x5556629266a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2660009010 tx=0x7f2660009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e829a9716dc3cf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f264effd700 1 -- 172.21.15.16:0/4127302773 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266001f670 con 0x55566291b720 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f264effd700 1 -- 172.21.15.16:0/4127302773 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f266000f070 con 0x55566291b720 2024-05-08T23:23:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.672+0000 7f264effd700 1 -- 172.21.15.16:0/4127302773 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266001c710 con 0x55566291b720 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 -- 172.21.15.16:0/4127302773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 msgr2=0x5556629266a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x5556629266a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2660009010 tx=0x7f2660009190 comp rx=0 tx=0).stop 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 -- 172.21.15.16:0/4127302773 shutdown_connections 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x5556629266a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x55566290fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/4127302773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662926160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 -- 172.21.15.16:0/4127302773 >> 172.21.15.16:0/4127302773 conn(0x555662873210 msgr2=0x55566290c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 -- 172.21.15.16:0/4127302773 shutdown_connections 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 -- 172.21.15.16:0/4127302773 wait complete. 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.673+0000 7f267a9b3b80 1 -- start start 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 Processor -- start 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x555662922140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662922680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x555662922bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556629848c0 con 0x55566291ae00 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662984a00 con 0x555662915890 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555662984bc0 con 0x55566291b720 2024-05-08T23:23:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f267a9b3b80 10 client.0 _ll_register_callbacks cb 0x55566280ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f2667fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x555662922140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f266cd96700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x555662922bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f26677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662922680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f2667fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x555662922140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:39110/0 (socket says 172.21.15.16:39110) 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.674+0000 7f2667fff700 1 -- 172.21.15.16:0/3220009849 learned_addr learned my addr 172.21.15.16:0/3220009849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f2667fff700 1 -- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 msgr2=0x555662922bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f2667fff700 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x555662922bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f2667fff700 1 -- 172.21.15.16:0/3220009849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 msgr2=0x555662922680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f2667fff700 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662922680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f2667fff700 1 -- 172.21.15.16:0/3220009849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555662692b60 con 0x555662915890 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f266cd96700 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x555662922bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f2667fff700 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x555662922140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2658000c00 tx=0x7f265801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a516160c05c2c48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f26477fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f26477fe700 1 -- 172.21.15.16:0/3220009849 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265802b040 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f265c010510 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f26477fe700 1 -- 172.21.15.16:0/3220009849 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f2658036e40 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f26477fe700 1 -- 172.21.15.16:0/3220009849 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2658036560 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f267a9b3b80 10 client.26171 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.675+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5556626eacd0 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f26477fe700 1 -- 172.21.15.16:0/3220009849 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f265c010510 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f26477fe700 1 -- 172.21.15.16:0/3220009849 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f265800f460 con 0x555662915890 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f267a9b3b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f267a9b3b80 10 client.26171 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f267a9b3b80 20 client.26171 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x555662986240 0x555662988660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f267a9b3b80 4 client.26171 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55566285dec0 con 0x555662986240 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.676+0000 7f26677fe700 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x555662986240 0x555662988660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f26477fe700 10 client.26171 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f26677fe700 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x555662986240 0x555662988660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82fbb827ac01c3d1 server_cookie=c0b98b0ba7cc6a00 in_seq=0 out_seq=0 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f26477fe700 1 -- 172.21.15.16:0/3220009849 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55566285dec0 con 0x555662986240 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f26477fe700 10 client.26171 handle_command_reply: tid=0 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 client.26171 shutdown 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 msgr2=0x555662922140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x555662922140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2658000c00 tx=0x7f265801a050 comp rx=0 tx=0).stop 2024-05-08T23:23:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 shutdown_connections 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55566291b720 0x555662922bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x555662986240 0x555662988660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555662915890 0x555662922140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 --2- 172.21.15.16:0/3220009849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55566291ae00 0x555662922680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f26477fe700 0 client.26171 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 >> 172.21.15.16:0/3220009849 conn(0x555662873210 msgr2=0x555662918ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 shutdown_connections 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 1 -- 172.21.15.16:0/3220009849 wait complete. 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.677+0000 7f267a9b3b80 20 client.26171 trim_cache size 0 max 16384 2024-05-08T23:23:35.683 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:23:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26500a1290 msgr2=0x7f26500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26500a1290 0x7f26500a3750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2658006010 tx=0x7f265800b540 comp rx=0 tx=0).stop 2024-05-08T23:23:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 msgr2=0x7f26681978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 0x7f26681978a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f265c00b550 tx=0x7f265c00b860 comp rx=0 tx=0).stop 2024-05-08T23:23:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 shutdown_connections 2024-05-08T23:23:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26500a1290 0x7f26500a3750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266819bc40 0x7f266819c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2668100060 0x7f2668197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 --2- 172.21.15.16:0/1960152306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2668105160 0x7f26681978a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 >> 172.21.15.16:0/1960152306 conn(0x7f2668076f40 msgr2=0x7f26680684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 shutdown_connections 2024-05-08T23:23:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.678+0000 7f266e7f9700 1 -- 172.21.15.16:0/1960152306 wait complete. 2024-05-08T23:23:35.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 -- 172.21.15.16:0/2810586503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100a10 msgr2=0x7faf30104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 --2- 172.21.15.16:0/2810586503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100a10 0x7faf30104a30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009990 tx=0x7faf2c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 -- 172.21.15.16:0/2810586503 shutdown_connections 2024-05-08T23:23:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 --2- 172.21.15.16:0/2810586503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf30105160 0x7faf3010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 --2- 172.21.15.16:0/2810586503 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30100060 0x7faf30100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 --2- 172.21.15.16:0/2810586503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100a10 0x7faf30104a30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 -- 172.21.15.16:0/2810586503 >> 172.21.15.16:0/2810586503 conn(0x7faf30076f40 msgr2=0x7faf30077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 -- 172.21.15.16:0/2810586503 shutdown_connections 2024-05-08T23:23:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.830+0000 7faf37d18700 1 -- 172.21.15.16:0/2810586503 wait complete. 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.831+0000 7faf37d18700 1 Processor -- start 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.831+0000 7faf37d18700 1 -- start start 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.831+0000 7faf37d18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 0x7faf30196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf37d18700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30105160 0x7faf30196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf37d18700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf3019ab00 0x7faf3019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf37d18700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3019b580 con 0x7faf30100060 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf37d18700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3019b6f0 con 0x7faf30105160 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf37d18700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3019b8b0 con 0x7faf3019ab00 2024-05-08T23:23:35.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 0x7faf30196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 0x7faf30196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35596/0 (socket says 172.21.15.16:35596) 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 -- 172.21.15.16:0/191261033 learned_addr learned my addr 172.21.15.16:0/191261033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf352b3700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30105160 0x7faf30196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf362b5700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf3019ab00 0x7faf3019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 -- 172.21.15.16:0/191261033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf3019ab00 msgr2=0x7faf3019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf3019ab00 0x7faf3019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 -- 172.21.15.16:0/191261033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30105160 msgr2=0x7faf30196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30105160 0x7faf30196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 -- 172.21.15.16:0/191261033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0095f0 con 0x7faf30100060 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf352b3700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30105160 0x7faf30196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf35ab4700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 0x7faf30196220 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faf24009970 tx=0x7faf24009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf22ffd700 1 -- 172.21.15.16:0/191261033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24012070 con 0x7faf30100060 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf22ffd700 1 -- 172.21.15.16:0/191261033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf2400d260 con 0x7faf30100060 2024-05-08T23:23:35.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf30068050 con 0x7faf30100060 2024-05-08T23:23:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.832+0000 7faf22ffd700 1 -- 172.21.15.16:0/191261033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24017930 con 0x7faf30100060 2024-05-08T23:23:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.833+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf300685b0 con 0x7faf30100060 2024-05-08T23:23:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.835+0000 7faf22ffd700 1 -- 172.21.15.16:0/191261033 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faf24017a90 con 0x7faf30100060 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.836+0000 7faf22ffd700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7faf1c0a1310 0x7faf1c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.836+0000 7faf352b3700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7faf1c0a1310 0x7faf1c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.836+0000 7faf22ffd700 1 -- 172.21.15.16:0/191261033 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7faf240ce260 con 0x7faf30100060 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.836+0000 7faf43ed2b80 1 Processor -- start 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.836+0000 7faf43ed2b80 1 -- start start 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf352b3700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7faf1c0a1310 0x7faf1c0a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006010 tx=0x7faf2c00b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c919661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c91960350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c91960cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c918b7150 con 0x557c91965e00 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c918b4b10 con 0x557c91966720 2024-05-08T23:23:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c918b0400 con 0x557c91960890 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c919661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c919661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35610/0 (socket says 172.21.15.16:35610) 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 -- 172.21.15.16:0/1521192831 learned_addr learned my addr 172.21.15.16:0/1521192831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf352b3700 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c91960350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf362b5700 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c91960cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 -- 172.21.15.16:0/1521192831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 msgr2=0x557c91960cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c91960cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 -- 172.21.15.16:0/1521192831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 msgr2=0x557c91960350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c91960350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 -- 172.21.15.16:0/1521192831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c917313b0 con 0x557c91965e00 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf362b5700 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c91960cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.837+0000 7faf35ab4700 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c919661e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7faf24097a40 tx=0x7faf2409abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69be28f6dfeff11c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.838+0000 7faf1bfff700 1 -- 172.21.15.16:0/1521192831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf240aeea0 con 0x557c91965e00 2024-05-08T23:23:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.838+0000 7faf1bfff700 1 -- 172.21.15.16:0/1521192831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf2409ee00 con 0x557c91965e00 2024-05-08T23:23:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.838+0000 7faf1bfff700 1 -- 172.21.15.16:0/1521192831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2409b490 con 0x557c91965e00 2024-05-08T23:23:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.838+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1521192831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 msgr2=0x557c919661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.838+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c919661e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7faf24097a40 tx=0x7faf2409abe0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1521192831 shutdown_connections 2024-05-08T23:23:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c91960cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c919661e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1521192831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c91960350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1521192831 >> 172.21.15.16:0/1521192831 conn(0x557c918be210 msgr2=0x557c9195adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1521192831 shutdown_connections 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1521192831 wait complete. 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 -- start start 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.839+0000 7faf43ed2b80 1 Processor -- start 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91960890 0x557c91969b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c9196a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9184c560 con 0x557c91965e00 2024-05-08T23:23:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9184c200 con 0x557c91960890 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c917336a0 con 0x557c91966720 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf43ed2b80 10 client.0 _ll_register_callbacks cb 0x557c91855b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35614/0 (socket says 172.21.15.16:35614) 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 -- 172.21.15.16:0/3317357794 learned_addr learned my addr 172.21.15.16:0/3317357794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf362b5700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c9196a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 -- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 msgr2=0x557c9196a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c9196a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 -- 172.21.15.16:0/3317357794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91960890 msgr2=0x557c91969b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91960890 0x557c91969b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91960890 0x557c91969b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 -- 172.21.15.16:0/3317357794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf300685b0 con 0x557c91965e00 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91960890 0x557c91969b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf352b3700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196a0a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7faf2c015000 tx=0x7faf2c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cf7e06f3796df3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf197fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.840+0000 7faf197fa700 1 -- 172.21.15.16:0/3317357794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c004910 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf197fa700 1 -- 172.21.15.16:0/3317357794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7faf2c004a70 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c91809270 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf197fa700 1 -- 172.21.15.16:0/3317357794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf2c005940 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 10 client.16449 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c916ddb60 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf197fa700 1 -- 172.21.15.16:0/3317357794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c91809270 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf197fa700 1 -- 172.21.15.16:0/3317357794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf2c031440 con 0x557c91965e00 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 20 client.16449 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c9197a210 0x557c9197c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 4 client.16449 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c9192f810 con 0x557c9197a210 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.841+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c9197a210 0x557c9197c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.842+0000 7faf197fa700 10 client.16449 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.842+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c9197a210 0x557c9197c630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff938e22626695d server_cookie=b6f4839c0209a80 in_seq=0 out_seq=0 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf197fa700 1 -- 172.21.15.16:0/3317357794 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c9192f810 con 0x557c9197a210 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf197fa700 10 client.16449 handle_command_reply: tid=0 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 client.16449 shutdown 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 msgr2=0x557c9196a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196a0a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7faf2c015000 tx=0x7faf2c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 shutdown_connections 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c9196a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c9197a210 0x557c9197c630 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91960890 0x557c91969b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3317357794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196a0a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf197fa700 0 client.16449 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 >> 172.21.15.16:0/3317357794 conn(0x557c918be210 msgr2=0x557c9195a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 shutdown_connections 2024-05-08T23:23:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3317357794 wait complete. 2024-05-08T23:23:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.843+0000 7faf43ed2b80 20 client.16449 trim_cache size 0 max 16384 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.857+0000 7faf43ed2b80 1 Processor -- start 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.857+0000 7faf43ed2b80 1 -- start start 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.857+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 0x557c9195ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91965e00 0x557c9196a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c91964c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c91967b10 con 0x557c91960890 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9195ba90 con 0x557c91965e00 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c919680b0 con 0x557c91966720 2024-05-08T23:23:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 0x557c9195ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 0x557c9195ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35622/0 (socket says 172.21.15.16:35622) 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 -- 172.21.15.16:0/3183295359 learned_addr learned my addr 172.21.15.16:0/3183295359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 -- 172.21.15.16:0/3183295359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 msgr2=0x557c91964c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c91964c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 -- 172.21.15.16:0/3183295359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91965e00 msgr2=0x557c9196a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91965e00 0x557c9196a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 -- 172.21.15.16:0/3183295359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c91876560 con 0x557c91960890 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf35ab4700 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 0x557c9195ad20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7faf2409af60 tx=0x7faf2409aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0baee8d1e1dcdd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf20ff9700 1 -- 172.21.15.16:0/3183295359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf240aeea0 con 0x557c91960890 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf20ff9700 1 -- 172.21.15.16:0/3183295359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf240b9270 con 0x557c91960890 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.858+0000 7faf20ff9700 1 -- 172.21.15.16:0/3183295359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2409a720 con 0x557c91960890 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3183295359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 msgr2=0x557c9195ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 0x557c9195ad20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7faf2409af60 tx=0x7faf2409aa60 comp rx=0 tx=0).stop 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3183295359 shutdown_connections 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91966720 0x557c91964c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91960890 0x557c9195ad20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/3183295359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91965e00 0x557c9196a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3183295359 >> 172.21.15.16:0/3183295359 conn(0x557c918be210 msgr2=0x557c91957210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3183295359 shutdown_connections 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 -- 172.21.15.16:0/3183295359 wait complete. 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 -- start start 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.859+0000 7faf43ed2b80 1 Processor -- start 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c9196ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c9196fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c919cf8e0 con 0x557c91965e00 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c919cfa50 con 0x557c91966720 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c919cfc10 con 0x557c91960890 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf43ed2b80 10 client.0 _ll_register_callbacks cb 0x557c91855b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35634/0 (socket says 172.21.15.16:35634) 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 -- 172.21.15.16:0/1719435300 learned_addr learned my addr 172.21.15.16:0/1719435300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf362b5700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c9196fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf35ab4700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c9196ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 -- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 msgr2=0x557c9196ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c9196ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 -- 172.21.15.16:0/1719435300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 msgr2=0x557c9196fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c9196fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 -- 172.21.15.16:0/1719435300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c917313b0 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf362b5700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c9196fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf352b3700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196f4b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faf2c031870 tx=0x7faf2c0314f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0cb73c1d8924c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.860+0000 7faf197fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf197fa700 1 -- 172.21.15.16:0/1719435300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c024740 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf197fa700 1 -- 172.21.15.16:0/1719435300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7faf2c0248a0 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf2409ee00 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf197fa700 1 -- 172.21.15.16:0/1719435300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf2c024a60 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 10 client.16455 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c91735cd0 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf197fa700 1 -- 172.21.15.16:0/1719435300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf2409ee00 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf197fa700 1 -- 172.21.15.16:0/1719435300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf2c00f460 con 0x557c91965e00 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:23:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 20 client.16455 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c919d10f0 0x557c919d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 4 client.16455 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c918a8ec0 con 0x557c919d10f0 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.861+0000 7faf35ab4700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c919d10f0 0x557c919d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf197fa700 10 client.16455 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf35ab4700 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c919d10f0 0x557c919d3510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2d98957242c9081 server_cookie=5c55b2ce8b46c080 in_seq=0 out_seq=0 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf197fa700 1 -- 172.21.15.16:0/1719435300 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c918a8ec0 con 0x557c919d10f0 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf197fa700 10 client.16455 handle_command_reply: tid=0 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 client.16455 shutdown 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 msgr2=0x557c9196f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196f4b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faf2c031870 tx=0x7faf2c0314f0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 shutdown_connections 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c919d10f0 0x557c919d3510 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c91966720 0x557c9196fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c91960890 0x557c9196ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf197fa700 0 client.16455 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 --2- 172.21.15.16:0/1719435300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c91965e00 0x557c9196f4b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 >> 172.21.15.16:0/1719435300 conn(0x557c918be210 msgr2=0x557c9196a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 shutdown_connections 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.862+0000 7faf43ed2b80 1 -- 172.21.15.16:0/1719435300 wait complete. 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf43ed2b80 20 client.16455 trim_cache size 0 max 16384 2024-05-08T23:23:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:23:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7faf1c0a1310 msgr2=0x7faf1c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7faf1c0a1310 0x7faf1c0a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faf2c006010 tx=0x7faf2c00b540 comp rx=0 tx=0).stop 2024-05-08T23:23:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 msgr2=0x7faf30196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 0x7faf30196220 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faf24009970 tx=0x7faf24009d30 comp rx=0 tx=0).stop 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 shutdown_connections 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7faf1c0a1310 0x7faf1c0a37d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf3019ab00 0x7faf3019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf30100060 0x7faf30196220 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 --2- 172.21.15.16:0/191261033 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf30105160 0x7faf30196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 >> 172.21.15.16:0/191261033 conn(0x7faf30076f40 msgr2=0x7faf300fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 shutdown_connections 2024-05-08T23:23:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:35.863+0000 7faf37d18700 1 -- 172.21.15.16:0/191261033 wait complete. 2024-05-08T23:23:35.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:35 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4257445840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:35.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:23:35.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:35 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4257445840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:35 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4257445840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:23:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.017+0000 7fc7c3811700 1 -- 172.21.15.16:0/3605256126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 msgr2=0x7fc7bc10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.017+0000 7fc7c3811700 1 --2- 172.21.15.16:0/3605256126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc10fbe0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400b0a0 tx=0x7fc7b400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 -- 172.21.15.16:0/3605256126 shutdown_connections 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 --2- 172.21.15.16:0/3605256126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc10fbe0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 --2- 172.21.15.16:0/3605256126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc101e20 0x7fc7bc0691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 --2- 172.21.15.16:0/3605256126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7bc0696f0 0x7fc7bc069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 -- 172.21.15.16:0/3605256126 >> 172.21.15.16:0/3605256126 conn(0x7fc7bc076ba0 msgr2=0x7fc7bc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 -- 172.21.15.16:0/3605256126 shutdown_connections 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 -- 172.21.15.16:0/3605256126 wait complete. 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 Processor -- start 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.018+0000 7fc7c3811700 1 -- start start 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc0696f0 0x7fc7bc199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7bc101e20 0x7fc7bc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc19e450 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc19e5c0 con 0x7fc7bc0696f0 2024-05-08T23:23:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc19e780 con 0x7fc7bc101e20 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35652/0 (socket says 172.21.15.16:35652) 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 -- 172.21.15.16:0/1367957247 learned_addr learned my addr 172.21.15.16:0/1367957247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7bc101e20 0x7fc7bc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc0696f0 0x7fc7bc199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 -- 172.21.15.16:0/1367957247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7bc101e20 msgr2=0x7fc7bc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7bc101e20 0x7fc7bc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 -- 172.21.15.16:0/1367957247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc0696f0 msgr2=0x7fc7bc199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc0696f0 0x7fc7bc199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 -- 172.21.15.16:0/1367957247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4009d00 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc19de10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400b830 tx=0x7fc7b4009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7b27fc700 1 -- 172.21.15.16:0/1367957247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400e040 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7b27fc700 1 -- 172.21.15.16:0/1367957247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7b4004e60 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7b27fc700 1 -- 172.21.15.16:0/1367957247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4005420 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc19ea00 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.019+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc1a62a0 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.022+0000 7fc7b27fc700 1 -- 172.21.15.16:0/1367957247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc7b401bb50 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.022+0000 7fc7b27fc700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc7a80a1290 0x7fc7a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.022+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc7a80a1290 0x7fc7a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.022+0000 7fc7b27fc700 1 -- 172.21.15.16:0/1367957247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc7b40ce8a0 con 0x7fc7bc10d7f0 2024-05-08T23:23:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.023+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc7a80a1290 0x7fc7a80a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005fd0 tx=0x7fc7b8005db0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.023+0000 7fc7cf9cbb80 1 Processor -- start 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.023+0000 7fc7cf9cbb80 1 -- start start 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b96410720 0x563b9640a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9640acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b96361150 con 0x563b9640a890 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9635eb10 con 0x563b96410720 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9635a400 con 0x563b9640fe00 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9640acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9640acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35656/0 (socket says 172.21.15.16:35656) 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 -- 172.21.15.16:0/802435687 learned_addr learned my addr 172.21.15.16:0/802435687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b96410720 0x563b9640a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 -- 172.21.15.16:0/802435687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 msgr2=0x563b964101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 -- 172.21.15.16:0/802435687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b96410720 msgr2=0x563b9640a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b96410720 0x563b9640a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 -- 172.21.15.16:0/802435687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b961db3b0 con 0x563b9640a890 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b96410720 0x563b9640a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.024+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9640acd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40a35c0 tx=0x7fc7b400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2564fbe97c542de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7a77fe700 1 -- 172.21.15.16:0/802435687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b409b700 con 0x563b9640a890 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7a77fe700 1 -- 172.21.15.16:0/802435687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7b409f530 con 0x563b9640a890 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7a77fe700 1 -- 172.21.15.16:0/802435687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b409b960 con 0x563b9640a890 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/802435687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 msgr2=0x563b9640acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9640acd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40a35c0 tx=0x7fc7b400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/802435687 shutdown_connections 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9640acd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/802435687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b96410720 0x563b9640a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/802435687 >> 172.21.15.16:0/802435687 conn(0x563b96368210 msgr2=0x563b96404df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.025+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/802435687 shutdown_connections 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.026+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/802435687 wait complete. 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.026+0000 7fc7cf9cbb80 1 -- start start 2024-05-08T23:23:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.026+0000 7fc7cf9cbb80 1 Processor -- start 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b96413b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 0x563b964145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b962f6560 con 0x563b96410720 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b962f6200 con 0x563b9640a890 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b961dd6a0 con 0x563b9640fe00 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7cf9cbb80 10 client.0 _ll_register_callbacks cb 0x563b962ffb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 0x563b964145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 0x563b964145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35668/0 (socket says 172.21.15.16:35668) 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 -- 172.21.15.16:0/607299586 learned_addr learned my addr 172.21.15.16:0/607299586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b96413b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 -- 172.21.15.16:0/607299586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 msgr2=0x563b964140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 -- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 msgr2=0x563b96413b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b96413b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 -- 172.21.15.16:0/607299586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc1a62a0 con 0x563b96410720 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 0x563b964145f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4098170 tx=0x7fc7b401be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3823974c70ee444 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.027+0000 7fc7a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/607299586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b409b700 con 0x563b96410720 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/607299586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc7b409abf0 con 0x563b96410720 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b962b3270 con 0x563b96410720 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/607299586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7b4067120 con 0x563b96410720 2024-05-08T23:23:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 10 client.16467 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563b96187b60 con 0x563b96410720 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/607299586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b962b3270 con 0x563b96410720 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/607299586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7b40a3660 con 0x563b96410720 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 10 client.16467 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 20 client.16467 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b96423b30 0x563b96425f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 4 client.16467 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.028+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b963d9810 con 0x563b96423b30 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.029+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b96423b30 0x563b96425f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.029+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b96423b30 0x563b96425f50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c1c791332315f38 server_cookie=db496e47f6ac2a3c in_seq=0 out_seq=0 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.029+0000 7fc7a4ff9700 10 client.16467 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/607299586 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b963d9810 con 0x563b96423b30 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7a4ff9700 10 client.16467 handle_command_reply: tid=0 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 client.16467 shutdown 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 msgr2=0x563b964145f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 0x563b964145f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4098170 tx=0x7fc7b401be00 comp rx=0 tx=0).stop 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 shutdown_connections 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b96410720 0x563b964145f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b96423b30 0x563b96425f50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b96413b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/607299586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b9640fe00 0x563b964140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7a4ff9700 0 client.16467 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 >> 172.21.15.16:0/607299586 conn(0x563b96368210 msgr2=0x563b964045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 shutdown_connections 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/607299586 wait complete. 2024-05-08T23:23:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.030+0000 7fc7cf9cbb80 20 client.16467 trim_cache size 0 max 16384 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 Processor -- start 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 -- start start 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b9640a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 0x563b96425f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96404d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b964151a0 con 0x563b9640fe00 2024-05-08T23:23:36.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b96401df0 con 0x563b9640a890 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.044+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9640d870 con 0x563b96410720 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 0x563b96425f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 0x563b96425f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35678/0 (socket says 172.21.15.16:35678) 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 -- 172.21.15.16:0/1701638272 learned_addr learned my addr 172.21.15.16:0/1701638272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96404d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b9640a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 -- 172.21.15.16:0/1701638272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 msgr2=0x563b96404d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96404d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 -- 172.21.15.16:0/1701638272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 msgr2=0x563b9640a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b9640a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 -- 172.21.15.16:0/1701638272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b96320560 con 0x563b9640fe00 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b9640a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 0x563b96425f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc19aa80 tx=0x7fc7ac0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2ed616711cc6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7a7fff700 1 -- 172.21.15.16:0/1701638272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac011c30 con 0x563b9640fe00 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7a7fff700 1 -- 172.21.15.16:0/1701638272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7ac016070 con 0x563b9640fe00 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.045+0000 7fc7a7fff700 1 -- 172.21.15.16:0/1701638272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac005540 con 0x563b9640fe00 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/1701638272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 msgr2=0x563b96425f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 0x563b96425f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc19aa80 tx=0x7fc7ac0113e0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/1701638272 shutdown_connections 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96404d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640a890 0x563b9640a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/1701638272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640fe00 0x563b96425f20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/1701638272 >> 172.21.15.16:0/1701638272 conn(0x563b96368210 msgr2=0x563b96401210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/1701638272 shutdown_connections 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/1701638272 wait complete. 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 -- start start 2024-05-08T23:23:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.046+0000 7fc7cf9cbb80 1 Processor -- start 2024-05-08T23:23:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9641b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640fe00 0x563b96418f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96419560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b96419c70 con 0x563b9640a890 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b96419de0 con 0x563b9640fe00 2024-05-08T23:23:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b96479040 con 0x563b96410720 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7cf9cbb80 10 client.0 _ll_register_callbacks cb 0x563b964205f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9641b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9641b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35684/0 (socket says 172.21.15.16:35684) 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 -- 172.21.15.16:0/2186899138 learned_addr learned my addr 172.21.15.16:0/2186899138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96419560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640fe00 0x563b96418f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 -- 172.21.15.16:0/2186899138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 msgr2=0x563b96419560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96419560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 -- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640fe00 msgr2=0x563b96418f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640fe00 0x563b96418f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 -- 172.21.15.16:0/2186899138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac00a950 con 0x563b9640a890 2024-05-08T23:23:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c1dae700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96419560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7c15ad700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9641b400 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005e20 tx=0x7fc7b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac45584c483d27dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.047+0000 7fc7a4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/2186899138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8024040 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/2186899138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc7b80266c0 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7b409f530 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/2186899138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7b8026880 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 10 client.16473 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563b961dfcd0 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/2186899138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7b409f530 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/2186899138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7b80105f0 con 0x563b9640a890 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 10 client.16473 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 20 client.16473 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b9647a810 0x563b9647cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 4 client.16473 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.048+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563b96352ec0 con 0x563b9647a810 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b9647a810 0x563b9647cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7a4ff9700 10 client.16473 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7c0dac700 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b9647a810 0x563b9647cc30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddc932a8114bdd9 server_cookie=f5bdb9071ec51d in_seq=0 out_seq=0 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7a4ff9700 1 -- 172.21.15.16:0/2186899138 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563b96352ec0 con 0x563b9647a810 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7a4ff9700 10 client.16473 handle_command_reply: tid=0 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 client.16473 shutdown 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 msgr2=0x563b9641b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9641b400 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005e20 tx=0x7fc7b80047c0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 shutdown_connections 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b96410720 0x563b96419560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9640a890 0x563b9641b400 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563b9647a810 0x563b9647cc30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 --2- 172.21.15.16:0/2186899138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563b9640fe00 0x563b96418f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7a4ff9700 0 client.16473 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.049+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 >> 172.21.15.16:0/2186899138 conn(0x563b96368210 msgr2=0x563b964123d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 shutdown_connections 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7cf9cbb80 1 -- 172.21.15.16:0/2186899138 wait complete. 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7cf9cbb80 20 client.16473 trim_cache size 0 max 16384 2024-05-08T23:23:36.057 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc7a80a1290 msgr2=0x7fc7a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc7a80a1290 0x7fc7a80a3750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005fd0 tx=0x7fc7b8005db0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 msgr2=0x7fc7bc19de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc19de10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400b830 tx=0x7fc7b4009430 comp rx=0 tx=0).stop 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 shutdown_connections 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc7a80a1290 0x7fc7a80a3750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7bc10d7f0 0x7fc7bc19de10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc7bc0696f0 0x7fc7bc199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 --2- 172.21.15.16:0/1367957247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7bc101e20 0x7fc7bc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 >> 172.21.15.16:0/1367957247 conn(0x7fc7bc076ba0 msgr2=0x7fc7bc0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 shutdown_connections 2024-05-08T23:23:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:36.050+0000 7fc7c3811700 1 -- 172.21.15.16:0/1367957247 wait complete. 2024-05-08T23:23:36.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:36 smithi016 ceph-mon[42541]: pgmap v124: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:36.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:36 smithi189 ceph-mon[46919]: pgmap v124: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:36.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:36 smithi155 ceph-mon[47105]: pgmap v124: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:37.064 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:23:37.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:23:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 -- 172.21.15.16:0/3315541948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c101180 msgr2=0x7eff6c101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 --2- 172.21.15.16:0/3315541948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c101180 0x7eff6c101560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7eff60009990 tx=0x7eff60009ca0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 -- 172.21.15.16:0/3315541948 shutdown_connections 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 --2- 172.21.15.16:0/3315541948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c1062a0 0x7eff6c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 --2- 172.21.15.16:0/3315541948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c101180 0x7eff6c101560 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 --2- 172.21.15.16:0/3315541948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101b30 0x7eff6c105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 -- 172.21.15.16:0/3315541948 >> 172.21.15.16:0/3315541948 conn(0x7eff6c0fc9e0 msgr2=0x7eff6c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 -- 172.21.15.16:0/3315541948 shutdown_connections 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 -- 172.21.15.16:0/3315541948 wait complete. 2024-05-08T23:23:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.217+0000 7eff710a1700 1 Processor -- start 2024-05-08T23:23:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 -- start start 2024-05-08T23:23:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101180 0x7eff6c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c1062a0 0x7eff6c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 0x7eff6c19c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19c670 con 0x7eff6c19bbf0 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19c7e0 con 0x7eff6c101180 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff710a1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c19c9a0 con 0x7eff6c1062a0 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff6b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 0x7eff6c19c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff6b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 0x7eff6c19c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32868/0 (socket says 172.21.15.16:32868) 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff6b59e700 1 -- 172.21.15.16:0/1294658063 learned_addr learned my addr 172.21.15.16:0/1294658063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101180 0x7eff6c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.218+0000 7eff6a59c700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c1062a0 0x7eff6c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6b59e700 1 -- 172.21.15.16:0/1294658063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c1062a0 msgr2=0x7eff6c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6b59e700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c1062a0 0x7eff6c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6b59e700 1 -- 172.21.15.16:0/1294658063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101180 msgr2=0x7eff6c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6b59e700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101180 0x7eff6c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6b59e700 1 -- 172.21.15.16:0/1294658063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600095f0 con 0x7eff6c19bbf0 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101180 0x7eff6c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff6b59e700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 0x7eff6c19c070 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7eff64007ee0 tx=0x7eff6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff53fff700 1 -- 172.21.15.16:0/1294658063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64017070 con 0x7eff6c19bbf0 2024-05-08T23:23:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff53fff700 1 -- 172.21.15.16:0/1294658063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff6400f040 con 0x7eff6c19bbf0 2024-05-08T23:23:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c1a3f00 con 0x7eff6c19bbf0 2024-05-08T23:23:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff53fff700 1 -- 172.21.15.16:0/1294658063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64005710 con 0x7eff6c19bbf0 2024-05-08T23:23:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.219+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c1a43e0 con 0x7eff6c19bbf0 2024-05-08T23:23:37.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.221+0000 7eff53fff700 1 -- 172.21.15.16:0/1294658063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7eff64005870 con 0x7eff6c19bbf0 2024-05-08T23:23:37.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.222+0000 7eff53fff700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7eff540a1310 0x7eff540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.222+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7eff540a1310 0x7eff540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.222+0000 7eff53fff700 1 -- 172.21.15.16:0/1294658063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7eff640cc7d0 con 0x7eff6c19bbf0 2024-05-08T23:23:37.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7eff540a1310 0x7eff540a37d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff60006010 tx=0x7eff6000b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 Processor -- start 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 -- start start 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278462260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127845c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56127845c910 0x56127845cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612783b31d0 con 0x561278461e80 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612783b0b90 con 0x56127845c910 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.223+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612783ac480 con 0x5612784627a0 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278462260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278462260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32870/0 (socket says 172.21.15.16:32870) 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 -- 172.21.15.16:0/4065913360 learned_addr learned my addr 172.21.15.16:0/4065913360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6a59c700 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127845c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6b59e700 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56127845c910 0x56127845cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 -- 172.21.15.16:0/4065913360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 msgr2=0x56127845c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127845c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 -- 172.21.15.16:0/4065913360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56127845c910 msgr2=0x56127845cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56127845c910 0x56127845cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 -- 172.21.15.16:0/4065913360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56127822d430 con 0x561278461e80 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6b59e700 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56127845c910 0x56127845cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278462260 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7eff60005410 tx=0x7eff6001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9acbc5450511db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff517fa700 1 -- 172.21.15.16:0/4065913360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6002b070 con 0x561278461e80 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff517fa700 1 -- 172.21.15.16:0/4065913360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff6001e040 con 0x561278461e80 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.224+0000 7eff517fa700 1 -- 172.21.15.16:0/4065913360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60004a90 con 0x561278461e80 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 -- 172.21.15.16:0/4065913360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 msgr2=0x561278462260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278462260 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7eff60005410 tx=0x7eff6001f000 comp rx=0 tx=0).stop 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 -- 172.21.15.16:0/4065913360 shutdown_connections 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56127845c910 0x56127845cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278462260 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/4065913360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127845c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 -- 172.21.15.16:0/4065913360 >> 172.21.15.16:0/4065913360 conn(0x5612783ba290 msgr2=0x561278456e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 -- 172.21.15.16:0/4065913360 shutdown_connections 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.225+0000 7eff7d25bb80 1 -- 172.21.15.16:0/4065913360 wait complete. 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 -- start start 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 Processor -- start 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56127845c910 0x561278465bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278466130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 0x561278466670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612783485e0 con 0x561278461e80 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561278348280 con 0x5612784627a0 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127822f720 con 0x56127845c910 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff7d25bb80 10 client.0 _ll_register_callbacks cb 0x561278351bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278466130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278466130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32876/0 (socket says 172.21.15.16:32876) 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 -- 172.21.15.16:0/2909942450 learned_addr learned my addr 172.21.15.16:0/2909942450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 -- 172.21.15.16:0/2909942450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56127845c910 msgr2=0x561278465bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56127845c910 0x561278465bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 -- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 msgr2=0x561278466670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 0x561278466670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.226+0000 7eff6a59c700 1 -- 172.21.15.16:0/2909942450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c10e690 con 0x561278461e80 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff6a59c700 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278466130 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00bf80 tx=0x7eff5c0127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25feb0aa503d9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff4eff5700 1 -- 172.21.15.16:0/2909942450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004440 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff4eff5700 1 -- 172.21.15.16:0/2909942450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7eff5c003680 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612783052f0 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff4eff5700 1 -- 172.21.15.16:0/2909942450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5c003840 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff7d25bb80 10 client.16485 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5612781d9be0 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff4eff5700 1 -- 172.21.15.16:0/2909942450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5612783052f0 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff4eff5700 1 -- 172.21.15.16:0/2909942450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff5c01ba20 con 0x561278461e80 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff7d25bb80 10 client.16485 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.227+0000 7eff7d25bb80 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff7d25bb80 20 client.16485 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561278476380 0x5612784787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff7d25bb80 4 client.16485 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56127842b890 con 0x561278476380 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561278476380 0x5612784787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561278476380 0x5612784787a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b757030e85ddbc2 server_cookie=1c1aa328c1318355 in_seq=0 out_seq=0 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.228+0000 7eff4eff5700 10 client.16485 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff4eff5700 1 -- 172.21.15.16:0/2909942450 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56127842b890 con 0x561278476380 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff4eff5700 10 client.16485 handle_command_reply: tid=0 2024-05-08T23:23:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 client.16485 shutdown 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 msgr2=0x561278466130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278466130 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00bf80 tx=0x7eff5c0127e0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 shutdown_connections 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561278476380 0x5612784787a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 0x561278466670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56127845c910 0x561278465bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff4eff5700 0 client.16485 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/2909942450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561278461e80 0x561278466130 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 >> 172.21.15.16:0/2909942450 conn(0x5612783ba290 msgr2=0x561278456620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 shutdown_connections 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 1 -- 172.21.15.16:0/2909942450 wait complete. 2024-05-08T23:23:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.229+0000 7eff7d25bb80 20 client.16485 trim_cache size 0 max 16384 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 Processor -- start 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 -- start start 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x561278456da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561278461e80 0x561278460d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 0x561278461240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561278467240 con 0x56127845c910 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56127845f920 con 0x5612784627a0 2024-05-08T23:23:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561278453e40 con 0x561278461e80 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x561278456da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x561278456da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32888/0 (socket says 172.21.15.16:32888) 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 -- 172.21.15.16:0/503452640 learned_addr learned my addr 172.21.15.16:0/503452640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 -- 172.21.15.16:0/503452640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561278461e80 msgr2=0x561278460d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6a59c700 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561278461e80 0x561278460d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561278461e80 0x561278460d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 -- 172.21.15.16:0/503452640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 msgr2=0x561278461240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 0x561278461240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6ad9d700 1 -- 172.21.15.16:0/503452640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561278465750 con 0x56127845c910 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.248+0000 7eff6a59c700 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561278461e80 0x561278460d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x561278456da0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7eff600036c0 tx=0x7eff60005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201a1ef7a5663e45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff51ffb700 1 -- 172.21.15.16:0/503452640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6002b040 con 0x56127845c910 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff51ffb700 1 -- 172.21.15.16:0/503452640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff60004b90 con 0x56127845c910 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff51ffb700 1 -- 172.21.15.16:0/503452640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60038290 con 0x56127845c910 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 -- 172.21.15.16:0/503452640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 msgr2=0x561278456da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x561278456da0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7eff600036c0 tx=0x7eff60005ab0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 -- 172.21.15.16:0/503452640 shutdown_connections 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612784627a0 0x561278461240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x561278456da0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/503452640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561278461e80 0x561278460d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 -- 172.21.15.16:0/503452640 >> 172.21.15.16:0/503452640 conn(0x5612783ba290 msgr2=0x5612784644e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 -- 172.21.15.16:0/503452640 shutdown_connections 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.249+0000 7eff7d25bb80 1 -- 172.21.15.16:0/503452640 wait complete. 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 -- start start 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 Processor -- start 2024-05-08T23:23:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x56127846bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561278461e80 0x56127846c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127846c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612784cc890 con 0x56127845c910 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612784cc9d0 con 0x561278461e80 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612784ccb10 con 0x5612784627a0 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff7d25bb80 10 client.0 _ll_register_callbacks cb 0x561278351bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x56127846bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x56127846bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32898/0 (socket says 172.21.15.16:32898) 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 -- 172.21.15.16:0/980738976 learned_addr learned my addr 172.21.15.16:0/980738976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6a59c700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561278461e80 0x56127846c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6b59e700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127846c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 -- 172.21.15.16:0/980738976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 msgr2=0x56127846c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127846c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 -- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561278461e80 msgr2=0x56127846c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.250+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561278461e80 0x56127846c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff6ad9d700 1 -- 172.21.15.16:0/980738976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56127822d430 con 0x56127845c910 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff6ad9d700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x56127846bbf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7eff600059c0 tx=0x7eff60004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16f51aaa0fb529 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff4eff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff4eff5700 1 -- 172.21.15.16:0/980738976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6002b050 con 0x56127845c910 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff4eff5700 1 -- 172.21.15.16:0/980738976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7eff6000b970 con 0x56127845c910 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff6001e040 con 0x56127845c910 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff4eff5700 1 -- 172.21.15.16:0/980738976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff6000bb70 con 0x56127845c910 2024-05-08T23:23:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 10 client.16491 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5612783725e0 con 0x56127845c910 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff4eff5700 1 -- 172.21.15.16:0/980738976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff6001e040 con 0x56127845c910 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff4eff5700 1 -- 172.21.15.16:0/980738976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff60048070 con 0x56127845c910 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 20 client.16491 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5612784cddb0 0x5612784d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 4 client.16491 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:23:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.251+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5612783a4f40 con 0x5612784cddb0 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.252+0000 7eff6a59c700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5612784cddb0 0x5612784d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.252+0000 7eff4eff5700 10 client.16491 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.252+0000 7eff6a59c700 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5612784cddb0 0x5612784d01a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7289ccf99fe3b22 server_cookie=e01416b70d71ede2 in_seq=0 out_seq=0 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff4eff5700 1 -- 172.21.15.16:0/980738976 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5612783a4f40 con 0x5612784cddb0 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff4eff5700 10 client.16491 handle_command_reply: tid=0 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff7d25bb80 1 client.16491 shutdown 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 msgr2=0x56127846bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x56127846bbf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7eff600059c0 tx=0x7eff60004500 comp rx=0 tx=0).stop 2024-05-08T23:23:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 shutdown_connections 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612784627a0 0x56127846c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.253+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56127845c910 0x56127846bbf0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5612784cddb0 0x5612784d01a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff7d25bb80 1 --2- 172.21.15.16:0/980738976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561278461e80 0x56127846c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 >> 172.21.15.16:0/980738976 conn(0x5612783ba290 msgr2=0x561278460d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff4eff5700 0 client.16491 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 shutdown_connections 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff7d25bb80 1 -- 172.21.15.16:0/980738976 wait complete. 2024-05-08T23:23:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff7d25bb80 20 client.16491 trim_cache size 0 max 16384 2024-05-08T23:23:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:23:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:23:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "fbe96596-3c2e-4a1d-ad3e-8fdfa5af07b7", 2024-05-08T23:23:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:23:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7eff540a1310 msgr2=0x7eff540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7eff540a1310 0x7eff540a37d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff60006010 tx=0x7eff6000b540 comp rx=0 tx=0).stop 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 msgr2=0x7eff6c19c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 0x7eff6c19c070 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7eff64007ee0 tx=0x7eff6400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 shutdown_connections 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7eff540a1310 0x7eff540a37d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff6c19bbf0 0x7eff6c19c070 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff6c101180 0x7eff6c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 --2- 172.21.15.16:0/1294658063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c1062a0 0x7eff6c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 >> 172.21.15.16:0/1294658063 conn(0x7eff6c0fc9e0 msgr2=0x7eff6c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 shutdown_connections 2024-05-08T23:23:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.254+0000 7eff710a1700 1 -- 172.21.15.16:0/1294658063 wait complete. 2024-05-08T23:23:37.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:23:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 -- 172.21.15.16:0/1514870037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 msgr2=0x7fed3c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 --2- 172.21.15.16:0/1514870037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c10de00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b0a0 tx=0x7fed3400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 -- 172.21.15.16:0/1514870037 shutdown_connections 2024-05-08T23:23:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 --2- 172.21.15.16:0/1514870037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c10de00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 --2- 172.21.15.16:0/1514870037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed3c101160 0x7fed3c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 --2- 172.21.15.16:0/1514870037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3c101b10 0x7fed3c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 -- 172.21.15.16:0/1514870037 >> 172.21.15.16:0/1514870037 conn(0x7fed3c0fc9e0 msgr2=0x7fed3c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 -- 172.21.15.16:0/1514870037 shutdown_connections 2024-05-08T23:23:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.408+0000 7fed439db700 1 -- 172.21.15.16:0/1514870037 wait complete. 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 Processor -- start 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 -- start start 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3c101160 0x7fed3c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed3c19bd00 0x7fed3c19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19c780 con 0x7fed3c106280 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19c8f0 con 0x7fed3c101160 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed439db700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19cab0 con 0x7fed3c19bd00 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed40f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed40f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32920/0 (socket says 172.21.15.16:32920) 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed40f76700 1 -- 172.21.15.16:0/2414138712 learned_addr learned my addr 172.21.15.16:0/2414138712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed41f78700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed3c19bd00 0x7fed3c19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.409+0000 7fed41777700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3c101160 0x7fed3c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed40f76700 1 -- 172.21.15.16:0/2414138712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed3c19bd00 msgr2=0x7fed3c19c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed40f76700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed3c19bd00 0x7fed3c19c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed40f76700 1 -- 172.21.15.16:0/2414138712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3c101160 msgr2=0x7fed3c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed40f76700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3c101160 0x7fed3c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed40f76700 1 -- 172.21.15.16:0/2414138712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009d00 con 0x7fed3c106280 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed40f76700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c1979b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e970 tx=0x7fed3000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed2e7fc700 1 -- 172.21.15.16:0/2414138712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30009790 con 0x7fed3c106280 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed2e7fc700 1 -- 172.21.15.16:0/2414138712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed3000c7e0 con 0x7fed3c106280 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c1a3fb0 con 0x7fed3c106280 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed2e7fc700 1 -- 172.21.15.16:0/2414138712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30005440 con 0x7fed3c106280 2024-05-08T23:23:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.410+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c1a44e0 con 0x7fed3c106280 2024-05-08T23:23:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.413+0000 7fed2e7fc700 1 -- 172.21.15.16:0/2414138712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fed3000c950 con 0x7fed3c106280 2024-05-08T23:23:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.414+0000 7fed2e7fc700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fed280a1310 0x7fed280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.414+0000 7fed41777700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fed280a1310 0x7fed280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.414+0000 7fed2e7fc700 1 -- 172.21.15.16:0/2414138712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fed30014070 con 0x7fed3c106280 2024-05-08T23:23:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.414+0000 7fed41777700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fed280a1310 0x7fed280a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fed38009c60 tx=0x7fed38009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 Processor -- start 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 -- start start 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd6e00 0x5621ebfd71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfd1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd1890 0x5621ebfd1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebf28150 con 0x5621ebfd7720 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebf25b10 con 0x5621ebfd6e00 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebf21400 con 0x5621ebfd1890 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed40f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfd1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed40f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfd1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32934/0 (socket says 172.21.15.16:32934) 2024-05-08T23:23:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed40f76700 1 -- 172.21.15.16:0/3786089903 learned_addr learned my addr 172.21.15.16:0/3786089903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed41f78700 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd1890 0x5621ebfd1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.415+0000 7fed41777700 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd6e00 0x5621ebfd71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed40f76700 1 -- 172.21.15.16:0/3786089903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd1890 msgr2=0x5621ebfd1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed40f76700 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd1890 0x5621ebfd1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed40f76700 1 -- 172.21.15.16:0/3786089903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd6e00 msgr2=0x5621ebfd71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed40f76700 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd6e00 0x5621ebfd71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed40f76700 1 -- 172.21.15.16:0/3786089903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ebda23b0 con 0x5621ebfd7720 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed41777700 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd6e00 0x5621ebfd71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed40f76700 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfd1350 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed300a1df0 tx=0x7fed3000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9434fbeff808a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed277fe700 1 -- 172.21.15.16:0/3786089903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3009dcd0 con 0x5621ebfd7720 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed277fe700 1 -- 172.21.15.16:0/3786089903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed300a2320 con 0x5621ebfd7720 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.416+0000 7fed277fe700 1 -- 172.21.15.16:0/3786089903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30016530 con 0x5621ebfd7720 2024-05-08T23:23:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 -- 172.21.15.16:0/3786089903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 msgr2=0x5621ebfd1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfd1350 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fed300a1df0 tx=0x7fed3000ccd0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 -- 172.21.15.16:0/3786089903 shutdown_connections 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd1890 0x5621ebfd1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd6e00 0x5621ebfd71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/3786089903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfd1350 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 -- 172.21.15.16:0/3786089903 >> 172.21.15.16:0/3786089903 conn(0x5621ebf2f210 msgr2=0x5621ebfcbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 -- 172.21.15.16:0/3786089903 shutdown_connections 2024-05-08T23:23:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 -- 172.21.15.16:0/3786089903 wait complete. 2024-05-08T23:23:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 -- start start 2024-05-08T23:23:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.417+0000 7fed4fb95b80 1 Processor -- start 2024-05-08T23:23:37.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfdab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfdb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfdb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebebd560 con 0x5621ebfd7720 2024-05-08T23:23:37.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebebd200 con 0x5621ebfd1890 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebda46a0 con 0x5621ebfd6e00 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed4fb95b80 10 client.0 _ll_register_callbacks cb 0x5621ebec6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfdb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfdb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32948/0 (socket says 172.21.15.16:32948) 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 -- 172.21.15.16:0/4136036209 learned_addr learned my addr 172.21.15.16:0/4136036209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed40f76700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfdb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41777700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfdab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 -- 172.21.15.16:0/4136036209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 msgr2=0x5621ebfdb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfdb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 -- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 msgr2=0x5621ebfdab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfdab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 -- 172.21.15.16:0/4136036209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c1a44e0 con 0x5621ebfd7720 2024-05-08T23:23:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41777700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfdab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed41f78700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfdb580 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fed34009610 tx=0x7fed3400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0120e562290e8a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.418+0000 7fed24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed24ff9700 1 -- 172.21.15.16:0/4136036209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e040 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed24ff9700 1 -- 172.21.15.16:0/4136036209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fed34007c20 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621ebe7a270 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed24ff9700 1 -- 172.21.15.16:0/4136036209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed34007e20 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 10 client.16503 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5621ebd4eb60 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed24ff9700 1 -- 172.21.15.16:0/4136036209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5621ebe7a270 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed24ff9700 1 -- 172.21.15.16:0/4136036209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed3400bd20 con 0x5621ebfd7720 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 10 client.16503 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 20 client.16503 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:37.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ebfef690 0x5621ebff1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 4 client.16503 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.419+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621ebfa0810 con 0x5621ebfef690 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.420+0000 7fed41777700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ebfef690 0x5621ebff1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.420+0000 7fed41777700 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ebfef690 0x5621ebff1ab0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e2656aad5870a0 server_cookie=ca5e08201d5a1951 in_seq=0 out_seq=0 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.420+0000 7fed24ff9700 10 client.16503 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed24ff9700 1 -- 172.21.15.16:0/4136036209 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621ebfa0810 con 0x5621ebfef690 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed24ff9700 10 client.16503 handle_command_reply: tid=0 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 client.16503 shutdown 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 msgr2=0x5621ebfdb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfdb580 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fed34009610 tx=0x7fed3400ba50 comp rx=0 tx=0).stop 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 shutdown_connections 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd7720 0x5621ebfdb580 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ebfef690 0x5621ebff1ab0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfdab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/4136036209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfdb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed24ff9700 0 client.16503 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 >> 172.21.15.16:0/4136036209 conn(0x5621ebf2f210 msgr2=0x5621ebfcb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 shutdown_connections 2024-05-08T23:23:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 1 -- 172.21.15.16:0/4136036209 wait complete. 2024-05-08T23:23:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.421+0000 7fed4fb95b80 20 client.16503 trim_cache size 0 max 16384 2024-05-08T23:23:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.439+0000 7fed4fb95b80 1 Processor -- start 2024-05-08T23:23:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.439+0000 7fed4fb95b80 1 -- start start 2024-05-08T23:23:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 0x5621ebfd1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfcbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd7720 0x5621ebfd5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebfa0810 con 0x5621ebfd1890 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebe7a270 con 0x5621ebfd7720 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebfec1e0 con 0x5621ebfd6e00 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 0x5621ebfd1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 0x5621ebfd1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32960/0 (socket says 172.21.15.16:32960) 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 -- 172.21.15.16:0/2195869750 learned_addr learned my addr 172.21.15.16:0/2195869750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41f78700 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd7720 0x5621ebfd5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed40f76700 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfcbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 -- 172.21.15.16:0/2195869750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 msgr2=0x5621ebfcbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfcbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 -- 172.21.15.16:0/2195869750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd7720 msgr2=0x5621ebfd5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd7720 0x5621ebfd5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 -- 172.21.15.16:0/2195869750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ebfd1f40 con 0x5621ebfd1890 2024-05-08T23:23:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed40f76700 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfcbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.440+0000 7fed41777700 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 0x5621ebfd1350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x5621ebff2350 tx=0x7fed38014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f004105d75aa840 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed27fff700 1 -- 172.21.15.16:0/2195869750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38023040 con 0x5621ebfd1890 2024-05-08T23:23:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed27fff700 1 -- 172.21.15.16:0/2195869750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed38025040 con 0x5621ebfd1890 2024-05-08T23:23:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed27fff700 1 -- 172.21.15.16:0/2195869750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38032a20 con 0x5621ebfd1890 2024-05-08T23:23:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 -- 172.21.15.16:0/2195869750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 msgr2=0x5621ebfd1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 0x5621ebfd1350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x5621ebff2350 tx=0x7fed38014000 comp rx=0 tx=0).stop 2024-05-08T23:23:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 -- 172.21.15.16:0/2195869750 shutdown_connections 2024-05-08T23:23:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd7720 0x5621ebfd5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd1890 0x5621ebfd1350 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/2195869750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd6e00 0x5621ebfcbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 -- 172.21.15.16:0/2195869750 >> 172.21.15.16:0/2195869750 conn(0x5621ebf2f210 msgr2=0x5621ebfd93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 -- 172.21.15.16:0/2195869750 shutdown_connections 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.441+0000 7fed4fb95b80 1 -- 172.21.15.16:0/2195869750 wait complete. 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 -- start start 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 Processor -- start 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfe66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 0x5621ebfe4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd7720 0x5621ebfe5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebfe5680 con 0x5621ebfd6e00 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebfe57f0 con 0x5621ebfd1890 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.442+0000 7fed4fb95b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebfe59b0 con 0x5621ebfd7720 2024-05-08T23:23:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed4fb95b80 10 client.0 _ll_register_callbacks cb 0x5621ebec6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 0x5621ebfe4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 0x5621ebfe4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32974/0 (socket says 172.21.15.16:32974) 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 -- 172.21.15.16:0/887866667 learned_addr learned my addr 172.21.15.16:0/887866667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed41f78700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd7720 0x5621ebfe5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed41777700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfe66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 -- 172.21.15.16:0/887866667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd7720 msgr2=0x5621ebfe5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd7720 0x5621ebfe5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 -- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 msgr2=0x5621ebfe66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfe66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 -- 172.21.15.16:0/887866667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed38017070 con 0x5621ebfd6e00 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed41777700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfe66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed41f78700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd7720 0x5621ebfe5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed40f76700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 0x5621ebfe4c00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed300963a0 tx=0x7fed300961e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d171af79b7ab4cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:23:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed24ff9700 1 -- 172.21.15.16:0/887866667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30016330 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed24ff9700 1 -- 172.21.15.16:0/887866667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fed300b8030 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed300a2320 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.443+0000 7fed24ff9700 1 -- 172.21.15.16:0/887866667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed30016490 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 10 client.16509 fetch_fsmap learned FSMap version 25 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5621ebee7560 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed24ff9700 1 -- 172.21.15.16:0/887866667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed300a2320 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed24ff9700 1 -- 172.21.15.16:0/887866667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed30099390 con 0x5621ebfd6e00 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 10 client.16509 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 20 client.16509 populate_metadata read hostname 'smithi016' 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ec046bd0 0x5621ec048fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:23:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 4 client.16509 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5621ebf19ec0 con 0x5621ec046bd0 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed41777700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ec046bd0 0x5621ec048fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed24ff9700 10 client.16509 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.444+0000 7fed41777700 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ec046bd0 0x5621ec048fc0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff2c4d576e78f3b server_cookie=289ad2eaf6b1f32 in_seq=0 out_seq=0 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed24ff9700 1 -- 172.21.15.16:0/887866667 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5621ebf19ec0 con 0x5621ec046bd0 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed24ff9700 10 client.16509 handle_command_reply: tid=0 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 client.16509 shutdown 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 msgr2=0x5621ebfe4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 0x5621ebfe4c00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed300963a0 tx=0x7fed300961e0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 shutdown_connections 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621ebfd7720 0x5621ebfe5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5621ec046bd0 0x5621ec048fc0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621ebfd1890 0x5621ebfe66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 --2- 172.21.15.16:0/887866667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5621ebfd6e00 0x5621ebfe4c00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed24ff9700 0 client.16509 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:23:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 >> 172.21.15.16:0/887866667 conn(0x5621ebf2f210 msgr2=0x5621ebfd5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 shutdown_connections 2024-05-08T23:23:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 1 -- 172.21.15.16:0/887866667 wait complete. 2024-05-08T23:23:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.445+0000 7fed4fb95b80 20 client.16509 trim_cache size 0 max 16384 2024-05-08T23:23:37.452 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:23:37.452 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:23:37.452 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout: "fbe96596-3c2e-4a1d-ad3e-8fdfa5af07b7": { 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "fbe96596-3c2e-4a1d-ad3e-8fdfa5af07b7", 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:23:37.453 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fed280a1310 msgr2=0x7fed280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fed280a1310 0x7fed280a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fed38009c60 tx=0x7fed38009260 comp rx=0 tx=0).stop 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 msgr2=0x7fed3c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c1979b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e970 tx=0x7fed3000ec80 comp rx=0 tx=0).stop 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 shutdown_connections 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fed280a1310 0x7fed280a37d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed3c19bd00 0x7fed3c19c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed3c101160 0x7fed3c197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 --2- 172.21.15.16:0/2414138712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed3c106280 0x7fed3c1979b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:23:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 >> 172.21.15.16:0/2414138712 conn(0x7fed3c0fc9e0 msgr2=0x7fed3c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:23:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 shutdown_connections 2024-05-08T23:23:37.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:23:37.446+0000 7fed439db700 1 -- 172.21.15.16:0/2414138712 wait complete. 2024-05-08T23:23:37.458 INFO:tasks.cephfs.filesystem:scrub status for tag:fbe96596-3c2e-4a1d-ad3e-8fdfa5af07b7 - {'path': '/', 'tag': 'fbe96596-3c2e-4a1d-ad3e-8fdfa5af07b7', 'options': 'recursive,force'} 2024-05-08T23:23:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:38 smithi016 ceph-mon[42541]: pgmap v125: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:38.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:38 smithi189 ceph-mon[46919]: pgmap v125: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:38 smithi155 ceph-mon[47105]: pgmap v125: 129 pgs: 129 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:39.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:39 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:23:39.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:39 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:39 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:23:39.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:39 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:39 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:23:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:39 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:39 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:23:39.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:39 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:39 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:23:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:39 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:39 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:23:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:39 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:23:40.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:40 smithi016 ceph-mon[42541]: pgmap v126: 129 pgs: 129 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-05-08T23:23:40.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:40 smithi189 ceph-mon[46919]: pgmap v126: 129 pgs: 129 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-05-08T23:23:40.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:40 smithi155 ceph-mon[47105]: pgmap v126: 129 pgs: 129 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-05-08T23:23:41.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:41 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:23:41.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:41 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:23:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:41 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:23:42.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:42 smithi016 ceph-mon[42541]: pgmap v127: 129 pgs: 129 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:42.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:42 smithi189 ceph-mon[46919]: pgmap v127: 129 pgs: 129 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:43.148 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:42 smithi155 ceph-mon[47105]: pgmap v127: 129 pgs: 129 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:64/600: open fd = 3 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:65/600: open fd = 3 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:66/600: open fd = 3 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.736 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:67/600: open fd = 3 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:68/600: open fd = 3 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:69/600: open fd = 3 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.737 INFO:tasks.workunit.client.0.smithi016.stdout:70/600: open fd = 3 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:71/600: open fd = 3 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:72/600: open fd = 3 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:73/600: open fd = 3 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.738 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:74/600: open fd = 3 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:75/600: open fd = 3 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:76/600: open fd = 3 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.739 INFO:tasks.workunit.client.0.smithi016.stdout:77/600: open fd = 3 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:78/600: open fd = 3 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:79/600: open fd = 3 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:80/600: open fd = 3 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.740 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:81/600: open fd = 3 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:82/600: open fd = 3 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:83/600: open fd = 3 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.741 INFO:tasks.workunit.client.0.smithi016.stdout:84/600: open fd = 3 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:85/600: open fd = 3 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:86/600: open fd = 3 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:87/600: open fd = 3 2024-05-08T23:23:43.742 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:88/600: open fd = 3 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:89/600: open fd = 3 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:90/600: open fd = 3 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.743 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:91/600: open fd = 3 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:92/600: open fd = 3 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:93/600: open fd = 3 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:94/600: open fd = 3 2024-05-08T23:23:43.744 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:95/600: open fd = 3 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:96/600: open fd = 3 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:97/600: open fd = 3 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.745 INFO:tasks.workunit.client.0.smithi016.stdout:98/600: open fd = 3 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:99/600: open fd = 3 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:100/600: open fd = 3 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:101/600: open fd = 3 2024-05-08T23:23:43.746 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:102/600: open fd = 3 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:103/600: open fd = 3 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:104/600: open fd = 3 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.747 INFO:tasks.workunit.client.0.smithi016.stdout:105/600: open fd = 3 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:106/600: open fd = 3 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:107/600: open fd = 3 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:108/600: open fd = 3 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.748 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:109/600: open fd = 3 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:110/600: open fd = 3 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:111/600: open fd = 3 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:112/600: open fd = 3 2024-05-08T23:23:43.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:113/600: open fd = 3 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:114/600: open fd = 3 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:115/600: open fd = 3 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:116/600: open fd = 3 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:117/600: open fd = 3 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:118/600: open fd = 3 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:119/600: open fd = 3 2024-05-08T23:23:43.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:120/600: open fd = 3 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:121/600: open fd = 3 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:122/600: open fd = 3 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.752 INFO:tasks.workunit.client.0.smithi016.stdout:123/600: open fd = 3 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:124/600: open fd = 3 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:125/600: open fd = 3 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:126/600: open fd = 3 2024-05-08T23:23:43.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:23:44.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:44 smithi016 ceph-mon[42541]: pgmap v128: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:44 smithi189 ceph-mon[46919]: pgmap v128: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:44 smithi155 ceph-mon[47105]: pgmap v128: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:45 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:23:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:45 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:45 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:45 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:45 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:45 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:23:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:45 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:45 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:45 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:45 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:45 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:23:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:45 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:45 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:45 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:45.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:45 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:46.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:23:46.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:23:46.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:46.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:46 smithi016 ceph-mon[42541]: pgmap v129: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:46.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:23:46.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:23:46.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:46.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:46 smithi189 ceph-mon[46919]: pgmap v129: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:23:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:23:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:23:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:46 smithi155 ceph-mon[47105]: pgmap v129: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:48 smithi016 ceph-mon[42541]: pgmap v130: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:48.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:48 smithi189 ceph-mon[46919]: pgmap v130: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:48.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:48 smithi155 ceph-mon[47105]: pgmap v130: 129 pgs: 129 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:23:50.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:50 smithi016 ceph-mon[42541]: pgmap v131: 129 pgs: 129 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:50 smithi189 ceph-mon[46919]: pgmap v131: 129 pgs: 129 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:50.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:50 smithi155 ceph-mon[47105]: pgmap v131: 129 pgs: 129 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:23:52.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:52 smithi016 ceph-mon[42541]: pgmap v132: 129 pgs: 129 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:52.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:52 smithi189 ceph-mon[46919]: pgmap v132: 129 pgs: 129 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:52 smithi155 ceph-mon[47105]: pgmap v132: 129 pgs: 129 active+clean; 3.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:54.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:54 smithi016 ceph-mon[42541]: pgmap v133: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-05-08T23:23:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:54 smithi189 ceph-mon[46919]: pgmap v133: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-05-08T23:23:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:54 smithi155 ceph-mon[47105]: pgmap v133: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-05-08T23:23:56.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:56 smithi016 ceph-mon[42541]: pgmap v134: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:56.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:56 smithi189 ceph-mon[46919]: pgmap v134: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:56.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:56 smithi155 ceph-mon[47105]: pgmap v134: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:58.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:23:58 smithi016 ceph-mon[42541]: pgmap v135: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:58.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:23:58 smithi189 ceph-mon[46919]: pgmap v135: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:23:58.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:23:58 smithi155 ceph-mon[47105]: pgmap v135: 129 pgs: 129 active+clean; 3.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-05-08T23:24:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:00 smithi016 ceph-mon[42541]: pgmap v136: 129 pgs: 129 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:24:00.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:00 smithi189 ceph-mon[46919]: pgmap v136: 129 pgs: 129 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:24:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:00 smithi155 ceph-mon[47105]: pgmap v136: 129 pgs: 129 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-05-08T23:24:02.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:02 smithi016 ceph-mon[42541]: pgmap v137: 129 pgs: 129 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:24:02.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:02 smithi189 ceph-mon[46919]: pgmap v137: 129 pgs: 129 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:24:02.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:02 smithi155 ceph-mon[47105]: pgmap v137: 129 pgs: 129 active+clean; 4.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T23:24:04.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:04 smithi016 ceph-mon[42541]: pgmap v138: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-08T23:24:04.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:04 smithi189 ceph-mon[46919]: pgmap v138: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-08T23:24:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:04 smithi155 ceph-mon[47105]: pgmap v138: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-05-08T23:24:06.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:06 smithi016 ceph-mon[42541]: pgmap v139: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-08T23:24:06.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:06 smithi189 ceph-mon[46919]: pgmap v139: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-08T23:24:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:06 smithi155 ceph-mon[47105]: pgmap v139: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-08T23:24:07.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:24:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 -- 172.21.15.16:0/279538085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac106280 msgr2=0x7f48ac10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 --2- 172.21.15.16:0/279538085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac106280 0x7f48ac10de00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b0a0 tx=0x7f48a400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 -- 172.21.15.16:0/279538085 shutdown_connections 2024-05-08T23:24:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 --2- 172.21.15.16:0/279538085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac106280 0x7f48ac10de00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 --2- 172.21.15.16:0/279538085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48ac101160 0x7f48ac101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 --2- 172.21.15.16:0/279538085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48ac101b10 0x7f48ac105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 -- 172.21.15.16:0/279538085 >> 172.21.15.16:0/279538085 conn(0x7f48ac0fc9e0 msgr2=0x7f48ac0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 -- 172.21.15.16:0/279538085 shutdown_connections 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.608+0000 7f48b301c700 1 -- 172.21.15.16:0/279538085 wait complete. 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 Processor -- start 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 -- start start 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48ac101160 0x7f48ac197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48ac106280 0x7f48ac197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 0x7f48ac19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac19c790 con 0x7f48ac19bd10 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac19c900 con 0x7f48ac101160 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b301c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac19cac0 con 0x7f48ac106280 2024-05-08T23:24:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b15b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 0x7f48ac19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b15b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 0x7f48ac19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45446/0 (socket says 172.21.15.16:45446) 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b15b9700 1 -- 172.21.15.16:0/2156462234 learned_addr learned my addr 172.21.15.16:0/2156462234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b0db8700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48ac101160 0x7f48ac197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48abfff700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48ac106280 0x7f48ac197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b15b9700 1 -- 172.21.15.16:0/2156462234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48ac106280 msgr2=0x7f48ac197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.609+0000 7f48b15b9700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48ac106280 0x7f48ac197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48b15b9700 1 -- 172.21.15.16:0/2156462234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48ac101160 msgr2=0x7f48ac197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48b15b9700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48ac101160 0x7f48ac197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48b15b9700 1 -- 172.21.15.16:0/2156462234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4009d00 con 0x7f48ac19bd10 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48b15b9700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 0x7f48ac19c190 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48a4006130 tx=0x7f48a4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48a9ffb700 1 -- 172.21.15.16:0/2156462234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400e040 con 0x7f48ac19bd10 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48a9ffb700 1 -- 172.21.15.16:0/2156462234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a401b450 con 0x7f48ac19bd10 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac1a3fc0 con 0x7f48ac19bd10 2024-05-08T23:24:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48a9ffb700 1 -- 172.21.15.16:0/2156462234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40052a0 con 0x7f48ac19bd10 2024-05-08T23:24:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.610+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac1a4490 con 0x7f48ac19bd10 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.613+0000 7f48a9ffb700 1 -- 172.21.15.16:0/2156462234 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f48a401b5c0 con 0x7f48ac19bd10 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.613+0000 7f48a9ffb700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f48940a1310 0x7f48940a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48b0db8700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f48940a1310 0x7f48940a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48a9ffb700 1 -- 172.21.15.16:0/2156462234 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f48a40ce2a0 con 0x7f48ac19bd10 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 Processor -- start 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48b0db8700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f48940a1310 0x7f48940a37d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f489c009c60 tx=0x7f489c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 -- start start 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d3545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d34e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d34f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d2a1150 con 0x55674d3541f0 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d29eb10 con 0x55674d354b10 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.614+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d29a400 con 0x55674d34eca0 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d3545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d3545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45458/0 (socket says 172.21.15.16:45458) 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 -- 172.21.15.16:0/3046103538 learned_addr learned my addr 172.21.15.16:0/3046103538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48abfff700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d34e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b15b9700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d34f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 -- 172.21.15.16:0/3046103538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 msgr2=0x55674d34f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d34f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 -- 172.21.15.16:0/3046103538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 msgr2=0x55674d34e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d34e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 -- 172.21.15.16:0/3046103538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55674d11b3b0 con 0x55674d3541f0 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48abfff700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d34e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b15b9700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d34f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d3545d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f489c00e4e0 tx=0x7f489c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d091ae8e5672c71f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f4892ffd700 1 -- 172.21.15.16:0/3046103538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c026430 con 0x55674d3541f0 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f4892ffd700 1 -- 172.21.15.16:0/3046103538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f489c021070 con 0x55674d3541f0 2024-05-08T23:24:07.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.615+0000 7f4892ffd700 1 -- 172.21.15.16:0/3046103538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c01d670 con 0x55674d3541f0 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3046103538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 msgr2=0x55674d3545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d3545d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f489c00e4e0 tx=0x7f489c014000 comp rx=0 tx=0).stop 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3046103538 shutdown_connections 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d34f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d3545d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3046103538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d34e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3046103538 >> 172.21.15.16:0/3046103538 conn(0x55674d2a8210 msgr2=0x55674d344df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3046103538 shutdown_connections 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.616+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3046103538 wait complete. 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 -- start start 2024-05-08T23:24:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 Processor -- start 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d357ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d358400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 0x55674d358940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d236560 con 0x55674d354b10 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d236200 con 0x55674d3541f0 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d11d6a0 con 0x55674d34eca0 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.617+0000 7f48bf1d6b80 10 client.0 _ll_register_callbacks cb 0x55674d23fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b15b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 0x55674d358940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d357ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d357ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:40366/0 (socket says 172.21.15.16:40366) 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b15b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 0x55674d358940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45472/0 (socket says 172.21.15.16:45472) 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 -- 172.21.15.16:0/3728651606 learned_addr learned my addr 172.21.15.16:0/3728651606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48abfff700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d358400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 -- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 msgr2=0x55674d358400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d358400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 -- 172.21.15.16:0/3728651606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 msgr2=0x55674d358940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 0x55674d358940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 -- 172.21.15.16:0/3728651606 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c017070 con 0x55674d34eca0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b15b9700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 0x55674d358940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d357ec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f489c005950 tx=0x7f489c01cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679a07bbc47ce136 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f488bfff700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f488bfff700 1 -- 172.21.15.16:0/3728651606 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c01c620 con 0x55674d34eca0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55674d1f3270 con 0x55674d34eca0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.618+0000 7f488bfff700 1 -- 172.21.15.16:0/3728651606 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f489c01c780 con 0x55674d34eca0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f488bfff700 1 -- 172.21.15.16:0/3728651606 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489c01c940 con 0x55674d34eca0 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 10 client.26287 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55674d0c7b60 con 0x55674d34eca0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f488bfff700 1 -- 172.21.15.16:0/3728651606 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55674d1f3270 con 0x55674d34eca0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f488bfff700 1 -- 172.21.15.16:0/3728651606 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f489c0038b0 con 0x55674d34eca0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 20 client.26287 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3683c0 0x55674d36a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 4 client.26287 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55674d319810 con 0x55674d3683c0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.619+0000 7f48abfff700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3683c0 0x55674d36a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.620+0000 7f48abfff700 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3683c0 0x55674d36a7e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c5c114a8fb199c server_cookie=159346fc99e61245 in_seq=0 out_seq=0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.620+0000 7f488bfff700 10 client.26287 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.620+0000 7f488bfff700 1 -- 172.21.15.16:0/3728651606 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55674d319810 con 0x55674d3683c0 2024-05-08T23:24:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.620+0000 7f488bfff700 10 client.26287 handle_command_reply: tid=0 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.620+0000 7f48bf1d6b80 1 client.26287 shutdown 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 msgr2=0x55674d357ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d357ec0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f489c005950 tx=0x7f489c01cdb0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 shutdown_connections 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d354b10 0x55674d358940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d357ec0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3683c0 0x55674d36a7e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3728651606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d358400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f488bfff700 0 client.26287 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 >> 172.21.15.16:0/3728651606 conn(0x55674d2a8210 msgr2=0x55674d3445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 shutdown_connections 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3728651606 wait complete. 2024-05-08T23:24:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.621+0000 7f48bf1d6b80 20 client.26287 trim_cache size 0 max 16384 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 Processor -- start 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 -- start start 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 0x55674d353820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d344d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d354b10 0x55674d358760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d359470 con 0x55674d34eca0 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d34f660 con 0x55674d3541f0 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d341dc0 con 0x55674d354b10 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48b0db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 0x55674d353820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48b0db8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 0x55674d353820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45482/0 (socket says 172.21.15.16:45482) 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.639+0000 7f48b0db8700 1 -- 172.21.15.16:0/1723927585 learned_addr learned my addr 172.21.15.16:0/1723927585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48b0db8700 1 -- 172.21.15.16:0/1723927585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d354b10 msgr2=0x55674d358760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48b0db8700 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d354b10 0x55674d358760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48b0db8700 1 -- 172.21.15.16:0/1723927585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 msgr2=0x55674d344d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48b0db8700 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d344d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48b0db8700 1 -- 172.21.15.16:0/1723927585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55674d260560 con 0x55674d34eca0 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48b0db8700 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 0x55674d353820 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f489c009ec0 tx=0x7f489c009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8111669536aeeef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48937fe700 1 -- 172.21.15.16:0/1723927585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0092d0 con 0x55674d34eca0 2024-05-08T23:24:07.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48937fe700 1 -- 172.21.15.16:0/1723927585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f489c00ed60 con 0x55674d34eca0 2024-05-08T23:24:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.640+0000 7f48937fe700 1 -- 172.21.15.16:0/1723927585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0053f0 con 0x55674d34eca0 2024-05-08T23:24:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/1723927585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 msgr2=0x55674d353820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 0x55674d353820 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f489c009ec0 tx=0x7f489c009e50 comp rx=0 tx=0).stop 2024-05-08T23:24:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/1723927585 shutdown_connections 2024-05-08T23:24:07.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d354b10 0x55674d358760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d34eca0 0x55674d353820 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/1723927585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d3541f0 0x55674d344d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/1723927585 >> 172.21.15.16:0/1723927585 conn(0x55674d2a8210 msgr2=0x55674d356790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/1723927585 shutdown_connections 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/1723927585 wait complete. 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 -- start start 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.641+0000 7f48bf1d6b80 1 Processor -- start 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d35d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d35d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d35dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d3bd9b0 con 0x55674d3541f0 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d3bdb20 con 0x55674d354b10 2024-05-08T23:24:07.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674d3bdce0 con 0x55674d34eca0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48bf1d6b80 10 client.0 _ll_register_callbacks cb 0x55674d23fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d35d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d35d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45496/0 (socket says 172.21.15.16:45496) 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 -- 172.21.15.16:0/3803620025 learned_addr learned my addr 172.21.15.16:0/3803620025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d35d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 -- 172.21.15.16:0/3803620025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 msgr2=0x55674d35d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d35d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 -- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 msgr2=0x55674d35dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d35dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 -- 172.21.15.16:0/3803620025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55674d0c7b60 con 0x55674d3541f0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d35d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f48abfff700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d35d7c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f48ac198980 tx=0x7f48a0003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbe925a51fc5887 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f488bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.642+0000 7f488bfff700 1 -- 172.21.15.16:0/3803620025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0017040 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f488bfff700 1 -- 172.21.15.16:0/3803620025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f48a0009980 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f489c021070 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f488bfff700 1 -- 172.21.15.16:0/3803620025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48a001d8e0 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 10 client.16527 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55674d11fcd0 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f488bfff700 1 -- 172.21.15.16:0/3803620025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f489c021070 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f488bfff700 1 -- 172.21.15.16:0/3803620025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f48a0012ae0 con 0x55674d3541f0 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 10 client.16527 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 20 client.16527 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3bf540 0x55674d3c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 4 client.16527 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55674d292ec0 con 0x55674d3bf540 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.643+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3bf540 0x55674d3c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f488bfff700 10 client.16527 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48b0db8700 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3bf540 0x55674d3c1960 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cee47b38391caae5 server_cookie=6cb19361c9c1366b in_seq=0 out_seq=0 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f488bfff700 1 -- 172.21.15.16:0/3803620025 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55674d292ec0 con 0x55674d3bf540 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f488bfff700 10 client.16527 handle_command_reply: tid=0 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 client.16527 shutdown 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 msgr2=0x55674d35d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d35d7c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f48ac198980 tx=0x7f48a0003e30 comp rx=0 tx=0).stop 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 shutdown_connections 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55674d3bf540 0x55674d3c1960 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55674d354b10 0x55674d35dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674d34eca0 0x55674d35d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 --2- 172.21.15.16:0/3803620025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674d3541f0 0x55674d35d7c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f488bfff700 0 client.16527 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 >> 172.21.15.16:0/3803620025 conn(0x55674d2a8210 msgr2=0x55674d3587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 shutdown_connections 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 1 -- 172.21.15.16:0/3803620025 wait complete. 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.644+0000 7f48bf1d6b80 20 client.16527 trim_cache size 0 max 16384 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:24:07.652 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f48940a1310 msgr2=0x7f48940a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f48940a1310 0x7f48940a37d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f489c009c60 tx=0x7f489c009260 comp rx=0 tx=0).stop 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 msgr2=0x7f48ac19c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 0x7f48ac19c190 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48a4006130 tx=0x7f48a4009500 comp rx=0 tx=0).stop 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 shutdown_connections 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f48940a1310 0x7f48940a37d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ac19bd10 0x7f48ac19c190 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48ac101160 0x7f48ac197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 --2- 172.21.15.16:0/2156462234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48ac106280 0x7f48ac197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 >> 172.21.15.16:0/2156462234 conn(0x7f48ac0fc9e0 msgr2=0x7f48ac0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 shutdown_connections 2024-05-08T23:24:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.645+0000 7f48b301c700 1 -- 172.21.15.16:0/2156462234 wait complete. 2024-05-08T23:24:07.659 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:24:07.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.795+0000 7f6c7a14d700 1 -- 172.21.15.16:0/593577294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74100900 msgr2=0x7f6c74104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.795+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/593577294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74100900 0x7f6c74104940 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009990 tx=0x7f6c68009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 -- 172.21.15.16:0/593577294 shutdown_connections 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/593577294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c74105070 0x7f6c7410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/593577294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c740fff50 0x7f6c74100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/593577294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c74100900 0x7f6c74104940 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 -- 172.21.15.16:0/593577294 >> 172.21.15.16:0/593577294 conn(0x7f6c740fb810 msgr2=0x7f6c740fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 -- 172.21.15.16:0/593577294 shutdown_connections 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 -- 172.21.15.16:0/593577294 wait complete. 2024-05-08T23:24:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.796+0000 7f6c7a14d700 1 Processor -- start 2024-05-08T23:24:07.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 -- start start 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740fff50 0x7f6c74197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c74105070 0x7f6c741977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 0x7f6c7419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419c5c0 con 0x7f6c7419bb40 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419c730 con 0x7f6c740fff50 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c7a14d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419c8f0 con 0x7f6c74105070 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 0x7f6c7419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 0x7f6c7419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45524/0 (socket says 172.21.15.16:45524) 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 -- 172.21.15.16:0/1985290105 learned_addr learned my addr 172.21.15.16:0/1985290105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c72ffd700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c74105070 0x7f6c741977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c737fe700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740fff50 0x7f6c74197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 -- 172.21.15.16:0/1985290105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c74105070 msgr2=0x7f6c741977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c74105070 0x7f6c741977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 -- 172.21.15.16:0/1985290105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740fff50 msgr2=0x7f6c74197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740fff50 0x7f6c74197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 -- 172.21.15.16:0/1985290105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680095f0 con 0x7f6c7419bb40 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.797+0000 7f6c73fff700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 0x7f6c7419bfc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b970 tx=0x7f6c6c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.798+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00c930 con 0x7f6c7419bb40 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.798+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c6c014440 con 0x7f6c7419bb40 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.798+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c741a3e50 con 0x7f6c7419bb40 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.798+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0075d0 con 0x7f6c7419bb40 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.798+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c741a42e0 con 0x7f6c7419bb40 2024-05-08T23:24:07.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.799+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c74109350 con 0x7f6c7419bb40 2024-05-08T23:24:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.800+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6c6c0145b0 con 0x7f6c7419bb40 2024-05-08T23:24:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.801+0000 7f6c70ff9700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6c5c0a1290 0x7f6c5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.801+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f6c6c006e40 con 0x7f6c7419bb40 2024-05-08T23:24:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.801+0000 7f6c737fe700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6c5c0a1290 0x7f6c5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:07.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.801+0000 7f6c737fe700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6c5c0a1290 0x7f6c5c0a3750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005950 tx=0x7f6c640058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:07.804+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6c6c0960c0 con 0x7f6c7419bb40 2024-05-08T23:24:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.076+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c74198560 con 0x7f6c7419bb40 2024-05-08T23:24:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.076+0000 7f6c70ff9700 1 -- 172.21.15.16:0/1985290105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f6c6c019090 con 0x7f6c7419bb40 2024-05-08T23:24:08.077 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:24:08.077 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.078+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6c5c0a1290 msgr2=0x7f6c5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.078+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6c5c0a1290 0x7f6c5c0a3750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005950 tx=0x7f6c640058e0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.078+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 msgr2=0x7f6c7419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.078+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 0x7f6c7419bfc0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b970 tx=0x7f6c6c00bc80 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 shutdown_connections 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6c5c0a1290 0x7f6c5c0a3750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7419bb40 0x7f6c7419bfc0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c740fff50 0x7f6c74197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 --2- 172.21.15.16:0/1985290105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c74105070 0x7f6c741977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 >> 172.21.15.16:0/1985290105 conn(0x7f6c740fb810 msgr2=0x7f6c74069690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 shutdown_connections 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.079+0000 7f6c7a14d700 1 -- 172.21.15.16:0/1985290105 wait complete. 2024-05-08T23:24:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:24:08.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 -- 172.21.15.16:0/169659366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0101180 msgr2=0x7f5ca0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/169659366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0101180 0x7f5ca0101560 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009990 tx=0x7f5c90009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 -- 172.21.15.16:0/169659366 shutdown_connections 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/169659366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca01062a0 0x7f5ca010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/169659366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0101180 0x7f5ca0101560 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/169659366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca0101b30 0x7f5ca0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 -- 172.21.15.16:0/169659366 >> 172.21.15.16:0/169659366 conn(0x7f5ca00fc9e0 msgr2=0x7f5ca00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 -- 172.21.15.16:0/169659366 shutdown_connections 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.233+0000 7f5ca61dd700 1 -- 172.21.15.16:0/169659366 wait complete. 2024-05-08T23:24:08.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.234+0000 7f5ca61dd700 1 Processor -- start 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.234+0000 7f5ca61dd700 1 -- start start 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.234+0000 7f5ca61dd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca0101180 0x7f5ca00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5ca61dd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca01062a0 0x7f5ca00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5ca61dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 0x7f5ca01a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5ca61dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01a5b80 con 0x7f5ca0073d20 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5ca61dd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01a5cf0 con 0x7f5ca01062a0 2024-05-08T23:24:08.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5ca61dd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01a5eb0 con 0x7f5ca0101180 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 0x7f5ca01a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 0x7f5ca01a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45544/0 (socket says 172.21.15.16:45544) 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 -- 172.21.15.16:0/3722472968 learned_addr learned my addr 172.21.15.16:0/3722472968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca0101180 0x7f5ca00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca01062a0 0x7f5ca00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 -- 172.21.15.16:0/3722472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca0101180 msgr2=0x7f5ca00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca0101180 0x7f5ca00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 -- 172.21.15.16:0/3722472968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca01062a0 msgr2=0x7f5ca00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca01062a0 0x7f5ca00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 -- 172.21.15.16:0/3722472968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900095f0 con 0x7f5ca0073d20 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca01062a0 0x7f5ca00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 0x7f5ca01a55b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5c98007ee0 tx=0x7f5c9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9cff9700 1 -- 172.21.15.16:0/3722472968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98017070 con 0x7f5ca0073d20 2024-05-08T23:24:08.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9cff9700 1 -- 172.21.15.16:0/3722472968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c9800f040 con 0x7f5ca0073d20 2024-05-08T23:24:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca01a6190 con 0x7f5ca0073d20 2024-05-08T23:24:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.235+0000 7f5c9cff9700 1 -- 172.21.15.16:0/3722472968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98005710 con 0x7f5ca0073d20 2024-05-08T23:24:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.236+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca01a6650 con 0x7f5ca0073d20 2024-05-08T23:24:08.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.238+0000 7f5c9cff9700 1 -- 172.21.15.16:0/3722472968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5c98005870 con 0x7f5ca0073d20 2024-05-08T23:24:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.239+0000 7f5c9cff9700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c880a1290 0x7f5c880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.239+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c880a1290 0x7f5c880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.239+0000 7f5c9cff9700 1 -- 172.21.15.16:0/3722472968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5c980cc830 con 0x7f5ca0073d20 2024-05-08T23:24:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.239+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c880a1290 0x7f5c880a3750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000b5c0 tx=0x7f5c90005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.239+0000 7f5cb2397b80 1 Processor -- start 2024-05-08T23:24:08.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 -- start start 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 0x562a70f3f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f3f890 0x562a70f3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70e96150 con 0x562a70f45720 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70e93b10 con 0x562a70f3f890 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70e8f400 con 0x562a70f44e00 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 0x562a70f3f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 0x562a70f3f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45558/0 (socket says 172.21.15.16:45558) 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 -- 172.21.15.16:0/3954159960 learned_addr learned my addr 172.21.15.16:0/3954159960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f3f890 0x562a70f3fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 -- 172.21.15.16:0/3954159960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 msgr2=0x562a70f451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 -- 172.21.15.16:0/3954159960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f3f890 msgr2=0x562a70f3fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f3f890 0x562a70f3fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9effd700 1 -- 172.21.15.16:0/3954159960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a70d103b0 con 0x562a70f45720 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.240+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.241+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 0x562a70f3f350 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5c94008c40 tx=0x7f5c94008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adade474303345e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.241+0000 7f5c85ffb700 1 -- 172.21.15.16:0/3954159960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400ed70 con 0x562a70f45720 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.241+0000 7f5c85ffb700 1 -- 172.21.15.16:0/3954159960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c9400eed0 con 0x562a70f45720 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.241+0000 7f5c85ffb700 1 -- 172.21.15.16:0/3954159960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94010430 con 0x562a70f45720 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.241+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3954159960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 msgr2=0x562a70f3f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.241+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 0x562a70f3f350 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5c94008c40 tx=0x7f5c94008f50 comp rx=0 tx=0).stop 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3954159960 shutdown_connections 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f3f890 0x562a70f3fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3954159960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f45720 0x562a70f3f350 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3954159960 >> 172.21.15.16:0/3954159960 conn(0x562a70e9d210 msgr2=0x562a70f39df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3954159960 shutdown_connections 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3954159960 wait complete. 2024-05-08T23:24:08.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 -- start start 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.242+0000 7f5cb2397b80 1 Processor -- start 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f48b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f49060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70e2b560 con 0x562a70f3f890 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70e2b200 con 0x562a70f45720 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70d126a0 con 0x562a70f44e00 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5cb2397b80 10 client.0 _ll_register_callbacks cb 0x562a70e34b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f48b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f48b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45566/0 (socket says 172.21.15.16:45566) 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/3394547486 learned_addr learned my addr 172.21.15.16:0/3394547486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f49060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 msgr2=0x562a70f49060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f49060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/3394547486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 msgr2=0x562a70f495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/3394547486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c94008870 con 0x562a70f3f890 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f49060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f48b20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5c90015000 tx=0x7f5c9001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8a76a6d8bf1976 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.243+0000 7f5c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5c7effd700 1 -- 172.21.15.16:0/3394547486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90004ba0 con 0x562a70f3f890 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5c7effd700 1 -- 172.21.15.16:0/3394547486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5c90004d00 con 0x562a70f3f890 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a70de8270 con 0x562a70f3f890 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5c7effd700 1 -- 172.21.15.16:0/3394547486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c9000f460 con 0x562a70f3f890 2024-05-08T23:24:08.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 10 client.16545 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a70cbcb60 con 0x562a70f3f890 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5c7effd700 1 -- 172.21.15.16:0/3394547486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a70de8270 con 0x562a70f3f890 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5c7effd700 1 -- 172.21.15.16:0/3394547486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c90030420 con 0x562a70f3f890 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 10 client.16545 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 10 client.16545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 20 client.16545 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70f592f0 0x562a70f5b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 4 client.16545 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.244+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a70f0e810 con 0x562a70f592f0 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.245+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70f592f0 0x562a70f5b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.245+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70f592f0 0x562a70f5b710 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=caa17ae534941588 server_cookie=71212ccbad84d19c in_seq=0 out_seq=0 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.245+0000 7f5c7effd700 10 client.16545 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5c7effd700 1 -- 172.21.15.16:0/3394547486 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a70f0e810 con 0x562a70f592f0 2024-05-08T23:24:08.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5c7effd700 10 client.16545 handle_command_reply: tid=0 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 client.16545 shutdown 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 msgr2=0x562a70f48b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f48b20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5c90015000 tx=0x7f5c9001a050 comp rx=0 tx=0).stop 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 shutdown_connections 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70f592f0 0x562a70f5b710 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f48b20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5c7effd700 0 client.16545 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3394547486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f44e00 0x562a70f49060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 >> 172.21.15.16:0/3394547486 conn(0x562a70e9d210 msgr2=0x562a70f395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 shutdown_connections 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3394547486 wait complete. 2024-05-08T23:24:08.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.246+0000 7f5cb2397b80 20 client.16545 trim_cache size 0 max 16384 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.260+0000 7f5cb2397b80 1 Processor -- start 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.260+0000 7f5cb2397b80 1 -- start start 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f54c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f44e00 0x562a70f39d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f45720 0x562a70f50160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70f4b680 con 0x562a70f3f890 2024-05-08T23:24:08.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70f36df0 con 0x562a70f44e00 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70f3be50 con 0x562a70f45720 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f54c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f54c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45576/0 (socket says 172.21.15.16:45576) 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/616103470 learned_addr learned my addr 172.21.15.16:0/616103470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f45720 0x562a70f50160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9effd700 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f44e00 0x562a70f39d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/616103470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f45720 msgr2=0x562a70f50160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f45720 0x562a70f50160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/616103470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f44e00 msgr2=0x562a70f39d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f44e00 0x562a70f39d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 -- 172.21.15.16:0/616103470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a70e55560 con 0x562a70f3f890 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9effd700 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f44e00 0x562a70f39d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f54c20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5c90005400 tx=0x7f5c900098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2992a69e0fcf2f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c867fc700 1 -- 172.21.15.16:0/616103470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90030aa0 con 0x562a70f3f890 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c867fc700 1 -- 172.21.15.16:0/616103470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c9001e040 con 0x562a70f3f890 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.261+0000 7f5c867fc700 1 -- 172.21.15.16:0/616103470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000f880 con 0x562a70f3f890 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 -- 172.21.15.16:0/616103470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 msgr2=0x562a70f54c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f54c20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5c90005400 tx=0x7f5c900098c0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 -- 172.21.15.16:0/616103470 shutdown_connections 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f45720 0x562a70f50160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f3f890 0x562a70f54c20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/616103470 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f44e00 0x562a70f39d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 -- 172.21.15.16:0/616103470 >> 172.21.15.16:0/616103470 conn(0x562a70e9d210 msgr2=0x562a70f3fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 -- 172.21.15.16:0/616103470 shutdown_connections 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 -- 172.21.15.16:0/616103470 wait complete. 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.262+0000 7f5cb2397b80 1 -- start start 2024-05-08T23:24:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 Processor -- start 2024-05-08T23:24:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f3f890 0x562a70f50360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 0x562a70f409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f40ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70f41460 con 0x562a70f44e00 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70f415d0 con 0x562a70f45720 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a70f41790 con 0x562a70f3f890 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5cb2397b80 10 client.0 _ll_register_callbacks cb 0x562a70e34b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 0x562a70f409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 0x562a70f409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45584/0 (socket says 172.21.15.16:45584) 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 -- 172.21.15.16:0/3168285397 learned_addr learned my addr 172.21.15.16:0/3168285397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f3f890 0x562a70f50360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9ffff700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f40ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 -- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f3f890 msgr2=0x562a70f50360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f3f890 0x562a70f50360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 -- 172.21.15.16:0/3168285397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 msgr2=0x562a70f40ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.263+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f40ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c9effd700 1 -- 172.21.15.16:0/3168285397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a70cbcb60 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c9effd700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 0x562a70f409a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5c94008c40 tx=0x7f5c940085a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ad77959259e0e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c7effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c7effd700 1 -- 172.21.15.16:0/3168285397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940053b0 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c7effd700 1 -- 172.21.15.16:0/3168285397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5c94005510 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c9400eed0 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c7effd700 1 -- 172.21.15.16:0/3168285397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c940056d0 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5cb2397b80 10 client.16551 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562a70d14cd0 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c7effd700 1 -- 172.21.15.16:0/3168285397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c9400eed0 con 0x562a70f44e00 2024-05-08T23:24:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5c7effd700 1 -- 172.21.15.16:0/3168285397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c94027590 con 0x562a70f44e00 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5cb2397b80 10 client.16551 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5cb2397b80 10 client.16551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.264+0000 7f5cb2397b80 20 client.16551 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70fafd90 0x562a70fb21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5cb2397b80 4 client.16551 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a70e87ec0 con 0x562a70fafd90 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70fafd90 0x562a70fb21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5c9f7fe700 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70fafd90 0x562a70fb21b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91f59ae85f4621f8 server_cookie=cbf69a65264db1e6 in_seq=0 out_seq=0 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5c7effd700 10 client.16551 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5c7effd700 1 -- 172.21.15.16:0/3168285397 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a70e87ec0 con 0x562a70fafd90 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5c7effd700 10 client.16551 handle_command_reply: tid=0 2024-05-08T23:24:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.265+0000 7f5cb2397b80 1 client.16551 shutdown 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 msgr2=0x562a70f409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 0x562a70f409a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5c94008c40 tx=0x7f5c940085a0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 shutdown_connections 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x562a70fafd90 0x562a70fb21b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562a70f45720 0x562a70f40ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5c7effd700 0 client.16551 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a70f3f890 0x562a70f50360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 --2- 172.21.15.16:0/3168285397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a70f44e00 0x562a70f409a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 >> 172.21.15.16:0/3168285397 conn(0x562a70e9d210 msgr2=0x562a70f473f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 shutdown_connections 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 1 -- 172.21.15.16:0/3168285397 wait complete. 2024-05-08T23:24:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5cb2397b80 20 client.16551 trim_cache size 0 max 16384 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:24:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c880a1290 msgr2=0x7f5c880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c880a1290 0x7f5c880a3750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000b5c0 tx=0x7f5c90005fb0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 msgr2=0x7f5ca01a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 0x7f5ca01a55b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5c98007ee0 tx=0x7f5c9800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 shutdown_connections 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5c880a1290 0x7f5c880a3750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ca0073d20 0x7f5ca01a55b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca0101180 0x7f5ca00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.266+0000 7f5ca61dd700 1 --2- 172.21.15.16:0/3722472968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ca01062a0 0x7f5ca00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.267+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 >> 172.21.15.16:0/3722472968 conn(0x7f5ca00fc9e0 msgr2=0x7f5ca00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.267+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 shutdown_connections 2024-05-08T23:24:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.267+0000 7f5ca61dd700 1 -- 172.21.15.16:0/3722472968 wait complete. 2024-05-08T23:24:08.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:24:08.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:08 smithi016 ceph-mon[42541]: pgmap v140: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-08T23:24:08.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:08 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1985290105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:08.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:08 smithi189 ceph-mon[46919]: pgmap v140: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-08T23:24:08.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:08 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1985290105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/2709354819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069500 msgr2=0x7fcf140698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/2709354819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069500 0x7fcf140698e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009990 tx=0x7fcf04009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/2709354819 shutdown_connections 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/2709354819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf1410d8f0 0x7fcf1410fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/2709354819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069500 0x7fcf140698e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/2709354819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14069e20 0x7fcf1410d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/2709354819 >> 172.21.15.16:0/2709354819 conn(0x7fcf14076fa0 msgr2=0x7fcf140773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/2709354819 shutdown_connections 2024-05-08T23:24:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.419+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/2709354819 wait complete. 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 Processor -- start 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 -- start start 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14069500 0x7fcf141994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 0x7fcf141999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf1410d8f0 0x7fcf1419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1419e3c0 con 0x7fcf14069e20 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1419e530 con 0x7fcf1410d8f0 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.420+0000 7fcf1a3a5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1419e6f0 con 0x7fcf14069500 2024-05-08T23:24:08.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 0x7fcf141999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 0x7fcf141999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45594/0 (socket says 172.21.15.16:45594) 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 -- 172.21.15.16:0/3568953921 learned_addr learned my addr 172.21.15.16:0/3568953921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf13fff700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14069500 0x7fcf141994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf18942700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf1410d8f0 0x7fcf1419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 -- 172.21.15.16:0/3568953921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14069500 msgr2=0x7fcf141994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14069500 0x7fcf141994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 -- 172.21.15.16:0/3568953921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf1410d8f0 msgr2=0x7fcf1419dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf1410d8f0 0x7fcf1419dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 -- 172.21.15.16:0/3568953921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf040095f0 con 0x7fcf14069e20 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf137fe700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 0x7fcf141999e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800e9a0 tx=0x7fcf0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf117fa700 1 -- 172.21.15.16:0/3568953921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08009790 con 0x7fcf14069e20 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf117fa700 1 -- 172.21.15.16:0/3568953921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf0800cbd0 con 0x7fcf14069e20 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1419e9d0 con 0x7fcf14069e20 2024-05-08T23:24:08.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf117fa700 1 -- 172.21.15.16:0/3568953921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08018680 con 0x7fcf14069e20 2024-05-08T23:24:08.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.421+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf141a6210 con 0x7fcf14069e20 2024-05-08T23:24:08.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.424+0000 7fcf117fa700 1 -- 172.21.15.16:0/3568953921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcf0800cd40 con 0x7fcf14069e20 2024-05-08T23:24:08.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.424+0000 7fcf117fa700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcefc0a1290 0x7fcefc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.424+0000 7fcf13fff700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcefc0a1290 0x7fcefc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.424+0000 7fcf117fa700 1 -- 172.21.15.16:0/3568953921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcf08014070 con 0x7fcf14069e20 2024-05-08T23:24:08.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.425+0000 7fcf13fff700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcefc0a1290 0x7fcefc0a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf040098e0 tx=0x7fcf0400b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.425+0000 7fcf2655fb80 1 Processor -- start 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.425+0000 7fcf2655fb80 1 -- start start 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc5e720 0x557c1cc58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 0x557c1cc58cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cbaf150 con 0x557c1cc58890 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cbacb10 con 0x557c1cc5de00 2024-05-08T23:24:08.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cba8400 con 0x557c1cc5e720 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 0x557c1cc58cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 0x557c1cc58cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45604/0 (socket says 172.21.15.16:45604) 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 -- 172.21.15.16:0/217865896 learned_addr learned my addr 172.21.15.16:0/217865896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf137fe700 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc5e720 0x557c1cc58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf13fff700 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 -- 172.21.15.16:0/217865896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc5e720 msgr2=0x557c1cc58350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc5e720 0x557c1cc58350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 -- 172.21.15.16:0/217865896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 msgr2=0x557c1cc5e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 -- 172.21.15.16:0/217865896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1ca293b0 con 0x557c1cc58890 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf13fff700 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcf18942700 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 0x557c1cc58cd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009d20 tx=0x7fcf0c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961946e6d2991948 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.426+0000 7fcefa7fc700 1 -- 172.21.15.16:0/217865896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c018070 con 0x557c1cc58890 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcefa7fc700 1 -- 172.21.15.16:0/217865896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf0c00f050 con 0x557c1cc58890 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcefa7fc700 1 -- 172.21.15.16:0/217865896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0059e0 con 0x557c1cc58890 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 -- 172.21.15.16:0/217865896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 msgr2=0x557c1cc58cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 0x557c1cc58cd0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009d20 tx=0x7fcf0c0077e0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 -- 172.21.15.16:0/217865896 shutdown_connections 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc58890 0x557c1cc58cd0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/217865896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc5e720 0x557c1cc58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 -- 172.21.15.16:0/217865896 >> 172.21.15.16:0/217865896 conn(0x557c1cbb6210 msgr2=0x557c1cc52df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 -- 172.21.15.16:0/217865896 shutdown_connections 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.427+0000 7fcf2655fb80 1 -- 172.21.15.16:0/217865896 wait complete. 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 -- start start 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 Processor -- start 2024-05-08T23:24:08.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc61ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc62530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cb44560 con 0x557c1cc5de00 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cb44200 con 0x557c1cc5e720 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ca2b6a0 con 0x557c1cc58890 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf2655fb80 10 client.0 _ll_register_callbacks cb 0x557c1cb4db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc61ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc61ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45614/0 (socket says 172.21.15.16:45614) 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.428+0000 7fcf13fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 -- 172.21.15.16:0/4025266513 learned_addr learned my addr 172.21.15.16:0/4025266513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf18942700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc62530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 -- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 msgr2=0x557c1cc61ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc61ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 -- 172.21.15.16:0/4025266513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 msgr2=0x557c1cc62530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc62530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 -- 172.21.15.16:0/4025266513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c009980 con 0x557c1cc5de00 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf18942700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc62530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf137fe700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc61ff0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcf0809a000 tx=0x7fcf08099ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b90c0568a443a6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcef37fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcef37fe700 1 -- 172.21.15.16:0/4025266513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0809a070 con 0x557c1cc5de00 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcef37fe700 1 -- 172.21.15.16:0/4025266513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcf0809dea0 con 0x557c1cc5de00 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c1cb01270 con 0x557c1cc5de00 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcef37fe700 1 -- 172.21.15.16:0/4025266513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf080a24f0 con 0x557c1cc5de00 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf2655fb80 10 client.16563 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.429+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c1c9d5b60 con 0x557c1cc5de00 2024-05-08T23:24:08.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcef37fe700 1 -- 172.21.15.16:0/4025266513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf08063070 con 0x557c1cc5de00 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcef37fe700 1 -- 172.21.15.16:0/4025266513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf08095e60 con 0x557c1cc5de00 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf2655fb80 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf2655fb80 10 client.16563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf2655fb80 20 client.16563 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1cc72280 0x557c1cc746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf2655fb80 4 client.16563 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c1cc27810 con 0x557c1cc72280 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf13fff700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1cc72280 0x557c1cc746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcef37fe700 10 client.16563 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:08.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.430+0000 7fcf13fff700 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1cc72280 0x557c1cc746a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=330c518d8f4bb310 server_cookie=a198ee09f8065e31 in_seq=0 out_seq=0 2024-05-08T23:24:08.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcef37fe700 1 -- 172.21.15.16:0/4025266513 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c1cc27810 con 0x557c1cc72280 2024-05-08T23:24:08.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcef37fe700 10 client.16563 handle_command_reply: tid=0 2024-05-08T23:24:08.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 client.16563 shutdown 2024-05-08T23:24:08.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 msgr2=0x557c1cc61ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc61ff0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcf0809a000 tx=0x7fcf08099ac0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 shutdown_connections 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1cc72280 0x557c1cc746a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc62530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc61ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcef37fe700 0 client.16563 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/4025266513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc61ff0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 >> 172.21.15.16:0/4025266513 conn(0x557c1cbb6210 msgr2=0x557c1cc525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 shutdown_connections 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.431+0000 7fcf2655fb80 1 -- 172.21.15.16:0/4025266513 wait complete. 2024-05-08T23:24:08.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.432+0000 7fcf2655fb80 20 client.16563 trim_cache size 0 max 16384 2024-05-08T23:24:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 Processor -- start 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 -- start start 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc54640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cc720b0 con 0x557c1cc5de00 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cc64640 con 0x557c1cc5e720 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cc79930 con 0x557c1cc58890 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc74670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45616/0 (socket says 172.21.15.16:45616) 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 -- 172.21.15.16:0/878275810 learned_addr learned my addr 172.21.15.16:0/878275810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf18942700 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc54640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf13fff700 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 -- 172.21.15.16:0/878275810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 msgr2=0x557c1cc58350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc58350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 -- 172.21.15.16:0/878275810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 msgr2=0x557c1cc54640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.446+0000 7fcf137fe700 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc54640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcf137fe700 1 -- 172.21.15.16:0/878275810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1cb6e560 con 0x557c1cc5de00 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcf13fff700 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcf137fe700 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc74670 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcf08096ae0 tx=0x7fcf08010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeb79a72753f950 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcefaffd700 1 -- 172.21.15.16:0/878275810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0809dea0 con 0x557c1cc5de00 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcefaffd700 1 -- 172.21.15.16:0/878275810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf08063070 con 0x557c1cc5de00 2024-05-08T23:24:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcefaffd700 1 -- 172.21.15.16:0/878275810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08095e60 con 0x557c1cc5de00 2024-05-08T23:24:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcf2655fb80 1 -- 172.21.15.16:0/878275810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 msgr2=0x557c1cc74670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.447+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc74670 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcf08096ae0 tx=0x7fcf08010960 comp rx=0 tx=0).stop 2024-05-08T23:24:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 -- 172.21.15.16:0/878275810 shutdown_connections 2024-05-08T23:24:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5e720 0x557c1cc54640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/878275810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5de00 0x557c1cc74670 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 -- 172.21.15.16:0/878275810 >> 172.21.15.16:0/878275810 conn(0x557c1cbb6210 msgr2=0x557c1cc603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 -- 172.21.15.16:0/878275810 shutdown_connections 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 -- 172.21.15.16:0/878275810 wait complete. 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 -- start start 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 Processor -- start 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc69460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.448+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf2655fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 0x557c1cc5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cc5b6b0 con 0x557c1cc5e720 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cc599a0 con 0x557c1cc5de00 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf2655fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cc59b60 con 0x557c1cc58890 2024-05-08T23:24:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf2655fb80 10 client.0 _ll_register_callbacks cb 0x557c1cb4db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 0x557c1cc5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 0x557c1cc5afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45630/0 (socket says 172.21.15.16:45630) 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 -- 172.21.15.16:0/1118611147 learned_addr learned my addr 172.21.15.16:0/1118611147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf13fff700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc69460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf137fe700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 -- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 msgr2=0x557c1cc69460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc69460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 -- 172.21.15.16:0/1118611147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 msgr2=0x557c1cc5aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 -- 172.21.15.16:0/1118611147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c1c9d5b60 con 0x557c1cc5e720 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf137fe700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf18942700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 0x557c1cc5afd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c70 tx=0x7fcf0c004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155483a9e9ce64ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcef37fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcef37fe700 1 -- 172.21.15.16:0/1118611147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c018040 con 0x557c1cc5e720 2024-05-08T23:24:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcef37fe700 1 -- 172.21.15.16:0/1118611147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fcf0c006000 con 0x557c1cc5e720 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.449+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf0c00f050 con 0x557c1cc5e720 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcef37fe700 1 -- 172.21.15.16:0/1118611147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0c005210 con 0x557c1cc5e720 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 10 client.16569 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557c1ca2dcd0 con 0x557c1cc5e720 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcef37fe700 1 -- 172.21.15.16:0/1118611147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf0c00f050 con 0x557c1cc5e720 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcef37fe700 1 -- 172.21.15.16:0/1118611147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf0c028ce0 con 0x557c1cc5e720 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 10 client.16569 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 10 client.16569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 20 client.16569 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1ccc8f20 0x557c1cccb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 4 client.16569 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c1cba0ec0 con 0x557c1ccc8f20 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.450+0000 7fcf13fff700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1ccc8f20 0x557c1cccb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf13fff700 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1ccc8f20 0x557c1cccb340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=387b9f2d963108e3 server_cookie=5e820c07ddc0f9da in_seq=0 out_seq=0 2024-05-08T23:24:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcef37fe700 10 client.16569 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcef37fe700 1 -- 172.21.15.16:0/1118611147 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c1cba0ec0 con 0x557c1ccc8f20 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcef37fe700 10 client.16569 handle_command_reply: tid=0 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 client.16569 shutdown 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 msgr2=0x557c1cc5afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 0x557c1cc5afd0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009c70 tx=0x7fcf0c004d70 comp rx=0 tx=0).stop 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 shutdown_connections 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c1cc5e720 0x557c1cc5afd0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c1cc58890 0x557c1cc69460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557c1ccc8f20 0x557c1cccb340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 --2- 172.21.15.16:0/1118611147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557c1cc5de00 0x557c1cc5aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 >> 172.21.15.16:0/1118611147 conn(0x557c1cbb6210 msgr2=0x557c1cc5cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcef37fe700 0 client.16569 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 shutdown_connections 2024-05-08T23:24:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 1 -- 172.21.15.16:0/1118611147 wait complete. 2024-05-08T23:24:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.451+0000 7fcf2655fb80 20 client.16569 trim_cache size 0 max 16384 2024-05-08T23:24:08.458 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcefc0a1290 msgr2=0x7fcefc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcefc0a1290 0x7fcefc0a3750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf040098e0 tx=0x7fcf0400b260 comp rx=0 tx=0).stop 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 msgr2=0x7fcf141999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 0x7fcf141999e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800e9a0 tx=0x7fcf0800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 shutdown_connections 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcefc0a1290 0x7fcefc0a3750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf1410d8f0 0x7fcf1419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14069500 0x7fcf141994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 --2- 172.21.15.16:0/3568953921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf14069e20 0x7fcf141999e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 >> 172.21.15.16:0/3568953921 conn(0x7fcf14076fa0 msgr2=0x7fcf1410f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 shutdown_connections 2024-05-08T23:24:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.452+0000 7fcf1a3a5700 1 -- 172.21.15.16:0/3568953921 wait complete. 2024-05-08T23:24:08.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:08 smithi155 ceph-mon[47105]: pgmap v140: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-08T23:24:08.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:08 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1985290105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:08.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:24:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 -- 172.21.15.16:0/1864862587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c100060 msgr2=0x7f9d1c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 --2- 172.21.15.16:0/1864862587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c100060 0x7f9d1c100440 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009990 tx=0x7f9d04009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 -- 172.21.15.16:0/1864862587 shutdown_connections 2024-05-08T23:24:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 --2- 172.21.15.16:0/1864862587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105160 0x7f9d1c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 --2- 172.21.15.16:0/1864862587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c100060 0x7f9d1c100440 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 --2- 172.21.15.16:0/1864862587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1c100a10 0x7f9d1c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 -- 172.21.15.16:0/1864862587 >> 172.21.15.16:0/1864862587 conn(0x7f9d1c076f40 msgr2=0x7f9d1c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 -- 172.21.15.16:0/1864862587 shutdown_connections 2024-05-08T23:24:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.605+0000 7f9d20c97700 1 -- 172.21.15.16:0/1864862587 wait complete. 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 Processor -- start 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 -- start start 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1c100060 0x7f9d1c101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105160 0x7f9d1c102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 0x7f9d1c1a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a5a50 con 0x7f9d1c102550 2024-05-08T23:24:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a5bc0 con 0x7f9d1c100060 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.606+0000 7f9d20c97700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a5d80 con 0x7f9d1c105160 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 0x7f9d1c1a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 0x7f9d1c1a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45652/0 (socket says 172.21.15.16:45652) 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 -- 172.21.15.16:0/2837091491 learned_addr learned my addr 172.21.15.16:0/2837091491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1c100060 0x7f9d1c101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105160 0x7f9d1c102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 -- 172.21.15.16:0/2837091491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105160 msgr2=0x7f9d1c102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105160 0x7f9d1c102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 -- 172.21.15.16:0/2837091491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1c100060 msgr2=0x7f9d1c101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1c100060 0x7f9d1c101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 -- 172.21.15.16:0/2837091491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040095f0 con 0x7f9d1c102550 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 0x7f9d1c1a5480 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400ef30 tx=0x7f9d1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d137fe700 1 -- 172.21.15.16:0/2837091491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14018070 con 0x7f9d1c102550 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c1a6000 con 0x7f9d1c102550 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d137fe700 1 -- 172.21.15.16:0/2837091491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d1400f040 con 0x7f9d1c102550 2024-05-08T23:24:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d137fe700 1 -- 172.21.15.16:0/2837091491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400cc70 con 0x7f9d1c102550 2024-05-08T23:24:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.607+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c1a6510 con 0x7f9d1c102550 2024-05-08T23:24:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.610+0000 7f9d137fe700 1 -- 172.21.15.16:0/2837091491 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d14007500 con 0x7f9d1c102550 2024-05-08T23:24:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.610+0000 7f9d137fe700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9d080a1290 0x7f9d080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.611+0000 7f9d137fe700 1 -- 172.21.15.16:0/2837091491 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f9d140ccb40 con 0x7f9d1c102550 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.611+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9d080a1290 0x7f9d080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.611+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9d080a1290 0x7f9d080a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d04006010 tx=0x7f9d0400b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.611+0000 7f9d2ce51b80 1 Processor -- start 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 -- start start 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 0x55816c7a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a0890 0x55816c7a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c6f7150 con 0x55816c7a6720 2024-05-08T23:24:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c6f4b10 con 0x55816c7a0890 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c6f0400 con 0x55816c7a5e00 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 0x55816c7a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 0x55816c7a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45664/0 (socket says 172.21.15.16:45664) 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 -- 172.21.15.16:0/392682035 learned_addr learned my addr 172.21.15.16:0/392682035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a0890 0x55816c7a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 -- 172.21.15.16:0/392682035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 msgr2=0x55816c7a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 -- 172.21.15.16:0/392682035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a0890 msgr2=0x55816c7a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.612+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a0890 0x55816c7a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d19d9b700 1 -- 172.21.15.16:0/392682035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55816c5713b0 con 0x55816c7a6720 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a0890 0x55816c7a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 0x55816c7a0350 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c008c00 tx=0x7f9d0c008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77250b9565a49c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d10ff9700 1 -- 172.21.15.16:0/392682035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00ece0 con 0x55816c7a6720 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d10ff9700 1 -- 172.21.15.16:0/392682035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d0c00ee40 con 0x55816c7a6720 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.613+0000 7f9d10ff9700 1 -- 172.21.15.16:0/392682035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0056b0 con 0x55816c7a6720 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/392682035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 msgr2=0x55816c7a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 0x55816c7a0350 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c008c00 tx=0x7f9d0c008f10 comp rx=0 tx=0).stop 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/392682035 shutdown_connections 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a0890 0x55816c7a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/392682035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a6720 0x55816c7a0350 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/392682035 >> 172.21.15.16:0/392682035 conn(0x55816c6fe210 msgr2=0x55816c79adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/392682035 shutdown_connections 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/392682035 wait complete. 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 -- start start 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.614+0000 7f9d2ce51b80 1 Processor -- start 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c7a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c68c560 con 0x55816c7a5e00 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c68c200 con 0x55816c7a6720 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c5736a0 con 0x55816c7a0890 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d2ce51b80 10 client.0 _ll_register_callbacks cb 0x55816c695b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c7a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c7a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45676/0 (socket says 172.21.15.16:45676) 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 -- 172.21.15.16:0/4290461468 learned_addr learned my addr 172.21.15.16:0/4290461468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 -- 172.21.15.16:0/4290461468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 msgr2=0x55816c7ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 -- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 msgr2=0x55816c7aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 -- 172.21.15.16:0/4290461468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c10d550 con 0x55816c7a5e00 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c7a9aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c000c00 tx=0x7f9d0c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6076faa844aa17c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.615+0000 7f9d01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d01ffb700 1 -- 172.21.15.16:0/4290461468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c01f070 con 0x55816c7a5e00 2024-05-08T23:24:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55816c649270 con 0x55816c7a5e00 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d01ffb700 1 -- 172.21.15.16:0/4290461468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f9d0c004960 con 0x55816c7a5e00 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d01ffb700 1 -- 172.21.15.16:0/4290461468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d0c008230 con 0x55816c7a5e00 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 10 client.16581 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f9d0c008830 con 0x55816c7a5e00 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d01ffb700 1 -- 172.21.15.16:0/4290461468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55816c649270 con 0x55816c7a5e00 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d01ffb700 1 -- 172.21.15.16:0/4290461468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d0c010ce0 con 0x55816c7a5e00 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 20 client.16581 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c7ba130 0x55816c7bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 4 client.16581 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.616+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55816c76f810 con 0x55816c7ba130 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.617+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c7ba130 0x55816c7bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.617+0000 7f9d01ffb700 10 client.16581 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.617+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c7ba130 0x55816c7bc550 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3eaa8f99655f54 server_cookie=602a46f46e89096a in_seq=0 out_seq=0 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d01ffb700 1 -- 172.21.15.16:0/4290461468 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55816c76f810 con 0x55816c7ba130 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d01ffb700 10 client.16581 handle_command_reply: tid=0 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 client.16581 shutdown 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 msgr2=0x55816c7a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c7a9aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c000c00 tx=0x7f9d0c005540 comp rx=0 tx=0).stop 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 shutdown_connections 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c7ba130 0x55816c7bc550 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d01ffb700 0 client.16581 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/4290461468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c7a9aa0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 >> 172.21.15.16:0/4290461468 conn(0x55816c6fe210 msgr2=0x55816c79a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 shutdown_connections 2024-05-08T23:24:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/4290461468 wait complete. 2024-05-08T23:24:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.618+0000 7f9d2ce51b80 20 client.16581 trim_cache size 0 max 16384 2024-05-08T23:24:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 Processor -- start 2024-05-08T23:24:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 -- start start 2024-05-08T23:24:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c79ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c79ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c7ace90 con 0x55816c7a5e00 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c79c670 con 0x55816c7a6720 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c7aaf30 con 0x55816c7a0890 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c79ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c79ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45678/0 (socket says 172.21.15.16:45678) 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.632+0000 7f9d19d9b700 1 -- 172.21.15.16:0/1726211970 learned_addr learned my addr 172.21.15.16:0/1726211970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d19d9b700 1 -- 172.21.15.16:0/1726211970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 msgr2=0x55816c7b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d19d9b700 1 -- 172.21.15.16:0/1726211970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 msgr2=0x55816c79ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c79ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d19d9b700 1 -- 172.21.15.16:0/1726211970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55816c51db60 con 0x55816c7a5e00 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c79ca00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00ee10 tx=0x7f9d0c005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987377df27bcfd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d117fa700 1 -- 172.21.15.16:0/1726211970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c01f030 con 0x55816c7a5e00 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d117fa700 1 -- 172.21.15.16:0/1726211970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d0c00e6e0 con 0x55816c7a5e00 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.633+0000 7f9d117fa700 1 -- 172.21.15.16:0/1726211970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004bc0 con 0x55816c7a5e00 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1726211970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 msgr2=0x55816c79ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c79ca00 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00ee10 tx=0x7f9d0c005660 comp rx=0 tx=0).stop 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1726211970 shutdown_connections 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c79ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a0890 0x55816c7b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1726211970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a5e00 0x55816c79ca00 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1726211970 >> 172.21.15.16:0/1726211970 conn(0x55816c6fe210 msgr2=0x55816c7a3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1726211970 shutdown_connections 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1726211970 wait complete. 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- start start 2024-05-08T23:24:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 Processor -- start 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 0x55816c7b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c7affd0 con 0x55816c7a0890 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c80f5f0 con 0x55816c7a6720 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816c80f730 con 0x55816c7a5e00 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d2ce51b80 10 client.0 _ll_register_callbacks cb 0x55816c695b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.634+0000 7f9d1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 0x55816c7b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 0x55816c7b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45680/0 (socket says 172.21.15.16:45680) 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d19d9b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:40580/0 (socket says 172.21.15.16:40580) 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 -- 172.21.15.16:0/1077944602 learned_addr learned my addr 172.21.15.16:0/1077944602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 -- 172.21.15.16:0/1077944602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 msgr2=0x55816c7af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 -- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 msgr2=0x55816c7af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 -- 172.21.15.16:0/1077944602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c008830 con 0x55816c7a0890 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1ad9d700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d1a59c700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 0x55816c7b1670 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009990 tx=0x7f9d04009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2180c2f29334ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.635+0000 7f9d01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d01ffb700 1 -- 172.21.15.16:0/1077944602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400fbd0 con 0x55816c7a0890 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d0c00ee40 con 0x55816c7a0890 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d01ffb700 1 -- 172.21.15.16:0/1077944602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f9d0400fd30 con 0x55816c7a0890 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d01ffb700 1 -- 172.21.15.16:0/1077944602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d04035040 con 0x55816c7a0890 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 10 client.16587 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55816c6b6560 con 0x55816c7a0890 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d01ffb700 1 -- 172.21.15.16:0/1077944602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d0c00ee40 con 0x55816c7a0890 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d01ffb700 1 -- 172.21.15.16:0/1077944602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9d0403b020 con 0x55816c7a0890 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 20 client.16587 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c810d50 0x55816c813170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 4 client.16587 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.636+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55816c6e8ec0 con 0x55816c810d50 2024-05-08T23:24:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c810d50 0x55816c813170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d19d9b700 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c810d50 0x55816c813170 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebce7f08d4cc671 server_cookie=9e62f8746dc1dd2f in_seq=0 out_seq=0 2024-05-08T23:24:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d01ffb700 10 client.16587 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d01ffb700 1 -- 172.21.15.16:0/1077944602 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55816c6e8ec0 con 0x55816c810d50 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d01ffb700 10 client.16587 handle_command_reply: tid=0 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 client.16587 shutdown 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 msgr2=0x55816c7b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 0x55816c7b1670 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009990 tx=0x7f9d04009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 shutdown_connections 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55816c810d50 0x55816c813170 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55816c7a6720 0x55816c7af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55816c7a0890 0x55816c7b1670 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d01ffb700 0 client.16587 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 --2- 172.21.15.16:0/1077944602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55816c7a5e00 0x55816c7af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 >> 172.21.15.16:0/1077944602 conn(0x55816c6fe210 msgr2=0x55816c7a3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.637+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 shutdown_connections 2024-05-08T23:24:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d2ce51b80 1 -- 172.21.15.16:0/1077944602 wait complete. 2024-05-08T23:24:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d2ce51b80 20 client.16587 trim_cache size 0 max 16384 2024-05-08T23:24:08.645 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:24:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9d080a1290 msgr2=0x7f9d080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9d080a1290 0x7f9d080a3750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d04006010 tx=0x7f9d0400b260 comp rx=0 tx=0).stop 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 msgr2=0x7f9d1c1a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 0x7f9d1c1a5480 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400ef30 tx=0x7f9d1400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 shutdown_connections 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f9d080a1290 0x7f9d080a3750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d1c102550 0x7f9d1c1a5480 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d1c100060 0x7f9d1c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 --2- 172.21.15.16:0/2837091491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105160 0x7f9d1c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 >> 172.21.15.16:0/2837091491 conn(0x7f9d1c076f40 msgr2=0x7f9d1c068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 shutdown_connections 2024-05-08T23:24:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:08.638+0000 7f9d20c97700 1 -- 172.21.15.16:0/2837091491 wait complete. 2024-05-08T23:24:09.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:24:09.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:24:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 -- 172.21.15.16:0/2287035610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100a10 msgr2=0x7fecb0104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 --2- 172.21.15.16:0/2287035610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100a10 0x7fecb0104a30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7feca0009990 tx=0x7feca0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 -- 172.21.15.16:0/2287035610 shutdown_connections 2024-05-08T23:24:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 --2- 172.21.15.16:0/2287035610 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecb0105160 0x7fecb010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 --2- 172.21.15.16:0/2287035610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb0100060 0x7fecb0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 --2- 172.21.15.16:0/2287035610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100a10 0x7fecb0104a30 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 -- 172.21.15.16:0/2287035610 >> 172.21.15.16:0/2287035610 conn(0x7fecb0076f40 msgr2=0x7fecb0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 -- 172.21.15.16:0/2287035610 shutdown_connections 2024-05-08T23:24:09.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.804+0000 7fecb5061700 1 -- 172.21.15.16:0/2287035610 wait complete. 2024-05-08T23:24:09.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.805+0000 7fecb5061700 1 Processor -- start 2024-05-08T23:24:09.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.805+0000 7fecb5061700 1 -- start start 2024-05-08T23:24:09.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecb5061700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 0x7fecb01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecb5061700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb0105160 0x7fecb0196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecb5061700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecb019aaf0 0x7fecb019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecb5061700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019b510 con 0x7fecb0100060 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecb5061700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019b680 con 0x7fecb019aaf0 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecb5061700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019b840 con 0x7fecb0105160 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 0x7fecb01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 0x7fecb01962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45694/0 (socket says 172.21.15.16:45694) 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 -- 172.21.15.16:0/3648034398 learned_addr learned my addr 172.21.15.16:0/3648034398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaf59e700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecb019aaf0 0x7fecb019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecae59c700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb0105160 0x7fecb0196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 -- 172.21.15.16:0/3648034398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb0105160 msgr2=0x7fecb0196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb0105160 0x7fecb0196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 -- 172.21.15.16:0/3648034398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecb019aaf0 msgr2=0x7fecb019afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecb019aaf0 0x7fecb019afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.806+0000 7fecaed9d700 1 -- 172.21.15.16:0/3648034398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca00095f0 con 0x7fecb0100060 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.807+0000 7fecaed9d700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 0x7fecb01962d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7feca400e9a0 tx=0x7feca400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.807+0000 7fec97fff700 1 -- 172.21.15.16:0/3648034398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4009790 con 0x7fecb0100060 2024-05-08T23:24:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.807+0000 7fec97fff700 1 -- 172.21.15.16:0/3648034398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feca400cbd0 con 0x7fecb0100060 2024-05-08T23:24:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.807+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb01a2e20 con 0x7fecb0100060 2024-05-08T23:24:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.807+0000 7fec97fff700 1 -- 172.21.15.16:0/3648034398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4018680 con 0x7fecb0100060 2024-05-08T23:24:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.807+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb01a3240 con 0x7fecb0100060 2024-05-08T23:24:09.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 Processor -- start 2024-05-08T23:24:09.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 -- start start 2024-05-08T23:24:09.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 0x5592e13d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2990 0x5592e13cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e13211d0 con 0x5592e13d2990 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e131eb90 con 0x5592e13d2070 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecc121bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e131a480 con 0x5592e13ccb20 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecaf59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 0x5592e13d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecaf59e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 0x5592e13d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:55136/0 (socket says 172.21.15.16:55136) 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecaf59e700 1 -- 172.21.15.16:0/2344468320 learned_addr learned my addr 172.21.15.16:0/2344468320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.811+0000 7fecae59c700 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fecaf59e700 1 -- 172.21.15.16:0/2344468320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 msgr2=0x5592e13ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fecaf59e700 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fecaf59e700 1 -- 172.21.15.16:0/2344468320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2990 msgr2=0x5592e13cc5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fecaf59e700 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2990 0x5592e13cc5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fecaf59e700 1 -- 172.21.15.16:0/2344468320 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e119b430 con 0x5592e13d2070 2024-05-08T23:24:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fecaf59e700 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 0x5592e13d2450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feca8011680 tx=0x7feca8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5924adbd89aff08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fec957fa700 1 -- 172.21.15.16:0/2344468320 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800e040 con 0x5592e13d2070 2024-05-08T23:24:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fec957fa700 1 -- 172.21.15.16:0/2344468320 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feca800f040 con 0x5592e13d2070 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fec957fa700 1 -- 172.21.15.16:0/2344468320 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca80046f0 con 0x5592e13d2070 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.812+0000 7fec97fff700 1 -- 172.21.15.16:0/3648034398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7feca400cd40 con 0x7fecb0100060 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 -- 172.21.15.16:0/2344468320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 msgr2=0x5592e13d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 0x5592e13d2450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feca8011680 tx=0x7feca8009920 comp rx=0 tx=0).stop 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fec97fff700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fec980a1290 0x7fec980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 -- 172.21.15.16:0/2344468320 shutdown_connections 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2070 0x5592e13d2450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2344468320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2990 0x5592e13cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 -- 172.21.15.16:0/2344468320 >> 172.21.15.16:0/2344468320 conn(0x5592e1328290 msgr2=0x5592e13c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fec97fff700 1 -- 172.21.15.16:0/3648034398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7feca4014070 con 0x7fecb0100060 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecaf59e700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fec980a1290 0x7fec980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 -- 172.21.15.16:0/2344468320 shutdown_connections 2024-05-08T23:24:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 -- 172.21.15.16:0/2344468320 wait complete. 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.813+0000 7fecc121bb80 1 -- start start 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaf59e700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fec980a1290 0x7fec980a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feca8006080 tx=0x7feca80042b0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 Processor -- start 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 0x5592e13d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2070 0x5592e13d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e13d6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e12b65e0 con 0x5592e13ccb20 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e12b6280 con 0x5592e13d2990 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e119d720 con 0x5592e13d2070 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecc121bb80 10 client.0 _ll_register_callbacks cb 0x5592e12bfbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 0x5592e13d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 0x5592e13d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45702/0 (socket says 172.21.15.16:45702) 2024-05-08T23:24:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 -- 172.21.15.16:0/2095017264 learned_addr learned my addr 172.21.15.16:0/2095017264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaf59e700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e13d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecae59c700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2070 0x5592e13d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 -- 172.21.15.16:0/2095017264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2070 msgr2=0x5592e13d61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2070 0x5592e13d61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 -- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 msgr2=0x5592e13d6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.814+0000 7fecaed9d700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e13d6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecaed9d700 1 -- 172.21.15.16:0/2095017264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca8011330 con 0x5592e13ccb20 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecaed9d700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 0x5592e13d5c90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7feca40ca6e0 tx=0x7feca40999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f60b4df6e98c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fec92ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fec92ff5700 1 -- 172.21.15.16:0/2095017264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca409a6f0 con 0x5592e13ccb20 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fec92ff5700 1 -- 172.21.15.16:0/2095017264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7feca4099ff0 con 0x5592e13ccb20 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592e12732f0 con 0x5592e13ccb20 2024-05-08T23:24:09.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fec92ff5700 1 -- 172.21.15.16:0/2095017264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feca409de20 con 0x5592e13ccb20 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 10 client.16596 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592e1147be0 con 0x5592e13ccb20 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fec92ff5700 1 -- 172.21.15.16:0/2095017264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5592e12732f0 con 0x5592e13ccb20 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fec92ff5700 1 -- 172.21.15.16:0/2095017264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feca4095de0 con 0x5592e13ccb20 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 20 client.16596 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e13e63d0 0x5592e13e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.815+0000 7fecc121bb80 4 client.16596 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.816+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592e1399890 con 0x5592e13e63d0 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.816+0000 7fecae59c700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e13e63d0 0x5592e13e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.816+0000 7fec92ff5700 10 client.16596 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.816+0000 7fecae59c700 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e13e63d0 0x5592e13e87c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2bf0da84f399eb0 server_cookie=564abf70b19388c7 in_seq=0 out_seq=0 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fec92ff5700 1 -- 172.21.15.16:0/2095017264 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592e1399890 con 0x5592e13e63d0 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fec92ff5700 10 client.16596 handle_command_reply: tid=0 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 client.16596 shutdown 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 msgr2=0x5592e13d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 0x5592e13d5c90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7feca40ca6e0 tx=0x7feca40999d0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 shutdown_connections 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e13e63d0 0x5592e13e87c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e13d6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13ccb20 0x5592e13d5c90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fec92ff5700 0 client.16596 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2095017264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2070 0x5592e13d61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 >> 172.21.15.16:0/2095017264 conn(0x5592e1328290 msgr2=0x5592e13c4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 shutdown_connections 2024-05-08T23:24:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 1 -- 172.21.15.16:0/2095017264 wait complete. 2024-05-08T23:24:09.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.817+0000 7fecc121bb80 20 client.16596 trim_cache size 0 max 16384 2024-05-08T23:24:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 Processor -- start 2024-05-08T23:24:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 -- start start 2024-05-08T23:24:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13ccb20 0x5592e13d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2990 0x5592e13d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e13c66c0 con 0x5592e13d2070 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e13ed960 con 0x5592e13ccb20 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecc121bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e13bf880 con 0x5592e13d2990 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45716/0 (socket says 172.21.15.16:45716) 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 -- 172.21.15.16:0/3081203438 learned_addr learned my addr 172.21.15.16:0/3081203438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 -- 172.21.15.16:0/3081203438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2990 msgr2=0x5592e13d6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2990 0x5592e13d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 -- 172.21.15.16:0/3081203438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13ccb20 msgr2=0x5592e13d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13ccb20 0x5592e13d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.837+0000 7fecae59c700 1 -- 172.21.15.16:0/3081203438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e12e05e0 con 0x5592e13d2070 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecae59c700 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13c4da0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7feca0009990 tx=0x7feca0005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c91a8794b534bc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fec95ffb700 1 -- 172.21.15.16:0/3081203438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0029160 con 0x5592e13d2070 2024-05-08T23:24:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fec95ffb700 1 -- 172.21.15.16:0/3081203438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feca001e040 con 0x5592e13d2070 2024-05-08T23:24:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fec95ffb700 1 -- 172.21.15.16:0/3081203438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca000bae0 con 0x5592e13d2070 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 -- 172.21.15.16:0/3081203438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 msgr2=0x5592e13c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13c4da0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7feca0009990 tx=0x7feca0005ab0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 -- 172.21.15.16:0/3081203438 shutdown_connections 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13d2990 0x5592e13d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13ccb20 0x5592e13d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 --2- 172.21.15.16:0/3081203438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13c4da0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 -- 172.21.15.16:0/3081203438 >> 172.21.15.16:0/3081203438 conn(0x5592e1328290 msgr2=0x5592e13d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 -- 172.21.15.16:0/3081203438 shutdown_connections 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.838+0000 7fecc121bb80 1 -- 172.21.15.16:0/3081203438 wait complete. 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 -- start start 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 Processor -- start 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13da250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13da790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e143c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e13daf00 con 0x5592e13d2070 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e13db070 con 0x5592e13d2990 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e143cd70 con 0x5592e13ccb20 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecc121bb80 10 client.0 _ll_register_callbacks cb 0x5592e12bfbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13da790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13da790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2024-05-08T23:24:09.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecae59c700 1 -- 172.21.15.16:0/2543381406 learned_addr learned my addr 172.21.15.16:0/2543381406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.839+0000 7fecaed9d700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13da250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecaf59e700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e143c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecae59c700 1 -- 172.21.15.16:0/2543381406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 msgr2=0x5592e13da250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecae59c700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13da250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecae59c700 1 -- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 msgr2=0x5592e143c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecae59c700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e143c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecae59c700 1 -- 172.21.15.16:0/2543381406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb010d550 con 0x5592e13d2070 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecae59c700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13da790 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7feca002d040 tx=0x7feca00056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7c52ed5a62f3ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fec92ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fec92ff5700 1 -- 172.21.15.16:0/2543381406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca0029730 con 0x5592e13d2070 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fec92ff5700 1 -- 172.21.15.16:0/2543381406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7feca000b760 con 0x5592e13d2070 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feca800f040 con 0x5592e13d2070 2024-05-08T23:24:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fec92ff5700 1 -- 172.21.15.16:0/2543381406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feca0029890 con 0x5592e13d2070 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecc121bb80 10 client.16602 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5592e119fd50 con 0x5592e13d2070 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.840+0000 7fec92ff5700 1 -- 172.21.15.16:0/2543381406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feca800f040 con 0x5592e13d2070 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fec92ff5700 1 -- 172.21.15.16:0/2543381406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feca0004a60 con 0x5592e13d2070 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecc121bb80 10 client.16602 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecc121bb80 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecc121bb80 20 client.16602 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e143e070 0x5592e1440460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecc121bb80 4 client.16602 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5592e1312f40 con 0x5592e143e070 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecaed9d700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e143e070 0x5592e1440460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fecaed9d700 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e143e070 0x5592e1440460 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e739d5c4ef1f5943 server_cookie=da666f6f8ca5b53e in_seq=0 out_seq=0 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.841+0000 7fec92ff5700 10 client.16602 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fec92ff5700 1 -- 172.21.15.16:0/2543381406 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5592e1312f40 con 0x5592e143e070 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fec92ff5700 10 client.16602 handle_command_reply: tid=0 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 client.16602 shutdown 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 msgr2=0x5592e13da790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13da790 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7feca002d040 tx=0x7feca00056b0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 shutdown_connections 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5592e143e070 0x5592e1440460 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5592e13d2990 0x5592e143c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fec92ff5700 0 client.16602 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e13ccb20 0x5592e13da250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 --2- 172.21.15.16:0/2543381406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592e13d2070 0x5592e13da790 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.842+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 >> 172.21.15.16:0/2543381406 conn(0x5592e1328290 msgr2=0x5592e13c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 shutdown_connections 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecc121bb80 1 -- 172.21.15.16:0/2543381406 wait complete. 2024-05-08T23:24:09.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecc121bb80 20 client.16602 trim_cache size 0 max 16384 2024-05-08T23:24:09.849 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:24:09.849 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:24:09.849 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "28d57522-d965-42d2-bcef-126299d329ae", 2024-05-08T23:24:09.849 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:24:09.849 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fec980a1290 msgr2=0x7fec980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fec980a1290 0x7fec980a3750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feca8006080 tx=0x7feca80042b0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 msgr2=0x7fecb01962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 0x7fecb01962d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7feca400e9a0 tx=0x7feca400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 shutdown_connections 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fec980a1290 0x7fec980a3750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fecb019aaf0 0x7fecb019afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecb0100060 0x7fecb01962d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 --2- 172.21.15.16:0/3648034398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb0105160 0x7fecb0196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.843+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 >> 172.21.15.16:0/3648034398 conn(0x7fecb0076f40 msgr2=0x7fecb00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.844+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 shutdown_connections 2024-05-08T23:24:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.844+0000 7fecb5061700 1 -- 172.21.15.16:0/3648034398 wait complete. 2024-05-08T23:24:09.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.996+0000 7f5745dd6700 1 -- 172.21.15.16:0/3338880084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 msgr2=0x7f5740101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.996+0000 7f5745dd6700 1 --2- 172.21.15.16:0/3338880084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f5740101540 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5730009990 tx=0x7f5730009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 -- 172.21.15.16:0/3338880084 shutdown_connections 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 --2- 172.21.15.16:0/3338880084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5740106280 0x7f574010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 --2- 172.21.15.16:0/3338880084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f5740101540 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 --2- 172.21.15.16:0/3338880084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740101b10 0x7f5740105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 -- 172.21.15.16:0/3338880084 >> 172.21.15.16:0/3338880084 conn(0x7f57400fc9e0 msgr2=0x7f57400fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 -- 172.21.15.16:0/3338880084 shutdown_connections 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 -- 172.21.15.16:0/3338880084 wait complete. 2024-05-08T23:24:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.997+0000 7f5745dd6700 1 Processor -- start 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 -- start start 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f57400732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5740106280 0x7f57400737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740073d20 0x7f57401a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401a5b10 con 0x7f5740101160 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401a5c50 con 0x7f5740106280 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f5745dd6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401a5e10 con 0x7f5740073d20 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f57400732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f57400732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45746/0 (socket says 172.21.15.16:45746) 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 -- 172.21.15.16:0/1232947118 learned_addr learned my addr 172.21.15.16:0/1232947118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573ffff700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740073d20 0x7f57401a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573effd700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5740106280 0x7f57400737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 -- 172.21.15.16:0/1232947118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740073d20 msgr2=0x7f57401a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740073d20 0x7f57401a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 -- 172.21.15.16:0/1232947118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5740106280 msgr2=0x7f57400737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5740106280 0x7f57400737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573f7fe700 1 -- 172.21.15.16:0/1232947118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57300095f0 con 0x7f5740101160 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.998+0000 7f573ffff700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740073d20 0x7f57401a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.999+0000 7f573f7fe700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f57400732a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f57300059d0 tx=0x7f573000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.999+0000 7f573cff9700 1 -- 172.21.15.16:0/1232947118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573001d070 con 0x7f5740101160 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.999+0000 7f573cff9700 1 -- 172.21.15.16:0/1232947118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5730022470 con 0x7f5740101160 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.999+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57401a6090 con 0x7f5740101160 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.999+0000 7f573cff9700 1 -- 172.21.15.16:0/1232947118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000fea0 con 0x7f5740101160 2024-05-08T23:24:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:09.999+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57401a6510 con 0x7f5740101160 2024-05-08T23:24:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.001+0000 7f573cff9700 1 -- 172.21.15.16:0/1232947118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f57300225e0 con 0x7f5740101160 2024-05-08T23:24:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.002+0000 7f573cff9700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57280a1310 0x7f57280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.002+0000 7f573effd700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57280a1310 0x7f57280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.002+0000 7f573cff9700 1 -- 172.21.15.16:0/1232947118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f57300ce320 con 0x7f5740101160 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.002+0000 7f5751f90b80 1 Processor -- start 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.002+0000 7f573effd700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57280a1310 0x7f57280a37d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5734005ea0 tx=0x7f5734005e30 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.002+0000 7f5751f90b80 1 -- start start 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdf1720 0x5646fbdeb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdeb890 0x5646fbdebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f5751f90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbd42150 con 0x5646fbdf0e00 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f5751f90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbd3fb10 con 0x5646fbdf1720 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f5751f90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbd3b400 con 0x5646fbdeb890 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45762/0 (socket says 172.21.15.16:45762) 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 -- 172.21.15.16:0/1077726130 learned_addr learned my addr 172.21.15.16:0/1077726130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573effd700 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdf1720 0x5646fbdeb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573ffff700 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdeb890 0x5646fbdebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 -- 172.21.15.16:0/1077726130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdeb890 msgr2=0x5646fbdebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdeb890 0x5646fbdebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 -- 172.21.15.16:0/1077726130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdf1720 msgr2=0x5646fbdeb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdf1720 0x5646fbdeb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 -- 172.21.15.16:0/1077726130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646fbbbc3b0 con 0x5646fbdf0e00 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.003+0000 7f573f7fe700 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf11e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5730097620 tx=0x7f573009ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1f88e56258a76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5725ffb700 1 -- 172.21.15.16:0/1077726130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573009d020 con 0x5646fbdf0e00 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5725ffb700 1 -- 172.21.15.16:0/1077726130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f573009b120 con 0x5646fbdf0e00 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5725ffb700 1 -- 172.21.15.16:0/1077726130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57300a11f0 con 0x5646fbdf0e00 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 -- 172.21.15.16:0/1077726130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 msgr2=0x5646fbdf11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf11e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5730097620 tx=0x7f573009ace0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 -- 172.21.15.16:0/1077726130 shutdown_connections 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdeb890 0x5646fbdebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf11e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 --2- 172.21.15.16:0/1077726130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdf1720 0x5646fbdeb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 -- 172.21.15.16:0/1077726130 >> 172.21.15.16:0/1077726130 conn(0x5646fbd49210 msgr2=0x5646fbde5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 -- 172.21.15.16:0/1077726130 shutdown_connections 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.004+0000 7f5751f90b80 1 -- 172.21.15.16:0/1077726130 wait complete. 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 -- start start 2024-05-08T23:24:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 Processor -- start 2024-05-08T23:24:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdf7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf0e00 0x5646fbdf4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 0x5646fbdf5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbcd7560 con 0x5646fbdf1720 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbcd7200 con 0x5646fbdeb890 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbbbe6a0 con 0x5646fbdf0e00 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.005+0000 7f5751f90b80 10 client.0 _ll_register_callbacks cb 0x5646fbce0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 0x5646fbdf5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 0x5646fbdf5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45778/0 (socket says 172.21.15.16:45778) 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 -- 172.21.15.16:0/2939275128 learned_addr learned my addr 172.21.15.16:0/2939275128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 -- 172.21.15.16:0/2939275128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf0e00 msgr2=0x5646fbdf4a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf0e00 0x5646fbdf4a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 -- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 msgr2=0x5646fbdf7710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdf7710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 -- 172.21.15.16:0/2939275128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573009fe70 con 0x5646fbdf1720 2024-05-08T23:24:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f573ffff700 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 0x5646fbdf5160 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f573800abc0 tx=0x7f573800aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762607e29d08f068 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f571effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f571effd700 1 -- 172.21.15.16:0/2939275128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738007bb0 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f571effd700 1 -- 172.21.15.16:0/2939275128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5738006e40 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646fbc94270 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f571effd700 1 -- 172.21.15.16:0/2939275128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5738007d10 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.006+0000 7f5751f90b80 10 client.16614 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5646fbb68b60 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f571effd700 1 -- 172.21.15.16:0/2939275128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5646fbc94270 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f571effd700 1 -- 172.21.15.16:0/2939275128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57380242e0 con 0x5646fbdf1720 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 20 client.16614 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe04fa0 0x5646fbe073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 4 client.16614 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.007+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646fbdba810 con 0x5646fbe04fa0 2024-05-08T23:24:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.008+0000 7f573f7fe700 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe04fa0 0x5646fbe073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.011+0000 7f573f7fe700 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe04fa0 0x5646fbe073c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf538cb1af51ac server_cookie=b50478b31d11439c in_seq=0 out_seq=0 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.011+0000 7f571effd700 10 client.16614 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f571effd700 1 -- 172.21.15.16:0/2939275128 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646fbdba810 con 0x5646fbe04fa0 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f571effd700 10 client.16614 handle_command_reply: tid=0 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 client.16614 shutdown 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 msgr2=0x5646fbdf5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 0x5646fbdf5160 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f573800abc0 tx=0x7f573800aca0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 shutdown_connections 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf1720 0x5646fbdf5160 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe04fa0 0x5646fbe073c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdf7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 --2- 172.21.15.16:0/2939275128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf0e00 0x5646fbdf4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f571effd700 0 client.16614 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 >> 172.21.15.16:0/2939275128 conn(0x5646fbd49210 msgr2=0x5646fbde55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 shutdown_connections 2024-05-08T23:24:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 1 -- 172.21.15.16:0/2939275128 wait complete. 2024-05-08T23:24:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.012+0000 7f5751f90b80 20 client.16614 trim_cache size 0 max 16384 2024-05-08T23:24:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 Processor -- start 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 -- start start 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdeb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbde5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbdf5520 con 0x5646fbdf0e00 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbde6ae0 con 0x5646fbdeb890 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f5751f90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbde0800 con 0x5646fbdf1720 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbde5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbde5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45784/0 (socket says 172.21.15.16:45784) 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 -- 172.21.15.16:0/144836938 learned_addr learned my addr 172.21.15.16:0/144836938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573f7fe700 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdeb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573ffff700 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 -- 172.21.15.16:0/144836938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 msgr2=0x5646fbdf3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 -- 172.21.15.16:0/144836938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 msgr2=0x5646fbdeb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdeb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.031+0000 7f573effd700 1 -- 172.21.15.16:0/144836938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646fbdebf40 con 0x5646fbdf0e00 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.032+0000 7f573effd700 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbde5d20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f573400ec40 tx=0x7f573400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd62933d2fdade53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.032+0000 7f57267fc700 1 -- 172.21.15.16:0/144836938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734010c50 con 0x5646fbdf0e00 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.032+0000 7f57267fc700 1 -- 172.21.15.16:0/144836938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5734010db0 con 0x5646fbdf0e00 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.032+0000 7f57267fc700 1 -- 172.21.15.16:0/144836938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573401e670 con 0x5646fbdf0e00 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 -- 172.21.15.16:0/144836938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 msgr2=0x5646fbde5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbde5d20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f573400ec40 tx=0x7f573400af90 comp rx=0 tx=0).stop 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 -- 172.21.15.16:0/144836938 shutdown_connections 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdeb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 --2- 172.21.15.16:0/144836938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbde5d20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 -- 172.21.15.16:0/144836938 >> 172.21.15.16:0/144836938 conn(0x5646fbd49210 msgr2=0x5646fbe00990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 -- 172.21.15.16:0/144836938 shutdown_connections 2024-05-08T23:24:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 -- 172.21.15.16:0/144836938 wait complete. 2024-05-08T23:24:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 -- start start 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.033+0000 7f5751f90b80 1 Processor -- start 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdfc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbdf92f0 con 0x5646fbdf0e00 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbdf9460 con 0x5646fbdeb890 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fbe5b480 con 0x5646fbdf1720 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f5751f90b80 10 client.0 _ll_register_callbacks cb 0x7f573800a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45798/0 (socket says 172.21.15.16:45798) 2024-05-08T23:24:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 -- 172.21.15.16:0/293978085 learned_addr learned my addr 172.21.15.16:0/293978085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573ffff700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573f7fe700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdfc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 -- 172.21.15.16:0/293978085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 msgr2=0x5646fbdf8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 -- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 msgr2=0x5646fbdfc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdfc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 -- 172.21.15.16:0/293978085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573400e920 con 0x5646fbdf0e00 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573f7fe700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdfc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f573effd700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf86e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5734010710 tx=0x7f5734027720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5773c40d84f230f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.034+0000 7f571effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f571effd700 1 -- 172.21.15.16:0/293978085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400a3f0 con 0x5646fbdf0e00 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f571effd700 1 -- 172.21.15.16:0/293978085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5734005250 con 0x5646fbdf0e00 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f573009b120 con 0x5646fbdf0e00 2024-05-08T23:24:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f571effd700 1 -- 172.21.15.16:0/293978085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5734005410 con 0x5646fbdf0e00 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 10 client.16620 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5646fbd01560 con 0x5646fbdf0e00 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f571effd700 1 -- 172.21.15.16:0/293978085 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f573009b120 con 0x5646fbdf0e00 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f571effd700 1 -- 172.21.15.16:0/293978085 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f573401e550 con 0x5646fbdf0e00 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 10 client.16620 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 20 client.16620 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe5ca00 0x5646fbe5ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 4 client.16620 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.035+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5646fbd33ec0 con 0x5646fbe5ca00 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f573f7fe700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe5ca00 0x5646fbe5ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f571effd700 10 client.16620 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f573f7fe700 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe5ca00 0x5646fbe5ee20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5bf839409eb322 server_cookie=a57d62797fa55a90 in_seq=0 out_seq=0 2024-05-08T23:24:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f571effd700 1 -- 172.21.15.16:0/293978085 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5646fbd33ec0 con 0x5646fbe5ca00 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f571effd700 10 client.16620 handle_command_reply: tid=0 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 client.16620 shutdown 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 msgr2=0x5646fbdf86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf86e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5734010710 tx=0x7f5734027720 comp rx=0 tx=0).stop 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 shutdown_connections 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646fbdf1720 0x5646fbdf8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5646fbe5ca00 0x5646fbe5ee20 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5646fbdeb890 0x5646fbdfc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 --2- 172.21.15.16:0/293978085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646fbdf0e00 0x5646fbdf86e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f571effd700 0 client.16620 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.036+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 >> 172.21.15.16:0/293978085 conn(0x5646fbd49210 msgr2=0x5646fbe00990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 shutdown_connections 2024-05-08T23:24:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5751f90b80 1 -- 172.21.15.16:0/293978085 wait complete. 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5751f90b80 20 client.16620 trim_cache size 0 max 16384 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: "28d57522-d965-42d2-bcef-126299d329ae": { 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "28d57522-d965-42d2-bcef-126299d329ae", 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:24:10.044 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:24:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57280a1310 msgr2=0x7f57280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57280a1310 0x7f57280a37d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5734005ea0 tx=0x7f5734005e30 comp rx=0 tx=0).stop 2024-05-08T23:24:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 msgr2=0x7f57400732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f57400732a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f57300059d0 tx=0x7f573000f460 comp rx=0 tx=0).stop 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 shutdown_connections 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57280a1310 0x7f57280a37d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5740073d20 0x7f57401a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5740101160 0x7f57400732a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 --2- 172.21.15.16:0/1232947118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5740106280 0x7f57400737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 >> 172.21.15.16:0/1232947118 conn(0x7f57400fc9e0 msgr2=0x7f57400fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 shutdown_connections 2024-05-08T23:24:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:10.037+0000 7f5745dd6700 1 -- 172.21.15.16:0/1232947118 wait complete. 2024-05-08T23:24:10.051 INFO:tasks.cephfs.filesystem:scrub status for tag:28d57522-d965-42d2-bcef-126299d329ae - {'path': '/', 'tag': '28d57522-d965-42d2-bcef-126299d329ae', 'options': 'recursive,force'} 2024-05-08T23:24:10.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:10 smithi016 ceph-mon[42541]: pgmap v141: 129 pgs: 129 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-05-08T23:24:10.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:10 smithi189 ceph-mon[46919]: pgmap v141: 129 pgs: 129 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-05-08T23:24:10.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:10 smithi155 ceph-mon[47105]: pgmap v141: 129 pgs: 129 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-05-08T23:24:11.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:11 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:24:11.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:11 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:11.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:11 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:24:11.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:11 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:11.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:11 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:24:11.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:11 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:24:11.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:11 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:11.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:11 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:24:11.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:11 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:11.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:11 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:24:11.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:11 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:24:11.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:11 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:11.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:11 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:24:11.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:11 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:11.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:11 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:24:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:12 smithi016 ceph-mon[42541]: pgmap v142: 129 pgs: 129 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-05-08T23:24:12.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:12 smithi189 ceph-mon[46919]: pgmap v142: 129 pgs: 129 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-05-08T23:24:12.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:12 smithi155 ceph-mon[47105]: pgmap v142: 129 pgs: 129 active+clean; 4.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-05-08T23:24:14.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:14 smithi016 ceph-mon[42541]: pgmap v143: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-05-08T23:24:14.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:14 smithi189 ceph-mon[46919]: pgmap v143: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-05-08T23:24:14.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:14 smithi155 ceph-mon[47105]: pgmap v143: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-05-08T23:24:16.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:16 smithi016 ceph-mon[42541]: pgmap v144: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:16.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:16 smithi189 ceph-mon[46919]: pgmap v144: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:16.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:16 smithi155 ceph-mon[47105]: pgmap v144: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:18 smithi016 ceph-mon[42541]: pgmap v145: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:18.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:18 smithi189 ceph-mon[46919]: pgmap v145: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:18.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:18 smithi155 ceph-mon[47105]: pgmap v145: 129 pgs: 129 active+clean; 4.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:20.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:20 smithi016 ceph-mon[42541]: pgmap v146: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-08T23:24:20.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:20 smithi189 ceph-mon[46919]: pgmap v146: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-08T23:24:20.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:20 smithi155 ceph-mon[47105]: pgmap v146: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-05-08T23:24:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:22 smithi016 ceph-mon[42541]: pgmap v147: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:22.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:22 smithi189 ceph-mon[46919]: pgmap v147: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:22.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:22 smithi155 ceph-mon[47105]: pgmap v147: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-08T23:24:24.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:24 smithi016 ceph-mon[42541]: pgmap v148: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-05-08T23:24:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:24 smithi189 ceph-mon[46919]: pgmap v148: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-05-08T23:24:24.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:24 smithi155 ceph-mon[47105]: pgmap v148: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-05-08T23:24:26.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:26 smithi016 ceph-mon[42541]: pgmap v149: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:26.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:26 smithi189 ceph-mon[46919]: pgmap v149: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:26.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:26 smithi155 ceph-mon[47105]: pgmap v149: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:28.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:28 smithi016 ceph-mon[42541]: pgmap v150: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:28.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:28 smithi189 ceph-mon[46919]: pgmap v150: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:28.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:28 smithi155 ceph-mon[47105]: pgmap v150: 129 pgs: 129 active+clean; 4.7 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:30.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:30 smithi016 ceph-mon[42541]: pgmap v151: 129 pgs: 129 active+clean; 4.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:30.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:30 smithi189 ceph-mon[46919]: pgmap v151: 129 pgs: 129 active+clean; 4.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:30.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:30 smithi155 ceph-mon[47105]: pgmap v151: 129 pgs: 129 active+clean; 4.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:32 smithi016 ceph-mon[42541]: pgmap v152: 129 pgs: 129 active+clean; 4.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:32.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:32 smithi189 ceph-mon[46919]: pgmap v152: 129 pgs: 129 active+clean; 4.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:32 smithi155 ceph-mon[47105]: pgmap v152: 129 pgs: 129 active+clean; 4.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:34 smithi016 ceph-mon[42541]: pgmap v153: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-05-08T23:24:34.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:34 smithi189 ceph-mon[46919]: pgmap v153: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-05-08T23:24:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:34 smithi155 ceph-mon[47105]: pgmap v153: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-05-08T23:24:36.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:36 smithi016 ceph-mon[42541]: pgmap v154: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:36.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:36 smithi189 ceph-mon[46919]: pgmap v154: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:36.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:36 smithi155 ceph-mon[47105]: pgmap v154: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:38 smithi016 ceph-mon[42541]: pgmap v155: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:38.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:38 smithi189 ceph-mon[46919]: pgmap v155: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:38.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:38 smithi155 ceph-mon[47105]: pgmap v155: 129 pgs: 129 active+clean; 4.9 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:40.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:24:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.203+0000 7ff4d53a4700 1 -- 172.21.15.16:0/1228624734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d0106290 msgr2=0x7ff4d010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.203+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/1228624734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d0106290 0x7ff4d010de10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b0a0 tx=0x7ff4c800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.203+0000 7ff4d53a4700 1 -- 172.21.15.16:0/1228624734 shutdown_connections 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.203+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/1228624734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d0106290 0x7ff4d010de10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.204+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/1228624734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4d0101170 0x7ff4d0101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.204+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/1228624734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d0101b20 0x7ff4d0105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.204+0000 7ff4d53a4700 1 -- 172.21.15.16:0/1228624734 >> 172.21.15.16:0/1228624734 conn(0x7ff4d00fc9d0 msgr2=0x7ff4d00fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.204+0000 7ff4d53a4700 1 -- 172.21.15.16:0/1228624734 shutdown_connections 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.204+0000 7ff4d53a4700 1 -- 172.21.15.16:0/1228624734 wait complete. 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.204+0000 7ff4d53a4700 1 Processor -- start 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 -- start start 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4d0101170 0x7ff4d0197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d0106290 0x7ff4d0197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 0x7ff4d019c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019c7a0 con 0x7ff4d019bd20 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019c910 con 0x7ff4d0101170 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4d53a4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019cad0 con 0x7ff4d0106290 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 0x7ff4d019c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4d0101170 0x7ff4d0197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 0x7ff4d019c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60694/0 (socket says 172.21.15.16:60694) 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/3654200572 learned_addr learned my addr 172.21.15.16:0/3654200572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.205+0000 7ff4c67fc700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d0106290 0x7ff4d0197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/3654200572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d0106290 msgr2=0x7ff4d0197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d0106290 0x7ff4d0197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/3654200572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4d0101170 msgr2=0x7ff4d0197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4d0101170 0x7ff4d0197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/3654200572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8009d00 con 0x7ff4d019bd20 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 0x7ff4d019c1a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80094c0 tx=0x7ff4c80094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4ccff9700 1 -- 172.21.15.16:0/3654200572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800e040 con 0x7ff4d019bd20 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4ccff9700 1 -- 172.21.15.16:0/3654200572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c8003c00 con 0x7ff4d019bd20 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d01a3fd0 con 0x7ff4d019bd20 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4ccff9700 1 -- 172.21.15.16:0/3654200572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80052a0 con 0x7ff4d019bd20 2024-05-08T23:24:40.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.206+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01a4450 con 0x7ff4d019bd20 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.209+0000 7ff4ccff9700 1 -- 172.21.15.16:0/3654200572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff4c801d480 con 0x7ff4d019bd20 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.209+0000 7ff4ccff9700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff4bc0a1290 0x7ff4bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.209+0000 7ff4ccff9700 1 -- 172.21.15.16:0/3654200572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7ff4c80ce200 con 0x7ff4d019bd20 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.209+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff4bc0a1290 0x7ff4bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.210+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff4bc0a1290 0x7ff4bc0a3750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005950 tx=0x7ff4c000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.210+0000 7ff4e155eb80 1 Processor -- start 2024-05-08T23:24:40.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.210+0000 7ff4e155eb80 1 -- start start 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 0x5613fb9f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9ed890 0x5613fb9edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb944150 con 0x5613fb9f2e00 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb941b10 con 0x5613fb9f3720 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb93d400 con 0x5613fb9ed890 2024-05-08T23:24:40.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 0x5613fb9f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 0x5613fb9f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60710/0 (socket says 172.21.15.16:60710) 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 -- 172.21.15.16:0/909692271 learned_addr learned my addr 172.21.15.16:0/909692271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9ed890 0x5613fb9edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 -- 172.21.15.16:0/909692271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9ed890 msgr2=0x5613fb9edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9ed890 0x5613fb9edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 -- 172.21.15.16:0/909692271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 msgr2=0x5613fb9ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 -- 172.21.15.16:0/909692271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613fb7be3b0 con 0x5613fb9f2e00 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 0x5613fb9f31e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0010fa0 tx=0x7ff4c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20654e8fb47cc882 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4c57fa700 1 -- 172.21.15.16:0/909692271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c001e620 con 0x5613fb9f2e00 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4c57fa700 1 -- 172.21.15.16:0/909692271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c0015070 con 0x5613fb9f2e00 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.211+0000 7ff4c57fa700 1 -- 172.21.15.16:0/909692271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c001c8c0 con 0x5613fb9f2e00 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 -- 172.21.15.16:0/909692271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 msgr2=0x5613fb9f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 0x5613fb9f31e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0010fa0 tx=0x7ff4c0013000 comp rx=0 tx=0).stop 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 -- 172.21.15.16:0/909692271 shutdown_connections 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9ed890 0x5613fb9edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f2e00 0x5613fb9f31e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/909692271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 -- 172.21.15.16:0/909692271 >> 172.21.15.16:0/909692271 conn(0x5613fb94b210 msgr2=0x5613fb9e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 -- 172.21.15.16:0/909692271 shutdown_connections 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.212+0000 7ff4e155eb80 1 -- 172.21.15.16:0/909692271 wait complete. 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 -- start start 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 Processor -- start 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9ed890 0x5613fb9f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 0x5613fb9f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb8d9560 con 0x5613fb9f3720 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb8d9200 con 0x5613fb9ed890 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb7c06a0 con 0x5613fb9f2e00 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4e155eb80 10 client.0 _ll_register_callbacks cb 0x5613fb8e2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 0x5613fb9f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 0x5613fb9f74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60714/0 (socket says 172.21.15.16:60714) 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/2133856896 learned_addr learned my addr 172.21.15.16:0/2133856896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9ed890 0x5613fb9f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.213+0000 7ff4c67fc700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/2133856896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 msgr2=0x5613fb9f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9ed890 msgr2=0x5613fb9f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9ed890 0x5613fb9f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4cf7fe700 1 -- 172.21.15.16:0/2133856896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c0010c50 con 0x5613fb9f3720 2024-05-08T23:24:40.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 0x5613fb9f74d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80062a0 tx=0x7ff4c809acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7314243094ee20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4b67fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4b67fc700 1 -- 172.21.15.16:0/2133856896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c809b0f0 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4b67fc700 1 -- 172.21.15.16:0/2133856896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7ff4c809fe40 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613fb896270 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4b67fc700 1 -- 172.21.15.16:0/2133856896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c809ef20 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4e155eb80 10 client.16632 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5613fb76ab60 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.214+0000 7ff4b67fc700 1 -- 172.21.15.16:0/2133856896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5613fb896270 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4b67fc700 1 -- 172.21.15.16:0/2133856896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4c801d730 con 0x5613fb9f3720 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4e155eb80 10 client.16632 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4e155eb80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4e155eb80 20 client.16632 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba06d90 0x5613fba091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4e155eb80 4 client.16632 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:40.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613fb9bc810 con 0x5613fba06d90 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.215+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba06d90 0x5613fba091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.216+0000 7ff4b67fc700 10 client.16632 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.216+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba06d90 0x5613fba091b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8087532e66b0e4ad server_cookie=b3441dca2cf1bf76 in_seq=0 out_seq=0 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.216+0000 7ff4b67fc700 1 -- 172.21.15.16:0/2133856896 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613fb9bc810 con 0x5613fba06d90 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.216+0000 7ff4b67fc700 10 client.16632 handle_command_reply: tid=0 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.216+0000 7ff4e155eb80 1 client.16632 shutdown 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 msgr2=0x5613fb9f74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 0x5613fb9f74d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80062a0 tx=0x7ff4c809acb0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 shutdown_connections 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9f3720 0x5613fb9f74d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba06d90 0x5613fba091b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9ed890 0x5613fb9f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2133856896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4b67fc700 0 client.16632 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 >> 172.21.15.16:0/2133856896 conn(0x5613fb94b210 msgr2=0x5613fb9e75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 shutdown_connections 2024-05-08T23:24:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2133856896 wait complete. 2024-05-08T23:24:40.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.217+0000 7ff4e155eb80 20 client.16632 trim_cache size 0 max 16384 2024-05-08T23:24:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.235+0000 7ff4e155eb80 1 Processor -- start 2024-05-08T23:24:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.235+0000 7ff4e155eb80 1 -- start start 2024-05-08T23:24:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb9e8b30 con 0x5613fb9ed890 2024-05-08T23:24:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb9f95c0 con 0x5613fb9f3720 2024-05-08T23:24:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb9e4dc0 con 0x5613fb9f2e00 2024-05-08T23:24:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9edc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60730/0 (socket says 172.21.15.16:60730) 2024-05-08T23:24:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 -- 172.21.15.16:0/2476816475 learned_addr learned my addr 172.21.15.16:0/2476816475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4c67fc700 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 -- 172.21.15.16:0/2476816475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 msgr2=0x5613fb9e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 -- 172.21.15.16:0/2476816475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 msgr2=0x5613fb9f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4ceffd700 1 -- 172.21.15.16:0/2476816475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613fb903560 con 0x5613fb9ed890 2024-05-08T23:24:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.236+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9edc70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0011010 tx=0x7ff4c0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdea1e323339f33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4c5ffb700 1 -- 172.21.15.16:0/2476816475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c002d040 con 0x5613fb9ed890 2024-05-08T23:24:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4c5ffb700 1 -- 172.21.15.16:0/2476816475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c002e040 con 0x5613fb9ed890 2024-05-08T23:24:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4c5ffb700 1 -- 172.21.15.16:0/2476816475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c001e420 con 0x5613fb9ed890 2024-05-08T23:24:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2476816475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 msgr2=0x5613fb9edc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9edc70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0011010 tx=0x7ff4c0011b60 comp rx=0 tx=0).stop 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2476816475 shutdown_connections 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f3720 0x5613fb9f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9edc70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/2476816475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f2e00 0x5613fb9e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.237+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2476816475 >> 172.21.15.16:0/2476816475 conn(0x5613fb94b210 msgr2=0x5613fb9f7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2476816475 shutdown_connections 2024-05-08T23:24:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 -- 172.21.15.16:0/2476816475 wait complete. 2024-05-08T23:24:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 -- start start 2024-05-08T23:24:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 Processor -- start 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f2e00 0x5613fb9fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f3720 0x5613fba5c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb9fce30 con 0x5613fb9ed890 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fb9fcfa0 con 0x5613fb9f2e00 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fba5c7d0 con 0x5613fb9f3720 2024-05-08T23:24:40.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4e155eb80 10 client.0 _ll_register_callbacks cb 0x5613fb8e2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60738/0 (socket says 172.21.15.16:60738) 2024-05-08T23:24:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4ceffd700 1 -- 172.21.15.16:0/3229720427 learned_addr learned my addr 172.21.15.16:0/3229720427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.238+0000 7ff4cf7fe700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f3720 0x5613fba5c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4c67fc700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f2e00 0x5613fb9fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4ceffd700 1 -- 172.21.15.16:0/3229720427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f3720 msgr2=0x5613fba5c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f3720 0x5613fba5c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4ceffd700 1 -- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f2e00 msgr2=0x5613fb9fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f2e00 0x5613fb9fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4ceffd700 1 -- 172.21.15.16:0/3229720427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613fb7be3b0 con 0x5613fb9ed890 2024-05-08T23:24:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4ceffd700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9fc150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000baa0 tx=0x7ff4c0004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def653be801225c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4b67fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4b67fc700 1 -- 172.21.15.16:0/3229720427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00051c0 con 0x5613fb9ed890 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4b67fc700 1 -- 172.21.15.16:0/3229720427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7ff4c0017070 con 0x5613fb9ed890 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c0015070 con 0x5613fb9ed890 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4b67fc700 1 -- 172.21.15.16:0/3229720427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c0005320 con 0x5613fb9ed890 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4e155eb80 10 client.16638 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:40.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5613fb7c2cd0 con 0x5613fb9ed890 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.239+0000 7ff4b67fc700 1 -- 172.21.15.16:0/3229720427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c0015070 con 0x5613fb9ed890 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4b67fc700 1 -- 172.21.15.16:0/3229720427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4c0034020 con 0x5613fb9ed890 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4e155eb80 10 client.16638 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4e155eb80 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4e155eb80 20 client.16638 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba5d9f0 0x5613fba5fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4e155eb80 4 client.16638 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5613fb935ec0 con 0x5613fba5d9f0 2024-05-08T23:24:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4c67fc700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba5d9f0 0x5613fba5fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4b67fc700 10 client.16638 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.240+0000 7ff4c67fc700 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba5d9f0 0x5613fba5fe10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b00fbcf2f2610f6 server_cookie=4e79ff6d1cf9f39a in_seq=0 out_seq=0 2024-05-08T23:24:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4b67fc700 1 -- 172.21.15.16:0/3229720427 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5613fb935ec0 con 0x5613fba5d9f0 2024-05-08T23:24:40.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4b67fc700 10 client.16638 handle_command_reply: tid=0 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 client.16638 shutdown 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 msgr2=0x5613fb9fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9fc150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000baa0 tx=0x7ff4c0004720 comp rx=0 tx=0).stop 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 shutdown_connections 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613fb9f3720 0x5613fba5c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fb9ed890 0x5613fb9fc150 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5613fba5d9f0 0x5613fba5fe10 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 --2- 172.21.15.16:0/3229720427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5613fb9f2e00 0x5613fb9fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 >> 172.21.15.16:0/3229720427 conn(0x5613fb94b210 msgr2=0x5613fb9f1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4b67fc700 0 client.16638 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 shutdown_connections 2024-05-08T23:24:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 1 -- 172.21.15.16:0/3229720427 wait complete. 2024-05-08T23:24:40.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4e155eb80 20 client.16638 trim_cache size 0 max 16384 2024-05-08T23:24:40.261 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:24:40.261 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:24:40.261 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:24:40.261 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:24:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff4bc0a1290 msgr2=0x7ff4bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff4bc0a1290 0x7ff4bc0a3750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005950 tx=0x7ff4c000b410 comp rx=0 tx=0).stop 2024-05-08T23:24:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 msgr2=0x7ff4d019c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.241+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 0x7ff4d019c1a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80094c0 tx=0x7ff4c80094f0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 shutdown_connections 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff4bc0a1290 0x7ff4bc0a3750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d019bd20 0x7ff4d019c1a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4d0101170 0x7ff4d0197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 --2- 172.21.15.16:0/3654200572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d0106290 0x7ff4d0197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 >> 172.21.15.16:0/3654200572 conn(0x7ff4d00fc9d0 msgr2=0x7ff4d00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 shutdown_connections 2024-05-08T23:24:40.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.242+0000 7ff4d53a4700 1 -- 172.21.15.16:0/3654200572 wait complete. 2024-05-08T23:24:40.267 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:24:40.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:24:40.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:40 smithi016 ceph-mon[42541]: pgmap v156: 129 pgs: 129 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 -- 172.21.15.16:0/50523603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 msgr2=0x7efeb0100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 --2- 172.21.15.16:0/50523603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0100350 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efea0009990 tx=0x7efea0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 -- 172.21.15.16:0/50523603 shutdown_connections 2024-05-08T23:24:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 --2- 172.21.15.16:0/50523603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeb0105090 0x7efeb010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 --2- 172.21.15.16:0/50523603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0100350 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 --2- 172.21.15.16:0/50523603 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0100920 0x7efeb0104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 -- 172.21.15.16:0/50523603 >> 172.21.15.16:0/50523603 conn(0x7efeb00fb7d0 msgr2=0x7efeb00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 -- 172.21.15.16:0/50523603 shutdown_connections 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.408+0000 7efeb4994700 1 -- 172.21.15.16:0/50523603 wait complete. 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.409+0000 7efeb4994700 1 Processor -- start 2024-05-08T23:24:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.409+0000 7efeb4994700 1 -- start start 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.409+0000 7efeb4994700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.409+0000 7efeb4994700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0105090 0x7efeb0196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.409+0000 7efeb4994700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeb019a9e0 0x7efeb019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeb4994700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019b460 con 0x7efeb00fff70 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeb4994700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019b5d0 con 0x7efeb0105090 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeb4994700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019b790 con 0x7efeb019a9e0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60758/0 (socket says 172.21.15.16:60758) 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 -- 172.21.15.16:0/3270233312 learned_addr learned my addr 172.21.15.16:0/3270233312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeaed9d700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeb019a9e0 0x7efeb019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeadd9b700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0105090 0x7efeb0196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 -- 172.21.15.16:0/3270233312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeb019a9e0 msgr2=0x7efeb019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeb019a9e0 0x7efeb019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 -- 172.21.15.16:0/3270233312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0105090 msgr2=0x7efeb0196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0105090 0x7efeb0196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 -- 172.21.15.16:0/3270233312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efeb00fff70 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeadd9b700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0105090 0x7efeb0196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:40.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeae59c700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0196100 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efea0004730 tx=0x7efea0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea001d070 con 0x7efeb00fff70 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efea000be80 con 0x7efeb00fff70 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb01a2c90 con 0x7efeb00fff70 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00173f0 con 0x7efeb00fff70 2024-05-08T23:24:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.410+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01a30b0 con 0x7efeb00fff70 2024-05-08T23:24:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.411+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0108300 con 0x7efeb00fff70 2024-05-08T23:24:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.413+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efea0022700 con 0x7efeb00fff70 2024-05-08T23:24:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.413+0000 7efe9f7fe700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe980a1290 0x7efe980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.413+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7efea00ceb20 con 0x7efeb00fff70 2024-05-08T23:24:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.413+0000 7efeadd9b700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe980a1290 0x7efe980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.414+0000 7efeadd9b700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe980a1290 0x7efe980a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efea4005fd0 tx=0x7efea4005ee0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.417+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efea0097cc0 con 0x7efeb00fff70 2024-05-08T23:24:40.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:40 smithi189 ceph-mon[46919]: pgmap v156: 129 pgs: 129 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:40.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:40 smithi155 ceph-mon[47105]: pgmap v156: 129 pgs: 129 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:40.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.693+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efeb0101440 con 0x7efeb00fff70 2024-05-08T23:24:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.694+0000 7efe9f7fe700 1 -- 172.21.15.16:0/3270233312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7efea009b2e0 con 0x7efeb00fff70 2024-05-08T23:24:40.695 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:24:40.695 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:24:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe980a1290 msgr2=0x7efe980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe980a1290 0x7efe980a3750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efea4005fd0 tx=0x7efea4005ee0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 msgr2=0x7efeb0196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0196100 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efea0004730 tx=0x7efea0004760 comp rx=0 tx=0).stop 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 shutdown_connections 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe980a1290 0x7efe980a3750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efeb019a9e0 0x7efeb019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb00fff70 0x7efeb0196100 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 --2- 172.21.15.16:0/3270233312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efeb0105090 0x7efeb0196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 >> 172.21.15.16:0/3270233312 conn(0x7efeb00fb7d0 msgr2=0x7efeb00fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 shutdown_connections 2024-05-08T23:24:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.696+0000 7efeb4994700 1 -- 172.21.15.16:0/3270233312 wait complete. 2024-05-08T23:24:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:24:40.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:24:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 -- 172.21.15.16:0/195913193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8100a10 msgr2=0x7f5ab8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 --2- 172.21.15.16:0/195913193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8100a10 0x7f5ab8104a50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5aac009990 tx=0x7f5aac009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 -- 172.21.15.16:0/195913193 shutdown_connections 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 --2- 172.21.15.16:0/195913193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab8105180 0x7f5ab810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 --2- 172.21.15.16:0/195913193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab8100060 0x7f5ab8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 --2- 172.21.15.16:0/195913193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8100a10 0x7f5ab8104a50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 -- 172.21.15.16:0/195913193 >> 172.21.15.16:0/195913193 conn(0x7f5ab8076f40 msgr2=0x7f5ab8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 -- 172.21.15.16:0/195913193 shutdown_connections 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.858+0000 7f5abd507700 1 -- 172.21.15.16:0/195913193 wait complete. 2024-05-08T23:24:40.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 Processor -- start 2024-05-08T23:24:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 -- start start 2024-05-08T23:24:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab8100060 0x7f5ab81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab8105180 0x7f5ab81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 0x7f5ab819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab819c710 con 0x7f5ab819bc90 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab819c880 con 0x7f5ab8100060 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5abd507700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab819ca40 con 0x7f5ab8105180 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 0x7f5ab819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 0x7f5ab819c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60784/0 (socket says 172.21.15.16:60784) 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2025873674 learned_addr learned my addr 172.21.15.16:0/2025873674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab8100060 0x7f5ab81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.859+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab8105180 0x7f5ab81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2025873674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab8105180 msgr2=0x7f5ab81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab8105180 0x7f5ab81978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2025873674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab8100060 msgr2=0x7f5ab81973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab8100060 0x7f5ab81973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2025873674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0095f0 con 0x7f5ab819bc90 2024-05-08T23:24:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 0x7f5ab819c110 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007ee0 tx=0x7f5ab000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5a9ffff700 1 -- 172.21.15.16:0/2025873674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5ab819bc90 2024-05-08T23:24:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab81a3fa0 con 0x7f5ab819bc90 2024-05-08T23:24:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5a9ffff700 1 -- 172.21.15.16:0/2025873674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab000f040 con 0x7f5ab819bc90 2024-05-08T23:24:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5a9ffff700 1 -- 172.21.15.16:0/2025873674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0005570 con 0x7f5ab819bc90 2024-05-08T23:24:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.860+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab81a4430 con 0x7f5ab819bc90 2024-05-08T23:24:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.862+0000 7f5a9ffff700 1 -- 172.21.15.16:0/2025873674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5ab000db20 con 0x7f5ab819bc90 2024-05-08T23:24:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.863+0000 7f5a9ffff700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5aa00a1290 0x7f5aa00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.863+0000 7f5a9ffff700 1 -- 172.21.15.16:0/2025873674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5ab00cc790 con 0x7f5ab819bc90 2024-05-08T23:24:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.863+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5aa00a1290 0x7f5aa00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5aa00a1290 0x7f5aa00a3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005950 tx=0x7f5aa800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 Processor -- start 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 -- start start 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9fe00 0x55ab97ea01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97e9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 0x55ab97e9acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97df1150 con 0x55ab97e9a890 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97deeb10 con 0x55ab97e9fe00 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.864+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97dea400 con 0x55ab97ea0720 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 0x55ab97e9acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 0x55ab97e9acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60792/0 (socket says 172.21.15.16:60792) 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2176544131 learned_addr learned my addr 172.21.15.16:0/2176544131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97e9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9fe00 0x55ab97ea01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2176544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 msgr2=0x55ab97e9a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97e9a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2176544131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9fe00 msgr2=0x55ab97ea01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9fe00 0x55ab97ea01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 -- 172.21.15.16:0/2176544131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab97c6b3b0 con 0x55ab97e9a890 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 0x55ab97e9acd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0095a00 tx=0x7f5ab0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfa227db5f4e12f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5a9d7fa700 1 -- 172.21.15.16:0/2176544131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab009d120 con 0x55ab97e9a890 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5a9d7fa700 1 -- 172.21.15.16:0/2176544131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab0004c40 con 0x55ab97e9a890 2024-05-08T23:24:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.865+0000 7f5a9d7fa700 1 -- 172.21.15.16:0/2176544131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00a1980 con 0x55ab97e9a890 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/2176544131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 msgr2=0x55ab97e9acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 0x55ab97e9acd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0095a00 tx=0x7f5ab0004ad0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/2176544131 shutdown_connections 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9a890 0x55ab97e9acd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9fe00 0x55ab97ea01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/2176544131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97e9a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/2176544131 >> 172.21.15.16:0/2176544131 conn(0x55ab97df8210 msgr2=0x55ab97e94df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/2176544131 shutdown_connections 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/2176544131 wait complete. 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 -- start start 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.866+0000 7f5ac96c1b80 1 Processor -- start 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97ea3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97ea45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97d86560 con 0x55ab97e9fe00 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97d86200 con 0x55ab97ea0720 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97c6d6a0 con 0x55ab97e9a890 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ac96c1b80 10 client.0 _ll_register_callbacks cb 0x55ab97d8fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60796/0 (socket says 172.21.15.16:60796) 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1857042855 learned_addr learned my addr 172.21.15.16:0/1857042855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97ea3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97ea45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 msgr2=0x55ab97ea3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97ea3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1857042855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 msgr2=0x55ab97ea45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97ea45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1857042855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab81a4430 con 0x55ab97e9fe00 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97ea3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea4080 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5aac015000 tx=0x7f5aac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04df1935929112c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.867+0000 7f5a9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5a9aff5700 1 -- 172.21.15.16:0/1857042855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac01d030 con 0x55ab97e9fe00 2024-05-08T23:24:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5a9aff5700 1 -- 172.21.15.16:0/1857042855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5aac00bcc0 con 0x55ab97e9fe00 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab97d43270 con 0x55ab97e9fe00 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5a9aff5700 1 -- 172.21.15.16:0/1857042855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac00be80 con 0x55ab97e9fe00 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 10 client.16656 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ab97c17b60 con 0x55ab97e9fe00 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5a9aff5700 1 -- 172.21.15.16:0/1857042855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ab97d43270 con 0x55ab97e9fe00 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5a9aff5700 1 -- 172.21.15.16:0/1857042855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5aac00f580 con 0x55ab97e9fe00 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 10 client.16656 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 10 client.16656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 20 client.16656 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97eb4250 0x55ab97eb6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 4 client.16656 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.868+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab97e69810 con 0x55ab97eb4250 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.869+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97eb4250 0x55ab97eb6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.869+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97eb4250 0x55ab97eb6670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e99d2751f2901578 server_cookie=cfabbda1e9b21c56 in_seq=0 out_seq=0 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.869+0000 7f5a9aff5700 10 client.16656 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5a9aff5700 1 -- 172.21.15.16:0/1857042855 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab97e69810 con 0x55ab97eb4250 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5a9aff5700 10 client.16656 handle_command_reply: tid=0 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 client.16656 shutdown 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 msgr2=0x55ab97ea4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea4080 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5aac015000 tx=0x7f5aac005800 comp rx=0 tx=0).stop 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 shutdown_connections 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97eb4250 0x55ab97eb6670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97ea45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97ea3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1857042855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea4080 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5a9aff5700 0 client.16656 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 >> 172.21.15.16:0/1857042855 conn(0x55ab97df8210 msgr2=0x55ab97e945a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 shutdown_connections 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1857042855 wait complete. 2024-05-08T23:24:40.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.870+0000 7f5ac96c1b80 20 client.16656 trim_cache size 0 max 16384 2024-05-08T23:24:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 Processor -- start 2024-05-08T23:24:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 -- start start 2024-05-08T23:24:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97e9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97e96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97e94d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97ea6b40 con 0x55ab97e9fe00 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97e91dc0 con 0x55ab97ea0720 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97e96830 con 0x55ab97e9a890 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97e96a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97e96a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60806/0 (socket says 172.21.15.16:60806) 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1058756942 learned_addr learned my addr 172.21.15.16:0/1058756942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.884+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97e9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97e94d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1058756942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 msgr2=0x55ab97e9a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97e9a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1058756942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 msgr2=0x55ab97e94d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97e94d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab67fc700 1 -- 172.21.15.16:0/1058756942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab97db0560 con 0x55ab97e9fe00 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97e94d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97e96a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0036c0 tx=0x7f5aac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec73f311af06319 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5a9dffb700 1 -- 172.21.15.16:0/1058756942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac01d040 con 0x55ab97e9fe00 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5a9dffb700 1 -- 172.21.15.16:0/1058756942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aac00fe90 con 0x55ab97e9fe00 2024-05-08T23:24:40.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.885+0000 7f5a9dffb700 1 -- 172.21.15.16:0/1058756942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0047f0 con 0x55ab97e9fe00 2024-05-08T23:24:40.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1058756942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 msgr2=0x55ab97e96a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97e96a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0036c0 tx=0x7f5aac009310 comp rx=0 tx=0).stop 2024-05-08T23:24:40.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1058756942 shutdown_connections 2024-05-08T23:24:40.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97ea0720 0x55ab97e94d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97e9a890 0x55ab97e9a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/1058756942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97e96a00 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1058756942 >> 172.21.15.16:0/1058756942 conn(0x55ab97df8210 msgr2=0x55ab97ea2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1058756942 shutdown_connections 2024-05-08T23:24:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/1058756942 wait complete. 2024-05-08T23:24:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 -- start start 2024-05-08T23:24:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.886+0000 7f5ac96c1b80 1 Processor -- start 2024-05-08T23:24:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9a890 0x55ab97eab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97ea9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97ea9e00 con 0x55ab97e9fe00 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97ea9f70 con 0x55ab97e9a890 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab97f09750 con 0x55ab97ea0720 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 10 client.0 _ll_register_callbacks cb 0x55ab97d8fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60814/0 (socket says 172.21.15.16:60814) 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 -- 172.21.15.16:0/4240162581 learned_addr learned my addr 172.21.15.16:0/4240162581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab77fe700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97ea9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9a890 0x55ab97eab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 -- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 msgr2=0x55ab97ea9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97ea9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 -- 172.21.15.16:0/4240162581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9a890 msgr2=0x55ab97eab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9a890 0x55ab97eab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 -- 172.21.15.16:0/4240162581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab97c17b60 con 0x55ab97e9fe00 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ab67fc700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea9150 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005460 tx=0x7f5aac00b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eba82efd2ec1124 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5a9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5a9aff5700 1 -- 172.21.15.16:0/4240162581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00bcc0 con 0x55ab97e9fe00 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5a9aff5700 1 -- 172.21.15.16:0/4240162581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f5aac000e40 con 0x55ab97e9fe00 2024-05-08T23:24:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.887+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ab0004c40 con 0x55ab97e9fe00 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5a9aff5700 1 -- 172.21.15.16:0/4240162581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac00be20 con 0x55ab97e9fe00 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 10 client.16662 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ab97c6fcd0 con 0x55ab97e9fe00 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5a9aff5700 1 -- 172.21.15.16:0/4240162581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ab0004c40 con 0x55ab97e9fe00 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5a9aff5700 1 -- 172.21.15.16:0/4240162581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5aac024570 con 0x55ab97e9fe00 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 10 client.16662 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 10 client.16662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 20 client.16662 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97f0a980 0x55ab97f0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 4 client.16662 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab97de2ec0 con 0x55ab97f0a980 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.888+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97f0a980 0x55ab97f0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ab6ffd700 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97f0a980 0x55ab97f0cd70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cb6bad20e6c79ed server_cookie=d1639601fc8fc9d9 in_seq=0 out_seq=0 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5a9aff5700 10 client.16662 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5a9aff5700 1 -- 172.21.15.16:0/4240162581 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab97de2ec0 con 0x55ab97f0a980 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5a9aff5700 10 client.16662 handle_command_reply: tid=0 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 client.16662 shutdown 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 msgr2=0x55ab97ea9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea9150 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5aac005460 tx=0x7f5aac00b5f0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 shutdown_connections 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab97ea0720 0x55ab97ea9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ab97f0a980 0x55ab97f0cd70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ab97e9a890 0x55ab97eab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5a9aff5700 0 client.16662 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:24:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 --2- 172.21.15.16:0/4240162581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab97e9fe00 0x55ab97ea9150 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 >> 172.21.15.16:0/4240162581 conn(0x55ab97df8210 msgr2=0x55ab97e9ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 shutdown_connections 2024-05-08T23:24:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 1 -- 172.21.15.16:0/4240162581 wait complete. 2024-05-08T23:24:40.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.889+0000 7f5ac96c1b80 20 client.16662 trim_cache size 0 max 16384 2024-05-08T23:24:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:24:40.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5aa00a1290 msgr2=0x7f5aa00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5aa00a1290 0x7f5aa00a3750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005950 tx=0x7f5aa800b410 comp rx=0 tx=0).stop 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 msgr2=0x7f5ab819c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 0x7f5ab819c110 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0007ee0 tx=0x7f5ab000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 shutdown_connections 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5aa00a1290 0x7f5aa00a3750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab819bc90 0x7f5ab819c110 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ab8100060 0x7f5ab81973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 --2- 172.21.15.16:0/2025873674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ab8105180 0x7f5ab81978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 >> 172.21.15.16:0/2025873674 conn(0x7f5ab8076f40 msgr2=0x7f5ab8069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 shutdown_connections 2024-05-08T23:24:40.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:40.890+0000 7f5abd507700 1 -- 172.21.15.16:0/2025873674 wait complete. 2024-05-08T23:24:40.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:24:41.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 -- 172.21.15.16:0/2190101750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a01062a0 msgr2=0x7fd8a010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2190101750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a01062a0 0x7fd8a010de20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd89800b0a0 tx=0x7fd89800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 -- 172.21.15.16:0/2190101750 shutdown_connections 2024-05-08T23:24:41.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2190101750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a01062a0 0x7fd8a010de20 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2190101750 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8a0101180 0x7fd8a0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2190101750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8a0101b30 0x7fd8a0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 -- 172.21.15.16:0/2190101750 >> 172.21.15.16:0/2190101750 conn(0x7fd8a00fc9e0 msgr2=0x7fd8a00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.043+0000 7fd8a565e700 1 -- 172.21.15.16:0/2190101750 shutdown_connections 2024-05-08T23:24:41.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.044+0000 7fd8a565e700 1 -- 172.21.15.16:0/2190101750 wait complete. 2024-05-08T23:24:41.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.044+0000 7fd8a565e700 1 Processor -- start 2024-05-08T23:24:41.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.044+0000 7fd8a565e700 1 -- start start 2024-05-08T23:24:41.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd8a565e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 0x7fd8a0197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd8a565e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8a01062a0 0x7fd8a0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd8a565e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8a019bde0 0x7fd8a019c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd8a565e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a019c860 con 0x7fd8a0101180 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd8a565e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a019c9d0 con 0x7fd8a01062a0 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd8a565e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a019cb90 con 0x7fd8a019bde0 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 0x7fd8a0197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 0x7fd8a0197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60828/0 (socket says 172.21.15.16:60828) 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 -- 172.21.15.16:0/2537982400 learned_addr learned my addr 172.21.15.16:0/2537982400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd897fff700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8a01062a0 0x7fd8a0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89f7fe700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8a019bde0 0x7fd8a019c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 -- 172.21.15.16:0/2537982400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8a019bde0 msgr2=0x7fd8a019c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8a019bde0 0x7fd8a019c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 -- 172.21.15.16:0/2537982400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8a01062a0 msgr2=0x7fd8a0197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8a01062a0 0x7fd8a0197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 -- 172.21.15.16:0/2537982400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898009d00 con 0x7fd8a0101180 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.045+0000 7fd89effd700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 0x7fd8a0197500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9a0 tx=0x7fd89000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.046+0000 7fd89cff9700 1 -- 172.21.15.16:0/2537982400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd890009790 con 0x7fd8a0101180 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.046+0000 7fd89cff9700 1 -- 172.21.15.16:0/2537982400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd890010430 con 0x7fd8a0101180 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.046+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a01a40f0 con 0x7fd8a0101180 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.046+0000 7fd89cff9700 1 -- 172.21.15.16:0/2537982400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000cea0 con 0x7fd8a0101180 2024-05-08T23:24:41.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.046+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a01a4620 con 0x7fd8a0101180 2024-05-08T23:24:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.048+0000 7fd89cff9700 1 -- 172.21.15.16:0/2537982400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd8900105a0 con 0x7fd8a0101180 2024-05-08T23:24:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.049+0000 7fd89cff9700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd88c0a1310 0x7fd88c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.049+0000 7fd89cff9700 1 -- 172.21.15.16:0/2537982400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fd890014070 con 0x7fd8a0101180 2024-05-08T23:24:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.049+0000 7fd897fff700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd88c0a1310 0x7fd88c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.049+0000 7fd897fff700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd88c0a1310 0x7fd88c0a37d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0198a50 tx=0x7fd88800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 Processor -- start 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 -- start start 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdb50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcd07150 con 0x557afcdba210 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcd04b10 con 0x557afcdb4ca0 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcd00400 con 0x557afcdbab30 2024-05-08T23:24:41.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd89effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd89effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdba5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60842/0 (socket says 172.21.15.16:60842) 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd89effd700 1 -- 172.21.15.16:0/1861828381 learned_addr learned my addr 172.21.15.16:0/1861828381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd89f7fe700 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdb50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.050+0000 7fd897fff700 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89effd700 1 -- 172.21.15.16:0/1861828381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 msgr2=0x557afcdb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89effd700 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89effd700 1 -- 172.21.15.16:0/1861828381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 msgr2=0x557afcdb50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89effd700 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdb50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89effd700 1 -- 172.21.15.16:0/1861828381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557afcb813b0 con 0x557afcdba210 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89f7fe700 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdb50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd89effd700 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdba5f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8900a1eb0 tx=0x7fd890099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c763f60ac9cfdb8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd8957fa700 1 -- 172.21.15.16:0/1861828381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89009dd90 con 0x557afcdba210 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd8957fa700 1 -- 172.21.15.16:0/1861828381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8900a23e0 con 0x557afcdba210 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.051+0000 7fd8957fa700 1 -- 172.21.15.16:0/1861828381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900a1f50 con 0x557afcdba210 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1861828381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 msgr2=0x557afcdba5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdba5f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd8900a1eb0 tx=0x7fd890099bb0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1861828381 shutdown_connections 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdb50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdba5f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1861828381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1861828381 >> 172.21.15.16:0/1861828381 conn(0x557afcd0e210 msgr2=0x557afcdaadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1861828381 shutdown_connections 2024-05-08T23:24:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1861828381 wait complete. 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 -- start start 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.052+0000 7fd8b1818b80 1 Processor -- start 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 0x557afcdbdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdba210 0x557afcdbe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdbe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcc9c560 con 0x557afcdb4ca0 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcc9c200 con 0x557afcdba210 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcb836a0 con 0x557afcdbab30 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8b1818b80 10 client.0 _ll_register_callbacks cb 0x557afcca5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:41.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 0x557afcdbdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 0x557afcdbdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60852/0 (socket says 172.21.15.16:60852) 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 -- 172.21.15.16:0/2450942119 learned_addr learned my addr 172.21.15.16:0/2450942119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89f7fe700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdbe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd897fff700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdba210 0x557afcdbe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 -- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 msgr2=0x557afcdbe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdbe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 -- 172.21.15.16:0/2450942119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdba210 msgr2=0x557afcdbe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdba210 0x557afcdbe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 -- 172.21.15.16:0/2450942119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a010e690 con 0x557afcdb4ca0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd89effd700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 0x557afcdbdea0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd8900969d0 tx=0x7fd890010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a6fa6aa86a3261 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.053+0000 7fd8867fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8867fc700 1 -- 172.21.15.16:0/2450942119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89009dd90 con 0x557afcdb4ca0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8867fc700 1 -- 172.21.15.16:0/2450942119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fd890099660 con 0x557afcdb4ca0 2024-05-08T23:24:41.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557afcc59270 con 0x557afcdb4ca0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8867fc700 1 -- 172.21.15.16:0/2450942119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8900a1f50 con 0x557afcdb4ca0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 10 client.16674 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fd8a01a4620 con 0x557afcdb4ca0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8867fc700 1 -- 172.21.15.16:0/2450942119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557afcc59270 con 0x557afcdb4ca0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8867fc700 1 -- 172.21.15.16:0/2450942119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd8900b2b10 con 0x557afcdb4ca0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 10 client.16674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 20 client.16674 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afcdce140 0x557afcdd0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 4 client.16674 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.054+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557afcd7f810 con 0x557afcdce140 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.055+0000 7fd897fff700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afcdce140 0x557afcdd0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.055+0000 7fd897fff700 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afcdce140 0x557afcdd0560 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba8cf5a40e74b9f3 server_cookie=dee213071d19cbc8 in_seq=0 out_seq=0 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.055+0000 7fd8867fc700 10 client.16674 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:41.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8867fc700 1 -- 172.21.15.16:0/2450942119 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557afcd7f810 con 0x557afcdce140 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8867fc700 10 client.16674 handle_command_reply: tid=0 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 client.16674 shutdown 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 msgr2=0x557afcdbdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 0x557afcdbdea0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd8900969d0 tx=0x7fd890010bf0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 shutdown_connections 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdbe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdb4ca0 0x557afcdbdea0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afcdce140 0x557afcdd0560 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2450942119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdba210 0x557afcdbe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 >> 172.21.15.16:0/2450942119 conn(0x557afcd0e210 msgr2=0x557afcdaa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8867fc700 0 client.16674 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 shutdown_connections 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2450942119 wait complete. 2024-05-08T23:24:41.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.056+0000 7fd8b1818b80 20 client.16674 trim_cache size 0 max 16384 2024-05-08T23:24:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.070+0000 7fd8b1818b80 1 Processor -- start 2024-05-08T23:24:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.070+0000 7fd8b1818b80 1 -- start start 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdd0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdaad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcdbf4a0 con 0x557afcdba210 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcda7dc0 con 0x557afcdb4ca0 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcdac670 con 0x557afcdbab30 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd89effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdd0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60868/0 (socket says 172.21.15.16:60868) 2024-05-08T23:24:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd89effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdd0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38400/0 (socket says 172.21.15.16:38400) 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 -- 172.21.15.16:0/2569923590 learned_addr learned my addr 172.21.15.16:0/2569923590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd89f7fe700 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdaad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 -- 172.21.15.16:0/2569923590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 msgr2=0x557afcdaad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdaad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 -- 172.21.15.16:0/2569923590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 msgr2=0x557afcdd0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdd0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 -- 172.21.15.16:0/2569923590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557afcb2db60 con 0x557afcdba210 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd897fff700 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdca020 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd888005f90 tx=0x7fd888013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1219ce73701b1c1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd895ffb700 1 -- 172.21.15.16:0/2569923590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88802b070 con 0x557afcdba210 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd895ffb700 1 -- 172.21.15.16:0/2569923590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd888015070 con 0x557afcdba210 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.071+0000 7fd895ffb700 1 -- 172.21.15.16:0/2569923590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888026070 con 0x557afcdba210 2024-05-08T23:24:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2569923590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 msgr2=0x557afcdca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdca020 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd888005f90 tx=0x7fd888013000 comp rx=0 tx=0).stop 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2569923590 shutdown_connections 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdbab30 0x557afcdaad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdb4ca0 0x557afcdd0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/2569923590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdca020 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2569923590 >> 172.21.15.16:0/2569923590 conn(0x557afcd0e210 msgr2=0x557afcdb5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2569923590 shutdown_connections 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.072+0000 7fd8b1818b80 1 -- 172.21.15.16:0/2569923590 wait complete. 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 -- start start 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 Processor -- start 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdb4ca0 0x557afcdc5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdc3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdbab30 0x557afcdc3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcdc41f0 con 0x557afcdba210 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afcdc4360 con 0x557afcdbab30 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557afce23600 con 0x557afcdb4ca0 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd8b1818b80 10 client.0 _ll_register_callbacks cb 0x557afcca5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdc3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdc3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60884/0 (socket says 172.21.15.16:60884) 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 -- 172.21.15.16:0/1524362711 learned_addr learned my addr 172.21.15.16:0/1524362711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd89f7fe700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdbab30 0x557afcdc3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd89effd700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdb4ca0 0x557afcdc5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 -- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdb4ca0 msgr2=0x557afcdc5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdb4ca0 0x557afcdc5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 -- 172.21.15.16:0/1524362711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdbab30 msgr2=0x557afcdc3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdbab30 0x557afcdc3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.073+0000 7fd897fff700 1 -- 172.21.15.16:0/1524362711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888010ba0 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd897fff700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdc3550 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd88802a000 tx=0x7fd888021190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9c7f9970bd91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8867fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8867fc700 1 -- 172.21.15.16:0/1524362711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88802b040 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8867fc700 1 -- 172.21.15.16:0/1524362711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fd888017070 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8900a23e0 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8867fc700 1 -- 172.21.15.16:0/1524362711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd888026040 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 10 client.16680 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557afccc6560 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8867fc700 1 -- 172.21.15.16:0/1524362711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8900a23e0 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8867fc700 1 -- 172.21.15.16:0/1524362711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd888038070 con 0x557afcdba210 2024-05-08T23:24:41.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 10 client.16680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 20 client.16680 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.074+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afce24cc0 0x557afce270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd8b1818b80 4 client.16680 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557afccf8ec0 con 0x557afce24cc0 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd89effd700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afce24cc0 0x557afce270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd8867fc700 10 client.16680 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd89effd700 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afce24cc0 0x557afce270e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=815747968cdf0538 server_cookie=88c76e0f96fa16d3 in_seq=0 out_seq=0 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd8867fc700 1 -- 172.21.15.16:0/1524362711 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557afccf8ec0 con 0x557afce24cc0 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd8867fc700 10 client.16680 handle_command_reply: tid=0 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.075+0000 7fd8b1818b80 1 client.16680 shutdown 2024-05-08T23:24:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 msgr2=0x557afcdc3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdc3550 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd88802a000 tx=0x7fd888021190 comp rx=0 tx=0).stop 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 shutdown_connections 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x557afce24cc0 0x557afce270e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557afcdbab30 0x557afcdc3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557afcdb4ca0 0x557afcdc5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8867fc700 0 client.16680 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 --2- 172.21.15.16:0/1524362711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557afcdba210 0x557afcdc3550 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 >> 172.21.15.16:0/1524362711 conn(0x557afcd0e210 msgr2=0x557afcdbc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 shutdown_connections 2024-05-08T23:24:41.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 1 -- 172.21.15.16:0/1524362711 wait complete. 2024-05-08T23:24:41.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8b1818b80 20 client.16680 trim_cache size 0 max 16384 2024-05-08T23:24:41.084 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd88c0a1310 msgr2=0x7fd88c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd88c0a1310 0x7fd88c0a37d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0198a50 tx=0x7fd88800b410 comp rx=0 tx=0).stop 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 msgr2=0x7fd8a0197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 0x7fd8a0197500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9a0 tx=0x7fd89000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 shutdown_connections 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd88c0a1310 0x7fd88c0a37d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8a019bde0 0x7fd8a019c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a0101180 0x7fd8a0197500 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 --2- 172.21.15.16:0/2537982400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8a01062a0 0x7fd8a0197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.076+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 >> 172.21.15.16:0/2537982400 conn(0x7fd8a00fc9e0 msgr2=0x7fd8a00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.077+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 shutdown_connections 2024-05-08T23:24:41.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.077+0000 7fd8a565e700 1 -- 172.21.15.16:0/2537982400 wait complete. 2024-05-08T23:24:41.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:24:41.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 -- 172.21.15.16:0/3078474950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc101b10 msgr2=0x7f90dc105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 --2- 172.21.15.16:0/3078474950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc101b10 0x7f90dc105b50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009990 tx=0x7f90cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 -- 172.21.15.16:0/3078474950 shutdown_connections 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 --2- 172.21.15.16:0/3078474950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc106280 0x7f90dc10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 --2- 172.21.15.16:0/3078474950 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc101160 0x7f90dc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 --2- 172.21.15.16:0/3078474950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc101b10 0x7f90dc105b50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 -- 172.21.15.16:0/3078474950 >> 172.21.15.16:0/3078474950 conn(0x7f90dc0fc9e0 msgr2=0x7f90dc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 -- 172.21.15.16:0/3078474950 shutdown_connections 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.229+0000 7f90e13c2700 1 -- 172.21.15.16:0/3078474950 wait complete. 2024-05-08T23:24:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.230+0000 7f90e13c2700 1 Processor -- start 2024-05-08T23:24:41.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.230+0000 7f90e13c2700 1 -- start start 2024-05-08T23:24:41.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.230+0000 7f90e13c2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc101160 0x7f90dc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.230+0000 7f90e13c2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc106280 0x7f90dc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90e13c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 0x7f90dc1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90e13c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc1a5bd0 con 0x7f90dc073d20 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90e13c2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc1a5d10 con 0x7f90dc106280 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90e13c2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc1a5ed0 con 0x7f90dc101160 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 0x7f90dc1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 0x7f90dc1a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60888/0 (socket says 172.21.15.16:60888) 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90daffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc101160 0x7f90dc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 -- 172.21.15.16:0/279192145 learned_addr learned my addr 172.21.15.16:0/279192145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90da7fc700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc106280 0x7f90dc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 -- 172.21.15.16:0/279192145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc101160 msgr2=0x7f90dc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc101160 0x7f90dc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 -- 172.21.15.16:0/279192145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc106280 msgr2=0x7f90dc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc106280 0x7f90dc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 -- 172.21.15.16:0/279192145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0095f0 con 0x7f90dc073d20 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90daffd700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc101160 0x7f90dc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90da7fc700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc106280 0x7f90dc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90db7fe700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 0x7f90dc1a5600 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f90d400ef30 tx=0x7f90d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90c3fff700 1 -- 172.21.15.16:0/279192145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4018070 con 0x7f90dc073d20 2024-05-08T23:24:41.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90dc1a6150 con 0x7f90dc073d20 2024-05-08T23:24:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.231+0000 7f90c3fff700 1 -- 172.21.15.16:0/279192145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90d400f040 con 0x7f90dc073d20 2024-05-08T23:24:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.232+0000 7f90c3fff700 1 -- 172.21.15.16:0/279192145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d400cc70 con 0x7f90dc073d20 2024-05-08T23:24:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.232+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc1a6610 con 0x7f90dc073d20 2024-05-08T23:24:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.234+0000 7f90c3fff700 1 -- 172.21.15.16:0/279192145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f90d4007500 con 0x7f90dc073d20 2024-05-08T23:24:41.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.235+0000 7f90c3fff700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f90c40a1290 0x7f90c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.235+0000 7f90daffd700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f90c40a1290 0x7f90c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.235+0000 7f90c3fff700 1 -- 172.21.15.16:0/279192145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f90d40ccb20 con 0x7f90dc073d20 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.235+0000 7f90daffd700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f90c40a1290 0x7f90c40a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90d0011490 tx=0x7f90d0009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.235+0000 7f90ed57cb80 1 Processor -- start 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 -- start start 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb0571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb051350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb051cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdafa8150 con 0x55dfdb057720 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdafa5b10 con 0x55dfdb056e00 2024-05-08T23:24:41.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdafa1400 con 0x55dfdb051890 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb051350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb051350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60896/0 (socket says 172.21.15.16:60896) 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 -- 172.21.15.16:0/1334053225 learned_addr learned my addr 172.21.15.16:0/1334053225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90db7fe700 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb051cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90daffd700 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb0571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 -- 172.21.15.16:0/1334053225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 msgr2=0x55dfdb051cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb051cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 -- 172.21.15.16:0/1334053225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 msgr2=0x55dfdb0571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb0571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 -- 172.21.15.16:0/1334053225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfdae223b0 con 0x55dfdb057720 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90daffd700 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb0571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.236+0000 7f90da7fc700 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb051350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b2e0 tx=0x7f90cc005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fab548873ca03bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90c17fa700 1 -- 172.21.15.16:0/1334053225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0049e0 con 0x55dfdb057720 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90c17fa700 1 -- 172.21.15.16:0/1334053225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90cc01e040 con 0x55dfdb057720 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90c17fa700 1 -- 172.21.15.16:0/1334053225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00b990 con 0x55dfdb057720 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90ed57cb80 1 -- 172.21.15.16:0/1334053225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 msgr2=0x55dfdb051350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb051350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00b2e0 tx=0x7f90cc005c80 comp rx=0 tx=0).stop 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90ed57cb80 1 -- 172.21.15.16:0/1334053225 shutdown_connections 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb051cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.237+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb0571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/1334053225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb051350 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- 172.21.15.16:0/1334053225 >> 172.21.15.16:0/1334053225 conn(0x55dfdafaf210 msgr2=0x55dfdb04bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- 172.21.15.16:0/1334053225 shutdown_connections 2024-05-08T23:24:41.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- 172.21.15.16:0/1334053225 wait complete. 2024-05-08T23:24:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- start start 2024-05-08T23:24:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 Processor -- start 2024-05-08T23:24:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 0x55dfdb05ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb056e00 0x55dfdb05b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb057720 0x55dfdb05b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdaf3d560 con 0x55dfdb051890 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdaf3d200 con 0x55dfdb057720 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdae246a0 con 0x55dfdb056e00 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90ed57cb80 10 client.0 _ll_register_callbacks cb 0x55dfdaf46b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90daffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 0x55dfdb05ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb056e00 0x55dfdb05b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90daffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 0x55dfdb05ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60898/0 (socket says 172.21.15.16:60898) 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.238+0000 7f90daffd700 1 -- 172.21.15.16:0/2985512331 learned_addr learned my addr 172.21.15.16:0/2985512331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90db7fe700 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb057720 0x55dfdb05b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90daffd700 1 -- 172.21.15.16:0/2985512331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb056e00 msgr2=0x55dfdb05b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90daffd700 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb056e00 0x55dfdb05b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90daffd700 1 -- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb057720 msgr2=0x55dfdb05b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90daffd700 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb057720 0x55dfdb05b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90daffd700 1 -- 172.21.15.16:0/2985512331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc10e670 con 0x55dfdb051890 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90daffd700 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 0x55dfdb05ab60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f90d0003b00 tx=0x7f90d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a16560978b7d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90beff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90beff5700 1 -- 172.21.15.16:0/2985512331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d000ec50 con 0x55dfdb051890 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90beff5700 1 -- 172.21.15.16:0/2985512331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f90d0003e10 con 0x55dfdb051890 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfdaefa270 con 0x55dfdb051890 2024-05-08T23:24:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90beff5700 1 -- 172.21.15.16:0/2985512331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90d000edb0 con 0x55dfdb051890 2024-05-08T23:24:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90ed57cb80 10 client.16692 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.239+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dfdadceb60 con 0x55dfdb051890 2024-05-08T23:24:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90beff5700 1 -- 172.21.15.16:0/2985512331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dfdaefa270 con 0x55dfdb051890 2024-05-08T23:24:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90beff5700 1 -- 172.21.15.16:0/2985512331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f90d001e510 con 0x55dfdb051890 2024-05-08T23:24:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90ed57cb80 10 client.16692 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90ed57cb80 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90ed57cb80 20 client.16692 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb06b220 0x55dfdb06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90ed57cb80 4 client.16692 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfdb020810 con 0x55dfdb06b220 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.240+0000 7f90da7fc700 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb06b220 0x55dfdb06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90beff5700 10 client.16692 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90da7fc700 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb06b220 0x55dfdb06d640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8ddffbb0a1212c server_cookie=30ad350db437f3a1 in_seq=0 out_seq=0 2024-05-08T23:24:41.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90beff5700 1 -- 172.21.15.16:0/2985512331 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfdb020810 con 0x55dfdb06b220 2024-05-08T23:24:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90beff5700 10 client.16692 handle_command_reply: tid=0 2024-05-08T23:24:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90ed57cb80 1 client.16692 shutdown 2024-05-08T23:24:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 msgr2=0x55dfdb05ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.241+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 0x55dfdb05ab60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f90d0003b00 tx=0x7f90d00047c0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 shutdown_connections 2024-05-08T23:24:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb06b220 0x55dfdb06d640 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb057720 0x55dfdb05b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb051890 0x55dfdb05ab60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90beff5700 0 client.16692 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2985512331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb056e00 0x55dfdb05b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 >> 172.21.15.16:0/2985512331 conn(0x55dfdafaf210 msgr2=0x55dfdb04b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 shutdown_connections 2024-05-08T23:24:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2985512331 wait complete. 2024-05-08T23:24:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.242+0000 7f90ed57cb80 20 client.16692 trim_cache size 0 max 16384 2024-05-08T23:24:41.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 Processor -- start 2024-05-08T23:24:41.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 -- start start 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb066bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb056350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb04bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdb0516d0 con 0x55dfdb057720 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdb072850 con 0x55dfdb056e00 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdb04ca50 con 0x55dfdb051890 2024-05-08T23:24:41.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb04bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb056350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb056350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:38474/0 (socket says 172.21.15.16:38474) 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90da7fc700 1 -- 172.21.15.16:0/741020819 learned_addr learned my addr 172.21.15.16:0/741020819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.256+0000 7f90daffd700 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb066bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90db7fe700 1 -- 172.21.15.16:0/741020819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 msgr2=0x55dfdb066bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90db7fe700 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb066bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90db7fe700 1 -- 172.21.15.16:0/741020819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 msgr2=0x55dfdb056350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90db7fe700 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb056350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90db7fe700 1 -- 172.21.15.16:0/741020819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfdaf67560 con 0x55dfdb057720 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90da7fc700 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb056350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90db7fe700 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb04bd20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f90d40c9f30 tx=0x7f90d4099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a1bfc16fe42a8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90c1ffb700 1 -- 172.21.15.16:0/741020819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d4012340 con 0x55dfdb057720 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90c1ffb700 1 -- 172.21.15.16:0/741020819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90d409a080 con 0x55dfdb057720 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.257+0000 7f90c1ffb700 1 -- 172.21.15.16:0/741020819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d40a1d70 con 0x55dfdb057720 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 -- 172.21.15.16:0/741020819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 msgr2=0x55dfdb04bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb04bd20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f90d40c9f30 tx=0x7f90d4099870 comp rx=0 tx=0).stop 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 -- 172.21.15.16:0/741020819 shutdown_connections 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb057720 0x55dfdb04bd20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb051890 0x55dfdb066bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/741020819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb056e00 0x55dfdb056350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 -- 172.21.15.16:0/741020819 >> 172.21.15.16:0/741020819 conn(0x55dfdafaf210 msgr2=0x55dfdb055c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 -- 172.21.15.16:0/741020819 shutdown_connections 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 -- 172.21.15.16:0/741020819 wait complete. 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 -- start start 2024-05-08T23:24:41.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.258+0000 7f90ed57cb80 1 Processor -- start 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb051890 0x55dfdb0601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 0x55dfdb060730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb057720 0x55dfdb0c0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdb060ed0 con 0x55dfdb056e00 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdb0c0c90 con 0x55dfdb051890 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdb0c0e50 con 0x55dfdb057720 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90ed57cb80 10 client.0 _ll_register_callbacks cb 0x55dfdaf46b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:41.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 0x55dfdb060730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 0x55dfdb060730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60918/0 (socket says 172.21.15.16:60918) 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 -- 172.21.15.16:0/2539424659 learned_addr learned my addr 172.21.15.16:0/2539424659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90daffd700 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb051890 0x55dfdb0601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 -- 172.21.15.16:0/2539424659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb057720 msgr2=0x55dfdb0c0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb057720 0x55dfdb0c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 -- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb051890 msgr2=0x55dfdb0601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb051890 0x55dfdb0601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 -- 172.21.15.16:0/2539424659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfdae223b0 con 0x55dfdb056e00 2024-05-08T23:24:41.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90da7fc700 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 0x55dfdb060730 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0098e0 tx=0x7f90cc005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f848c7db035dae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.259+0000 7f90beff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90beff5700 1 -- 172.21.15.16:0/2539424659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc004bf0 con 0x55dfdb056e00 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90beff5700 1 -- 172.21.15.16:0/2539424659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f90cc004d50 con 0x55dfdb056e00 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90cc01e040 con 0x55dfdb056e00 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90beff5700 1 -- 172.21.15.16:0/2539424659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90cc00b6b0 con 0x55dfdb056e00 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 10 client.16698 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f90dc1a6610 con 0x55dfdb056e00 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90beff5700 1 -- 172.21.15.16:0/2539424659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90cc01e040 con 0x55dfdb056e00 2024-05-08T23:24:41.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90beff5700 1 -- 172.21.15.16:0/2539424659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f90cc0303d0 con 0x55dfdb056e00 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 10 client.16698 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 20 client.16698 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb0c21b0 0x55dfdb0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 4 client.16698 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.260+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dfdaf99ec0 con 0x55dfdb0c21b0 2024-05-08T23:24:41.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90daffd700 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb0c21b0 0x55dfdb0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90daffd700 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb0c21b0 0x55dfdb0c45d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949c89418f7e956a server_cookie=a49cf86de8828e31 in_seq=0 out_seq=0 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90beff5700 10 client.16698 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90beff5700 1 -- 172.21.15.16:0/2539424659 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dfdaf99ec0 con 0x55dfdb0c21b0 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90beff5700 10 client.16698 handle_command_reply: tid=0 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90ed57cb80 1 client.16698 shutdown 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 msgr2=0x55dfdb060730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.261+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 0x55dfdb060730 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0098e0 tx=0x7f90cc005670 comp rx=0 tx=0).stop 2024-05-08T23:24:41.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 shutdown_connections 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dfdb057720 0x55dfdb0c0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dfdb0c21b0 0x55dfdb0c45d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dfdb051890 0x55dfdb0601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 --2- 172.21.15.16:0/2539424659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdb056e00 0x55dfdb060730 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90beff5700 0 client.16698 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 >> 172.21.15.16:0/2539424659 conn(0x55dfdafaf210 msgr2=0x55dfdb05b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 shutdown_connections 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 1 -- 172.21.15.16:0/2539424659 wait complete. 2024-05-08T23:24:41.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90ed57cb80 20 client.16698 trim_cache size 0 max 16384 2024-05-08T23:24:41.278 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:24:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f90c40a1290 msgr2=0x7f90c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f90c40a1290 0x7f90c40a3750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90d0011490 tx=0x7f90d0009260 comp rx=0 tx=0).stop 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 msgr2=0x7f90dc1a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 0x7f90dc1a5600 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f90d400ef30 tx=0x7f90d400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 shutdown_connections 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f90c40a1290 0x7f90c40a3750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90dc073d20 0x7f90dc1a5600 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90dc101160 0x7f90dc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 --2- 172.21.15.16:0/279192145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90dc106280 0x7f90dc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.262+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 >> 172.21.15.16:0/279192145 conn(0x7f90dc0fc9e0 msgr2=0x7f90dc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.263+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 shutdown_connections 2024-05-08T23:24:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:41.263+0000 7f90e13c2700 1 -- 172.21.15.16:0/279192145 wait complete. 2024-05-08T23:24:41.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:41 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3270233312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:41.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:41 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3270233312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:41.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:41 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3270233312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:24:42.283 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:24:42.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:24:42.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:42 smithi016 ceph-mon[42541]: pgmap v157: 129 pgs: 129 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:42.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:42 smithi189 ceph-mon[46919]: pgmap v157: 129 pgs: 129 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 -- 172.21.15.16:0/3516971473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101b10 msgr2=0x7fa3c8105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/3516971473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101b10 0x7fa3c8105b50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009990 tx=0x7fa3bc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 -- 172.21.15.16:0/3516971473 shutdown_connections 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/3516971473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3c8106280 0x7fa3c810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/3516971473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3c8101160 0x7fa3c8101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/3516971473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101b10 0x7fa3c8105b50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 -- 172.21.15.16:0/3516971473 >> 172.21.15.16:0/3516971473 conn(0x7fa3c80fc9e0 msgr2=0x7fa3c80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 -- 172.21.15.16:0/3516971473 shutdown_connections 2024-05-08T23:24:42.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.435+0000 7fa3ccc14700 1 -- 172.21.15.16:0/3516971473 wait complete. 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 Processor -- start 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 -- start start 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3c8101160 0x7fa3c80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3c8106280 0x7fa3c80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 0x7fa3c81a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a5b40 con 0x7fa3c8073d20 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a5c80 con 0x7fa3c8101160 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3ccc14700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81a5e40 con 0x7fa3c8106280 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 0x7fa3c81a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 0x7fa3c81a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60942/0 (socket says 172.21.15.16:60942) 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1064496830 learned_addr learned my addr 172.21.15.16:0/1064496830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3c8101160 0x7fa3c80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.436+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3c8106280 0x7fa3c80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1064496830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3c8106280 msgr2=0x7fa3c80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3c8106280 0x7fa3c80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1064496830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3c8101160 msgr2=0x7fa3c80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3c8101160 0x7fa3c80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1064496830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0095f0 con 0x7fa3c8073d20 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 0x7fa3c81a5570 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ef30 tx=0x7fa3c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3b77fe700 1 -- 172.21.15.16:0/1064496830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0018070 con 0x7fa3c8073d20 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3b77fe700 1 -- 172.21.15.16:0/1064496830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3c000f040 con 0x7fa3c8073d20 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c81a60c0 con 0x7fa3c8073d20 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3b77fe700 1 -- 172.21.15.16:0/1064496830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000cbc0 con 0x7fa3c8073d20 2024-05-08T23:24:42.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.437+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c81a6550 con 0x7fa3c8073d20 2024-05-08T23:24:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.439+0000 7fa3b77fe700 1 -- 172.21.15.16:0/1064496830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa3c0007500 con 0x7fa3c8073d20 2024-05-08T23:24:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.440+0000 7fa3b77fe700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa3b00a1290 0x7fa3b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.440+0000 7fa3b77fe700 1 -- 172.21.15.16:0/1064496830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa3c00cca90 con 0x7fa3c8073d20 2024-05-08T23:24:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.440+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa3b00a1290 0x7fa3b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.441+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa3b00a1290 0x7fa3b00a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005950 tx=0x7fa3b800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.441+0000 7fa3d8dceb80 1 Processor -- start 2024-05-08T23:24:42.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 -- start start 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3330ae80 0x556f3330b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f333053d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f33305910 0x556f33305d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f3325c1d0 con 0x556f3330b7a0 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33259b90 con 0x556f33305910 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33255480 con 0x556f3330ae80 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f333053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c659c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3330ae80 0x556f3330b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f333053d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60948/0 (socket says 172.21.15.16:60948) 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 -- 172.21.15.16:0/2971778215 learned_addr learned my addr 172.21.15.16:0/2971778215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f33305910 0x556f33305d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 -- 172.21.15.16:0/2971778215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3330ae80 msgr2=0x556f3330b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3330ae80 0x556f3330b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 -- 172.21.15.16:0/2971778215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f33305910 msgr2=0x556f33305d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f33305910 0x556f33305d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.442+0000 7fa3c5d9b700 1 -- 172.21.15.16:0/2971778215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f330d6430 con 0x556f3330b7a0 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.443+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f333053d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8067dd0 tx=0x7fa3bc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758830772a2ce9d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.443+0000 7fa3b4ff9700 1 -- 172.21.15.16:0/2971778215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc01d070 con 0x556f3330b7a0 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.443+0000 7fa3b4ff9700 1 -- 172.21.15.16:0/2971778215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3bc01e040 con 0x556f3330b7a0 2024-05-08T23:24:42.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.443+0000 7fa3b4ff9700 1 -- 172.21.15.16:0/2971778215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc004760 con 0x556f3330b7a0 2024-05-08T23:24:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.443+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/2971778215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 msgr2=0x556f333053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.443+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f333053d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8067dd0 tx=0x7fa3bc005040 comp rx=0 tx=0).stop 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/2971778215 shutdown_connections 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f33305910 0x556f33305d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3330ae80 0x556f3330b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/2971778215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f333053d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/2971778215 >> 172.21.15.16:0/2971778215 conn(0x556f33263290 msgr2=0x556f332ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/2971778215 shutdown_connections 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/2971778215 wait complete. 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- start start 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 Processor -- start 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33311770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f3330ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f331f15e0 con 0x556f3330b7a0 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f331f1280 con 0x556f3330ae80 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.444+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f330d8720 con 0x556f33305910 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3d8dceb80 10 client.0 _ll_register_callbacks cb 0x556f331fabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60964/0 (socket says 172.21.15.16:60964) 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1320346905 learned_addr learned my addr 172.21.15.16:0/1320346905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33311770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f3330ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1320346905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 msgr2=0x556f33311770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33311770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 msgr2=0x556f3330ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f3330ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1320346905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c810e670 con 0x556f3330b7a0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33311770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330f210 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00090a0 tx=0x7fa3c0099550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6064f0308417eaa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3adffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3adffb700 1 -- 172.21.15.16:0/1320346905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00b2250 con 0x556f3330b7a0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3adffb700 1 -- 172.21.15.16:0/1320346905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa3c0099ff0 con 0x556f3330b7a0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.445+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f331ae2f0 con 0x556f3330b7a0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3adffb700 1 -- 172.21.15.16:0/1320346905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c00998f0 con 0x556f3330b7a0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 10 client.16710 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f33082be0 con 0x556f3330b7a0 2024-05-08T23:24:42.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3adffb700 1 -- 172.21.15.16:0/1320346905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556f331ae2f0 con 0x556f3330b7a0 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3adffb700 1 -- 172.21.15.16:0/1320346905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa3c0098de0 con 0x556f3330b7a0 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 10 client.16710 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 20 client.16710 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f3331f170 0x556f33321590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 4 client.16710 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f332d4890 con 0x556f3331f170 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.446+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f3331f170 0x556f33321590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.447+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f3331f170 0x556f33321590 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f2878f537e41d server_cookie=a57f4b3b7a4de8a6 in_seq=0 out_seq=0 2024-05-08T23:24:42.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.447+0000 7fa3adffb700 10 client.16710 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.447+0000 7fa3adffb700 1 -- 172.21.15.16:0/1320346905 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f332d4890 con 0x556f3331f170 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3adffb700 10 client.16710 handle_command_reply: tid=0 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 client.16710 shutdown 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 msgr2=0x556f3330f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330f210 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00090a0 tx=0x7fa3c0099550 comp rx=0 tx=0).stop 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 shutdown_connections 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330f210 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33311770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f3331f170 0x556f33321590 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1320346905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f3330ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3adffb700 0 client.16710 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 >> 172.21.15.16:0/1320346905 conn(0x556f33263290 msgr2=0x556f332ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 shutdown_connections 2024-05-08T23:24:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1320346905 wait complete. 2024-05-08T23:24:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.448+0000 7fa3d8dceb80 20 client.16710 trim_cache size 0 max 16384 2024-05-08T23:24:42.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:42 smithi155 ceph-mon[47105]: pgmap v157: 129 pgs: 129 active+clean; 5.1 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 Processor -- start 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 -- start start 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f332ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33309c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33311c50 con 0x556f3330b7a0 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f332fce40 con 0x556f3330ae80 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33301ed0 con 0x556f33305910 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60978/0 (socket says 172.21.15.16:60978) 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/286564247 learned_addr learned my addr 172.21.15.16:0/286564247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c659c700 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f332ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33309c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/286564247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 msgr2=0x556f332ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f332ffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/286564247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 msgr2=0x556f33309c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33309c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/286564247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f3321b5e0 con 0x556f3330b7a0 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33309c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.467+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330a140 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00995f0 tx=0x7fa3c000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db453debd0c701e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3b57fa700 1 -- 172.21.15.16:0/286564247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00b2250 con 0x556f3330b7a0 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3b57fa700 1 -- 172.21.15.16:0/286564247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3c0099df0 con 0x556f3330b7a0 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3b57fa700 1 -- 172.21.15.16:0/286564247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c009d720 con 0x556f3330b7a0 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/286564247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 msgr2=0x556f3330a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330a140 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00995f0 tx=0x7fa3c000c800 comp rx=0 tx=0).stop 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/286564247 shutdown_connections 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f3330a140 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f332ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/286564247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33309c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/286564247 >> 172.21.15.16:0/286564247 conn(0x556f33263290 msgr2=0x556f3330d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/286564247 shutdown_connections 2024-05-08T23:24:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.468+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/286564247 wait complete. 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 -- start start 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 Processor -- start 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33307610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33307b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f33308090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33375700 con 0x556f3330b7a0 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33375870 con 0x556f3330ae80 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f33375a30 con 0x556f33305910 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3d8dceb80 10 client.0 _ll_register_callbacks cb 0x556f331fabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f33308090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f33308090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60988/0 (socket says 172.21.15.16:60988) 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c5d9b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33307b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1444830091 learned_addr learned my addr 172.21.15.16:0/1444830091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33307610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1444830091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 msgr2=0x556f33307610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.469+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33307610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 msgr2=0x556f33307b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33307b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3c6d9d700 1 -- 172.21.15.16:0/1444830091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f33082be0 con 0x556f3330b7a0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3c5d9b700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33307b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3c6d9d700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f33308090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0096360 tx=0x7fa3c0099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1770d0488131a827 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3adffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3adffb700 1 -- 172.21.15.16:0/1444830091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0095c30 con 0x556f3330b7a0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3adffb700 1 -- 172.21.15.16:0/1444830091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa3c0099ff0 con 0x556f3330b7a0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3bc01e040 con 0x556f3330b7a0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3adffb700 1 -- 172.21.15.16:0/1444830091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c00a1ce0 con 0x556f3330b7a0 2024-05-08T23:24:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 10 client.16716 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f330dad50 con 0x556f3330b7a0 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3adffb700 1 -- 172.21.15.16:0/1444830091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3bc01e040 con 0x556f3330b7a0 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3adffb700 1 -- 172.21.15.16:0/1444830091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa3c00989a0 con 0x556f3330b7a0 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 10 client.16716 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 20 client.16716 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f33376c90 0x556f33379080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 4 client.16716 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.470+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556f3324df40 con 0x556f33376c90 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.471+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f33376c90 0x556f33379080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.471+0000 7fa3c659c700 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f33376c90 0x556f33379080 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309d22b3f62ed932 server_cookie=66e6e6b9e6d19190 in_seq=0 out_seq=0 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.471+0000 7fa3adffb700 10 client.16716 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.472+0000 7fa3adffb700 1 -- 172.21.15.16:0/1444830091 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556f3324df40 con 0x556f33376c90 2024-05-08T23:24:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.472+0000 7fa3adffb700 10 client.16716 handle_command_reply: tid=0 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.472+0000 7fa3d8dceb80 1 client.16716 shutdown 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.472+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 msgr2=0x556f33308090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.472+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f33308090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0096360 tx=0x7fa3c0099300 comp rx=0 tx=0).stop 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 shutdown_connections 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f3330b7a0 0x556f33308090 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f33305910 0x556f33307610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f33376c90 0x556f33379080 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 --2- 172.21.15.16:0/1444830091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f3330ae80 0x556f33307b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 >> 172.21.15.16:0/1444830091 conn(0x556f33263290 msgr2=0x556f33309c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3adffb700 0 client.16716 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 shutdown_connections 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 1 -- 172.21.15.16:0/1444830091 wait complete. 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3d8dceb80 20 client.16716 trim_cache size 0 max 16384 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:24:42.482 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e677fe45-0f02-48ee-95ac-b56304f2252e", 2024-05-08T23:24:42.483 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:24:42.483 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:24:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa3b00a1290 msgr2=0x7fa3b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa3b00a1290 0x7fa3b00a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005950 tx=0x7fa3b800b410 comp rx=0 tx=0).stop 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 msgr2=0x7fa3c81a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 0x7fa3c81a5570 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ef30 tx=0x7fa3c000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 shutdown_connections 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa3b00a1290 0x7fa3b00a3750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8073d20 0x7fa3c81a5570 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3c8101160 0x7fa3c80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 --2- 172.21.15.16:0/1064496830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3c8106280 0x7fa3c80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.473+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 >> 172.21.15.16:0/1064496830 conn(0x7fa3c80fc9e0 msgr2=0x7fa3c80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.474+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 shutdown_connections 2024-05-08T23:24:42.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.474+0000 7fa3ccc14700 1 -- 172.21.15.16:0/1064496830 wait complete. 2024-05-08T23:24:42.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 -- 172.21.15.16:0/1922278249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c106280 msgr2=0x7fbd3c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 --2- 172.21.15.16:0/1922278249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c106280 0x7fbd3c10de00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400b0a0 tx=0x7fbd3400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 -- 172.21.15.16:0/1922278249 shutdown_connections 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 --2- 172.21.15.16:0/1922278249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c106280 0x7fbd3c10de00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 --2- 172.21.15.16:0/1922278249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd3c101160 0x7fbd3c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 --2- 172.21.15.16:0/1922278249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c101b10 0x7fbd3c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 -- 172.21.15.16:0/1922278249 >> 172.21.15.16:0/1922278249 conn(0x7fbd3c0fc9e0 msgr2=0x7fbd3c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 -- 172.21.15.16:0/1922278249 shutdown_connections 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.626+0000 7fbd41a95700 1 -- 172.21.15.16:0/1922278249 wait complete. 2024-05-08T23:24:42.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.627+0000 7fbd41a95700 1 Processor -- start 2024-05-08T23:24:42.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.627+0000 7fbd41a95700 1 -- start start 2024-05-08T23:24:42.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd41a95700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd3c101160 0x7fbd3c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd41a95700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c106280 0x7fbd3c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd41a95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 0x7fbd3c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd41a95700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19c7e0 con 0x7fbd3c19bd60 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd41a95700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19c950 con 0x7fbd3c101160 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd41a95700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c19cb10 con 0x7fbd3c106280 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 0x7fbd3c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 0x7fbd3c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32774/0 (socket says 172.21.15.16:32774) 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 -- 172.21.15.16:0/2518866606 learned_addr learned my addr 172.21.15.16:0/2518866606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd3c101160 0x7fbd3c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3affd700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c106280 0x7fbd3c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 -- 172.21.15.16:0/2518866606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c106280 msgr2=0x7fbd3c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c106280 0x7fbd3c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 -- 172.21.15.16:0/2518866606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd3c101160 msgr2=0x7fbd3c197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd3c101160 0x7fbd3c197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 -- 172.21.15.16:0/2518866606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd34009d00 con 0x7fbd3c19bd60 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3affd700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c106280 0x7fbd3c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 0x7fbd3c19c1e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbd340094e0 tx=0x7fbd34009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.628+0000 7fbd38ff9700 1 -- 172.21.15.16:0/2518866606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400e040 con 0x7fbd3c19bd60 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.629+0000 7fbd38ff9700 1 -- 172.21.15.16:0/2518866606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd34007e20 con 0x7fbd3c19bd60 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.629+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c1a4010 con 0x7fbd3c19bd60 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.629+0000 7fbd38ff9700 1 -- 172.21.15.16:0/2518866606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34005040 con 0x7fbd3c19bd60 2024-05-08T23:24:42.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.629+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c1a44e0 con 0x7fbd3c19bd60 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.631+0000 7fbd38ff9700 1 -- 172.21.15.16:0/2518866606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbd3401b450 con 0x7fbd3c19bd60 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.632+0000 7fbd38ff9700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbd280a1310 0x7fbd280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.632+0000 7fbd38ff9700 1 -- 172.21.15.16:0/2518866606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fbd340ce2d0 con 0x7fbd3c19bd60 2024-05-08T23:24:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.632+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbd280a1310 0x7fbd280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.632+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbd280a1310 0x7fbd280a37d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400ac50 tx=0x7fbd2400a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.632+0000 7fbd4dc4fb80 1 Processor -- start 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 -- start start 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 0x55eb99e22100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99d74150 con 0x55eb99e21ca0 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99d71b10 con 0x55eb99e27b10 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99d6d400 con 0x55eb99e271f0 2024-05-08T23:24:42.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 0x55eb99e22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 0x55eb99e22100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32788/0 (socket says 172.21.15.16:32788) 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 -- 172.21.15.16:0/342150054 learned_addr learned my addr 172.21.15.16:0/342150054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3affd700 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 -- 172.21.15.16:0/342150054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 msgr2=0x55eb99e275d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e275d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 -- 172.21.15.16:0/342150054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 msgr2=0x55eb99e21760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e21760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 -- 172.21.15.16:0/342150054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb99bee3b0 con 0x55eb99e21ca0 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.633+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 0x55eb99e22100 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbd340a2ff0 tx=0x7fbd34007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dfbe01a4e9b5c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd31ffb700 1 -- 172.21.15.16:0/342150054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3409b130 con 0x55eb99e21ca0 2024-05-08T23:24:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd31ffb700 1 -- 172.21.15.16:0/342150054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd3409d070 con 0x55eb99e21ca0 2024-05-08T23:24:42.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd31ffb700 1 -- 172.21.15.16:0/342150054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34097840 con 0x55eb99e21ca0 2024-05-08T23:24:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/342150054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 msgr2=0x55eb99e22100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 0x55eb99e22100 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbd340a2ff0 tx=0x7fbd34007f90 comp rx=0 tx=0).stop 2024-05-08T23:24:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/342150054 shutdown_connections 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e21ca0 0x55eb99e22100 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e275d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/342150054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e21760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.634+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/342150054 >> 172.21.15.16:0/342150054 conn(0x55eb99d7b210 msgr2=0x55eb99e17df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/342150054 shutdown_connections 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/342150054 wait complete. 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 -- start start 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 Processor -- start 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e2b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e27b10 0x55eb99e2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.635+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99d09560 con 0x55eb99e271f0 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99d09200 con 0x55eb99e21ca0 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99bf06a0 con 0x55eb99e27b10 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd4dc4fb80 10 client.0 _ll_register_callbacks cb 0x55eb99d12b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e2b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e2b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32802/0 (socket says 172.21.15.16:32802) 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 -- 172.21.15.16:0/266353340 learned_addr learned my addr 172.21.15.16:0/266353340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e27b10 0x55eb99e2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 -- 172.21.15.16:0/266353340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e27b10 msgr2=0x55eb99e2b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e27b10 0x55eb99e2b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 -- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 msgr2=0x55eb99e2af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e2af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 -- 172.21.15.16:0/266353340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c1a44e0 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e27b10 0x55eb99e2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd3affd700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e2b460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c003960 tx=0x7fbd2c003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d01d79573e6d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd1affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd1affd700 1 -- 172.21.15.16:0/266353340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c009c50 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd1affd700 1 -- 172.21.15.16:0/266353340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fbd2c009db0 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.636+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb99cc6270 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd1affd700 1 -- 172.21.15.16:0/266353340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2c01f550 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 10 client.16728 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55eb99b9ab60 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd1affd700 1 -- 172.21.15.16:0/266353340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eb99cc6270 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd1affd700 1 -- 172.21.15.16:0/266353340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd2c01b470 con 0x55eb99e271f0 2024-05-08T23:24:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 10 client.16728 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 20 client.16728 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e3b570 0x55eb99e3d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 4 client.16728 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb99dec810 con 0x55eb99e3b570 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.637+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e3b570 0x55eb99e3d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.638+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e3b570 0x55eb99e3d990 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b99c5d44ad1b84 server_cookie=cb4091e2cb5473f2 in_seq=0 out_seq=0 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.638+0000 7fbd1affd700 10 client.16728 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.638+0000 7fbd1affd700 1 -- 172.21.15.16:0/266353340 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb99dec810 con 0x55eb99e3b570 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.638+0000 7fbd1affd700 10 client.16728 handle_command_reply: tid=0 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.638+0000 7fbd4dc4fb80 1 client.16728 shutdown 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 msgr2=0x55eb99e2b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e2b460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c003960 tx=0x7fbd2c003c70 comp rx=0 tx=0).stop 2024-05-08T23:24:42.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 shutdown_connections 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e27b10 0x55eb99e2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e3b570 0x55eb99e3d990 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/266353340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e2b460 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd1affd700 0 client.16728 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 >> 172.21.15.16:0/266353340 conn(0x55eb99d7b210 msgr2=0x55eb99e175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 shutdown_connections 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/266353340 wait complete. 2024-05-08T23:24:42.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.639+0000 7fbd4dc4fb80 20 client.16728 trim_cache size 0 max 16384 2024-05-08T23:24:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 Processor -- start 2024-05-08T23:24:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 -- start start 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 0x55eb99e17d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99e38ba0 con 0x55eb99e27b10 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99e2c590 con 0x55eb99e21ca0 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.657+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99e18910 con 0x55eb99e271f0 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 0x55eb99e17d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 0x55eb99e17d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32806/0 (socket says 172.21.15.16:32806) 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 -- 172.21.15.16:0/1524603275 learned_addr learned my addr 172.21.15.16:0/1524603275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3affd700 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 -- 172.21.15.16:0/1524603275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 msgr2=0x55eb99e374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 -- 172.21.15.16:0/1524603275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 msgr2=0x55eb99e22080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e22080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 -- 172.21.15.16:0/1524603275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb99d33560 con 0x55eb99e27b10 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e22080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 0x55eb99e17d20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd3409b0c0 tx=0x7fbd3400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af44dcb466daeaa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd327fc700 1 -- 172.21.15.16:0/1524603275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3409b130 con 0x55eb99e27b10 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd327fc700 1 -- 172.21.15.16:0/1524603275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd340a3520 con 0x55eb99e27b10 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.658+0000 7fbd327fc700 1 -- 172.21.15.16:0/1524603275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a1290 con 0x55eb99e27b10 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/1524603275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 msgr2=0x55eb99e17d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 0x55eb99e17d20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd3409b0c0 tx=0x7fbd3400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/1524603275 shutdown_connections 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e27b10 0x55eb99e17d20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e21ca0 0x55eb99e22080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/1524603275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e271f0 0x55eb99e374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/1524603275 >> 172.21.15.16:0/1524603275 conn(0x55eb99d7b210 msgr2=0x55eb99e14210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/1524603275 shutdown_connections 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/1524603275 wait complete. 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.659+0000 7fbd4dc4fb80 1 -- start start 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 Processor -- start 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e21ca0 0x55eb99e32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99e30c00 con 0x55eb99e271f0 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99e30d70 con 0x55eb99e27b10 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb99e30f30 con 0x55eb99e21ca0 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd4dc4fb80 10 client.0 _ll_register_callbacks cb 0x55eb99d12b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:24:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e328a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32820/0 (socket says 172.21.15.16:32820) 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 -- 172.21.15.16:0/3300125305 learned_addr learned my addr 172.21.15.16:0/3300125305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e21ca0 0x55eb99e32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3bfff700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 -- 172.21.15.16:0/3300125305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e21ca0 msgr2=0x55eb99e32360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e21ca0 0x55eb99e32360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 -- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 msgr2=0x55eb99e304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.660+0000 7fbd3affd700 1 -- 172.21.15.16:0/3300125305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb99bee3b0 con 0x55eb99e271f0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd3affd700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e328a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c000c00 tx=0x7fbd2c01cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee90ce910f93afc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd1affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd1affd700 1 -- 172.21.15.16:0/3300125305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c009d70 con 0x55eb99e271f0 2024-05-08T23:24:42.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd1affd700 1 -- 172.21.15.16:0/3300125305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fbd2c02c070 con 0x55eb99e271f0 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd3409d070 con 0x55eb99e271f0 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd1affd700 1 -- 172.21.15.16:0/3300125305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd2c009ed0 con 0x55eb99e271f0 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 10 client.16734 fetch_fsmap learned FSMap version 25 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55eb99bf2cd0 con 0x55eb99e271f0 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd1affd700 1 -- 172.21.15.16:0/3300125305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbd3409d070 con 0x55eb99e271f0 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd1affd700 1 -- 172.21.15.16:0/3300125305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd2c004b10 con 0x55eb99e271f0 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 10 client.16734 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 20 client.16734 populate_metadata read hostname 'smithi016' 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e91c90 0x55eb99e940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 4 client.16734 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.661+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb99d65ec0 con 0x55eb99e91c90 2024-05-08T23:24:42.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.662+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e91c90 0x55eb99e940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.662+0000 7fbd1affd700 10 client.16734 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.662+0000 7fbd3b7fe700 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e91c90 0x55eb99e940b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e028aef31d4ca34f server_cookie=21d3cd71295adba9 in_seq=0 out_seq=0 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd1affd700 1 -- 172.21.15.16:0/3300125305 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eb99d65ec0 con 0x55eb99e91c90 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd1affd700 10 client.16734 handle_command_reply: tid=0 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 client.16734 shutdown 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 msgr2=0x55eb99e328a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e328a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c000c00 tx=0x7fbd2c01cee0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 shutdown_connections 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55eb99e91c90 0x55eb99e940b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55eb99e27b10 0x55eb99e304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb99e21ca0 0x55eb99e32360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd1affd700 0 client.16734 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 --2- 172.21.15.16:0/3300125305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb99e271f0 0x55eb99e328a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 >> 172.21.15.16:0/3300125305 conn(0x55eb99d7b210 msgr2=0x55eb99e374c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 shutdown_connections 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 1 -- 172.21.15.16:0/3300125305 wait complete. 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.663+0000 7fbd4dc4fb80 20 client.16734 trim_cache size 0 max 16384 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stdout: "e677fe45-0f02-48ee-95ac-b56304f2252e": { 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:24:42.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e677fe45-0f02-48ee-95ac-b56304f2252e", 2024-05-08T23:24:42.670 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:24:42.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:24:42.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:24:42.670 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:24:42.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbd280a1310 msgr2=0x7fbd280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbd280a1310 0x7fbd280a37d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400ac50 tx=0x7fbd2400a380 comp rx=0 tx=0).stop 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 msgr2=0x7fbd3c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 0x7fbd3c19c1e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbd340094e0 tx=0x7fbd34009510 comp rx=0 tx=0).stop 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 shutdown_connections 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbd280a1310 0x7fbd280a37d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3c19bd60 0x7fbd3c19c1e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd3c101160 0x7fbd3c197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 --2- 172.21.15.16:0/2518866606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd3c106280 0x7fbd3c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 >> 172.21.15.16:0/2518866606 conn(0x7fbd3c0fc9e0 msgr2=0x7fbd3c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 shutdown_connections 2024-05-08T23:24:42.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:24:42.664+0000 7fbd41a95700 1 -- 172.21.15.16:0/2518866606 wait complete. 2024-05-08T23:24:42.678 INFO:tasks.cephfs.filesystem:scrub status for tag:e677fe45-0f02-48ee-95ac-b56304f2252e - {'path': '/', 'tag': 'e677fe45-0f02-48ee-95ac-b56304f2252e', 'options': 'recursive,force'} 2024-05-08T23:24:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:43 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:24:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:43 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:43 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:24:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:43 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:43 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:24:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:43 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:43 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:24:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:43 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:43.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:43 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:24:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:43 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:43 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:24:43.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:43 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:24:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:44 smithi155 ceph-mon[47105]: pgmap v158: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:44.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:44 smithi016 ceph-mon[42541]: pgmap v158: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:44.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:44 smithi189 ceph-mon[46919]: pgmap v158: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-05-08T23:24:45.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:45 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:24:45.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:45 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:24:45.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:45 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:127/600: open fd = 3 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:128/600: open fd = 3 2024-05-08T23:24:46.314 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:129/600: open fd = 3 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:130/600: open fd = 3 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.315 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:131/600: open fd = 3 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:132/600: open fd = 3 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:133/600: open fd = 3 2024-05-08T23:24:46.316 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:134/600: open fd = 3 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:135/600: open fd = 3 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:136/600: open fd = 3 2024-05-08T23:24:46.317 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.318 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.318 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.318 INFO:tasks.workunit.client.0.smithi016.stdout:137/600: open fd = 3 2024-05-08T23:24:46.318 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.318 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.318 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:138/600: open fd = 3 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:139/600: open fd = 3 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:140/600: open fd = 3 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:141/600: open fd = 3 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.319 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:142/600: open fd = 3 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:143/600: open fd = 3 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:144/600: open fd = 3 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:145/600: open fd = 3 2024-05-08T23:24:46.320 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:146/600: open fd = 3 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:147/600: open fd = 3 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:148/600: open fd = 3 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.321 INFO:tasks.workunit.client.0.smithi016.stdout:149/600: open fd = 3 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:150/600: open fd = 3 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:151/600: open fd = 3 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:152/600: open fd = 3 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.322 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:153/600: open fd = 3 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:154/600: open fd = 3 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:155/600: open fd = 3 2024-05-08T23:24:46.323 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:156/600: open fd = 3 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:157/600: open fd = 3 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:158/600: open fd = 3 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.324 INFO:tasks.workunit.client.0.smithi016.stdout:159/600: open fd = 3 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:160/600: open fd = 3 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:161/600: open fd = 3 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:162/600: open fd = 3 2024-05-08T23:24:46.325 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:163/600: open fd = 3 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:164/600: open fd = 3 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:165/600: open fd = 3 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.326 INFO:tasks.workunit.client.0.smithi016.stdout:166/600: open fd = 3 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:167/600: open fd = 3 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.327 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:168/600: open fd = 3 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:169/600: open fd = 3 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.328 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:170/600: open fd = 3 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:171/600: open fd = 3 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:172/600: open fd = 3 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:173/600: open fd = 3 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.329 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:174/600: open fd = 3 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:175/600: open fd = 3 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:176/600: open fd = 3 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.330 INFO:tasks.workunit.client.0.smithi016.stdout:177/600: open fd = 3 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:178/600: open fd = 3 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:179/600: open fd = 3 2024-05-08T23:24:46.331 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:180/600: open fd = 3 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:181/600: open fd = 3 2024-05-08T23:24:46.332 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:182/600: open fd = 3 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:183/600: open fd = 3 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:184/600: open fd = 3 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.333 INFO:tasks.workunit.client.0.smithi016.stdout:185/600: open fd = 3 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:186/600: open fd = 3 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:187/600: open fd = 3 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:188/600: open fd = 3 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.334 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:24:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:24:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:24:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:24:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:24:46.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:46 smithi155 ceph-mon[47105]: pgmap v159: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:46.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:24:46.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:24:46.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:24:46.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:24:46.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:46 smithi016 ceph-mon[42541]: pgmap v159: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:46.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:24:46.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:24:46.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:24:46.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:24:46.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:46 smithi189 ceph-mon[46919]: pgmap v159: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:48.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:48 smithi016 ceph-mon[42541]: pgmap v160: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:48.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:48 smithi189 ceph-mon[46919]: pgmap v160: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:48.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:48 smithi155 ceph-mon[47105]: pgmap v160: 129 pgs: 129 active+clean; 5.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:50.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:50 smithi016 ceph-mon[42541]: pgmap v161: 129 pgs: 129 active+clean; 5.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-08T23:24:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:50 smithi189 ceph-mon[46919]: pgmap v161: 129 pgs: 129 active+clean; 5.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-08T23:24:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:50 smithi155 ceph-mon[47105]: pgmap v161: 129 pgs: 129 active+clean; 5.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-08T23:24:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:52 smithi016 ceph-mon[42541]: pgmap v162: 129 pgs: 129 active+clean; 5.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:52.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:52 smithi189 ceph-mon[46919]: pgmap v162: 129 pgs: 129 active+clean; 5.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:52 smithi155 ceph-mon[47105]: pgmap v162: 129 pgs: 129 active+clean; 5.3 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:24:54.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:54 smithi016 ceph-mon[42541]: pgmap v163: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-08T23:24:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:54 smithi189 ceph-mon[46919]: pgmap v163: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-08T23:24:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:54 smithi155 ceph-mon[47105]: pgmap v163: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-05-08T23:24:56.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:56 smithi016 ceph-mon[42541]: pgmap v164: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-08T23:24:56.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:56 smithi189 ceph-mon[46919]: pgmap v164: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-08T23:24:56.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:56 smithi155 ceph-mon[47105]: pgmap v164: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-08T23:24:58.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:24:58 smithi016 ceph-mon[42541]: pgmap v165: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-08T23:24:58.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:24:58 smithi189 ceph-mon[46919]: pgmap v165: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-08T23:24:58.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:24:58 smithi155 ceph-mon[47105]: pgmap v165: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-05-08T23:25:00.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:00 smithi016 ceph-mon[42541]: pgmap v166: 129 pgs: 129 active+clean; 5.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-08T23:25:00.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:00 smithi189 ceph-mon[46919]: pgmap v166: 129 pgs: 129 active+clean; 5.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-08T23:25:00.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:00 smithi155 ceph-mon[47105]: pgmap v166: 129 pgs: 129 active+clean; 5.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-05-08T23:25:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:02 smithi016 ceph-mon[42541]: pgmap v167: 129 pgs: 129 active+clean; 5.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:25:02.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:02 smithi189 ceph-mon[46919]: pgmap v167: 129 pgs: 129 active+clean; 5.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:25:02.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:02 smithi155 ceph-mon[47105]: pgmap v167: 129 pgs: 129 active+clean; 5.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-05-08T23:25:04.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:04 smithi016 ceph-mon[42541]: pgmap v168: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-05-08T23:25:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:04 smithi189 ceph-mon[46919]: pgmap v168: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-05-08T23:25:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:04 smithi155 ceph-mon[47105]: pgmap v168: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 61 KiB/s wr, 30 op/s 2024-05-08T23:25:06.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:06 smithi016 ceph-mon[42541]: pgmap v169: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-05-08T23:25:06.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:06 smithi189 ceph-mon[46919]: pgmap v169: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-05-08T23:25:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:06 smithi155 ceph-mon[47105]: pgmap v169: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-05-08T23:25:08.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:08 smithi016 ceph-mon[42541]: pgmap v170: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-05-08T23:25:08.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:08 smithi189 ceph-mon[46919]: pgmap v170: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-05-08T23:25:08.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:08 smithi155 ceph-mon[47105]: pgmap v170: 129 pgs: 129 active+clean; 5.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-05-08T23:25:10.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:10 smithi016 ceph-mon[42541]: pgmap v171: 129 pgs: 129 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 63 KiB/s wr, 30 op/s 2024-05-08T23:25:10.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:10 smithi189 ceph-mon[46919]: pgmap v171: 129 pgs: 129 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 63 KiB/s wr, 30 op/s 2024-05-08T23:25:10.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:10 smithi155 ceph-mon[47105]: pgmap v171: 129 pgs: 129 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 63 KiB/s wr, 30 op/s 2024-05-08T23:25:12.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:12 smithi016 ceph-mon[42541]: pgmap v172: 129 pgs: 129 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 25 op/s 2024-05-08T23:25:12.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:12 smithi189 ceph-mon[46919]: pgmap v172: 129 pgs: 129 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 25 op/s 2024-05-08T23:25:12.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:12 smithi155 ceph-mon[47105]: pgmap v172: 129 pgs: 129 active+clean; 5.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 25 op/s 2024-05-08T23:25:12.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:25:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3790575145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c0682e0 msgr2=0x7f0a6c0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3790575145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c0682e0 0x7f0a6c0686c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0a540099c0 tx=0x7f0a54009cd0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3790575145 shutdown_connections 2024-05-08T23:25:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3790575145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6c106290 0x7f0a6c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3790575145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c0682e0 0x7f0a6c0686c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3790575145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a6c068c90 0x7f0a6c105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3790575145 >> 172.21.15.16:0/3790575145 conn(0x7f0a6c076f40 msgr2=0x7f0a6c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3790575145 shutdown_connections 2024-05-08T23:25:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.827+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3790575145 wait complete. 2024-05-08T23:25:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.828+0000 7f0a72b4d700 1 Processor -- start 2024-05-08T23:25:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.828+0000 7f0a72b4d700 1 -- start start 2024-05-08T23:25:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.828+0000 7f0a72b4d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a6c0682e0 0x7f0a6c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a72b4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 0x7f0a6c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a72b4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6c19bbe0 0x7f0a6c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a72b4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c19c660 con 0x7f0a6c106290 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a72b4d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c19c7d0 con 0x7f0a6c19bbe0 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a72b4d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c19c990 con 0x7f0a6c0682e0 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 0x7f0a6c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 0x7f0a6c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45432/0 (socket says 172.21.15.16:45432) 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3168384131 learned_addr learned my addr 172.21.15.16:0/3168384131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a710ea700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6c19bbe0 0x7f0a6c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a6c0682e0 0x7f0a6c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3168384131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a6c0682e0 msgr2=0x7f0a6c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a6c0682e0 0x7f0a6c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3168384131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6c19bbe0 msgr2=0x7f0a6c19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6c19bbe0 0x7f0a6c19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3168384131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540095f0 con 0x7f0a6c106290 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 0x7f0a6c197840 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00d950 tx=0x7f0a5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a69ffb700 1 -- 172.21.15.16:0/3168384131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c004220 con 0x7f0a6c106290 2024-05-08T23:25:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.829+0000 7f0a69ffb700 1 -- 172.21.15.16:0/3168384131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a5c007620 con 0x7f0a6c106290 2024-05-08T23:25:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.830+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c1a3ef0 con 0x7f0a6c106290 2024-05-08T23:25:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.830+0000 7f0a69ffb700 1 -- 172.21.15.16:0/3168384131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c005560 con 0x7f0a6c106290 2024-05-08T23:25:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.830+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1a4370 con 0x7f0a6c106290 2024-05-08T23:25:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.832+0000 7f0a69ffb700 1 -- 172.21.15.16:0/3168384131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0a5c004750 con 0x7f0a6c106290 2024-05-08T23:25:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.833+0000 7f0a69ffb700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a580a1290 0x7f0a580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.833+0000 7f0a69ffb700 1 -- 172.21.15.16:0/3168384131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f0a5c0ccdc0 con 0x7f0a6c106290 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.833+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a580a1290 0x7f0a580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.833+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a580a1290 0x7f0a580a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0a54006010 tx=0x7f0a54019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 Processor -- start 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 -- start start 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bee00 0x55c1655bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165510150 con 0x55c1655bf720 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16550db10 con 0x55c1655bee00 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165509400 con 0x55c1655b9890 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655b9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45434/0 (socket says 172.21.15.16:45434) 2024-05-08T23:25:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 -- 172.21.15.16:0/2404468772 learned_addr learned my addr 172.21.15.16:0/2404468772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a708e9700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bee00 0x55c1655bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a710ea700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 -- 172.21.15.16:0/2404468772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 msgr2=0x55c1655b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 -- 172.21.15.16:0/2404468772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bee00 msgr2=0x55c1655bf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bee00 0x55c1655bf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a6bfff700 1 -- 172.21.15.16:0/2404468772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c16538a3b0 con 0x55c1655bf720 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a710ea700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.834+0000 7f0a708e9700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bee00 0x55c1655bf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.835+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655b9350 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c005cd0 tx=0x7f0a5c09a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b769204fd0681b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.835+0000 7f0a62ffd700 1 -- 172.21.15.16:0/2404468772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c09d960 con 0x55c1655bf720 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.835+0000 7f0a62ffd700 1 -- 172.21.15.16:0/2404468772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a5c0a1fb0 con 0x55c1655bf720 2024-05-08T23:25:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.835+0000 7f0a62ffd700 1 -- 172.21.15.16:0/2404468772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c099220 con 0x55c1655bf720 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.835+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/2404468772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 msgr2=0x55c1655b9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.835+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655b9350 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c005cd0 tx=0x7f0a5c09a2b0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/2404468772 shutdown_connections 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bee00 0x55c1655bf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/2404468772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655b9350 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/2404468772 >> 172.21.15.16:0/2404468772 conn(0x55c165517210 msgr2=0x55c1655b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/2404468772 shutdown_connections 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/2404468772 wait complete. 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 -- start start 2024-05-08T23:25:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 Processor -- start 2024-05-08T23:25:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.836+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 0x55c1655c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bf720 0x55c1655c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1654a5560 con 0x55c1655bee00 2024-05-08T23:25:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1654a5200 con 0x55c1655bf720 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16538c6a0 con 0x55c1655b9890 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a7ed07b80 10 client.0 _ll_register_callbacks cb 0x55c1654aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 0x55c1655c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 0x55c1655c2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45442/0 (socket says 172.21.15.16:45442) 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 -- 172.21.15.16:0/1425382067 learned_addr learned my addr 172.21.15.16:0/1425382067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a710ea700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bf720 0x55c1655c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 -- 172.21.15.16:0/1425382067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 msgr2=0x55c1655c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a708e9700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 -- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bf720 msgr2=0x55c1655c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bf720 0x55c1655c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 -- 172.21.15.16:0/1425382067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c1a4370 con 0x55c1655bee00 2024-05-08T23:25:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a708e9700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a710ea700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bf720 0x55c1655c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 0x55c1655c2fd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c099ce0 tx=0x7f0a5c09da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c01b44f57f9fc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a4bfff700 1 -- 172.21.15.16:0/1425382067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0652a0 con 0x55c1655bee00 2024-05-08T23:25:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a4bfff700 1 -- 172.21.15.16:0/1425382067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f0a5c065400 con 0x55c1655bee00 2024-05-08T23:25:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.837+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c165462270 con 0x55c1655bee00 2024-05-08T23:25:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a4bfff700 1 -- 172.21.15.16:0/1425382067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a5c099020 con 0x55c1655bee00 2024-05-08T23:25:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 10 client.16746 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c165336b60 con 0x55c1655bee00 2024-05-08T23:25:12.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a4bfff700 1 -- 172.21.15.16:0/1425382067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c165462270 con 0x55c1655bee00 2024-05-08T23:25:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a4bfff700 1 -- 172.21.15.16:0/1425382067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a5c0a1c40 con 0x55c1655bee00 2024-05-08T23:25:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 10 client.16746 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 20 client.16746 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c1655d2d00 0x55c1655d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 4 client.16746 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c165588810 con 0x55c1655d2d00 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.838+0000 7f0a708e9700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c1655d2d00 0x55c1655d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.839+0000 7f0a708e9700 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c1655d2d00 0x55c1655d5120 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b36524ca9d39c03 server_cookie=82427a982bf58918 in_seq=0 out_seq=0 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.839+0000 7f0a4bfff700 10 client.16746 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.839+0000 7f0a4bfff700 1 -- 172.21.15.16:0/1425382067 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c165588810 con 0x55c1655d2d00 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.839+0000 7f0a4bfff700 10 client.16746 handle_command_reply: tid=0 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.839+0000 7f0a7ed07b80 1 client.16746 shutdown 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 msgr2=0x55c1655c2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 0x55c1655c2fd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c099ce0 tx=0x7f0a5c09da10 comp rx=0 tx=0).stop 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 shutdown_connections 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c1655d2d00 0x55c1655d5120 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655bf720 0x55c1655c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a4bfff700 0 client.16746 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655b9890 0x55c1655c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/1425382067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bee00 0x55c1655c2fd0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 >> 172.21.15.16:0/1425382067 conn(0x55c165517210 msgr2=0x55c1655b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 shutdown_connections 2024-05-08T23:25:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/1425382067 wait complete. 2024-05-08T23:25:12.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.840+0000 7f0a7ed07b80 20 client.16746 trim_cache size 0 max 16384 2024-05-08T23:25:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.858+0000 7f0a7ed07b80 1 Processor -- start 2024-05-08T23:25:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.858+0000 7f0a7ed07b80 1 -- start start 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655d2d00 0x55c1655ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1655cfd70 con 0x55c1655bf720 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1655da2f0 con 0x55c1655b9890 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1655b9690 con 0x55c1655d2d00 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45456/0 (socket says 172.21.15.16:45456) 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3071984414 learned_addr learned my addr 172.21.15.16:0/3071984414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a710ea700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655d2d00 0x55c1655ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3071984414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655d2d00 msgr2=0x55c1655ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655d2d00 0x55c1655ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3071984414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 msgr2=0x55c1655b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 -- 172.21.15.16:0/3071984414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1654cf560 con 0x55c1655bf720 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a710ea700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655d2d00 0x55c1655ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655ca160 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c099830 tx=0x7f0a5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e35b40ab6a9136a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a637fe700 1 -- 172.21.15.16:0/3071984414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0ada00 con 0x55c1655bf720 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a637fe700 1 -- 172.21.15.16:0/3071984414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a5c099900 con 0x55c1655bf720 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.859+0000 7f0a637fe700 1 -- 172.21.15.16:0/3071984414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0a1d80 con 0x55c1655bf720 2024-05-08T23:25:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3071984414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 msgr2=0x55c1655ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655ca160 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c099830 tx=0x7f0a5c000bc0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3071984414 shutdown_connections 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655d2d00 0x55c1655ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3071984414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655bf720 0x55c1655ca160 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3071984414 >> 172.21.15.16:0/3071984414 conn(0x55c165517210 msgr2=0x55c1655c1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3071984414 shutdown_connections 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3071984414 wait complete. 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.860+0000 7f0a7ed07b80 1 -- start start 2024-05-08T23:25:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 Processor -- start 2024-05-08T23:25:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655bf720 0x55c1655c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 0x55c1656281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1655c6f90 con 0x55c1655d2d00 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1656286f0 con 0x55c1655b9890 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165628830 con 0x55c1655bf720 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a7ed07b80 10 client.0 _ll_register_callbacks cb 0x55c1654aeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a710ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 0x55c1656281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a710ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 0x55c1656281b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45472/0 (socket says 172.21.15.16:45472) 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a710ea700 1 -- 172.21.15.16:0/3022324897 learned_addr learned my addr 172.21.15.16:0/3022324897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655bf720 0x55c1655c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a710ea700 1 -- 172.21.15.16:0/3022324897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655bf720 msgr2=0x55c1655c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a710ea700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655bf720 0x55c1655c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.861+0000 7f0a710ea700 1 -- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 msgr2=0x55c1655c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a710ea700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a710ea700 1 -- 172.21.15.16:0/3022324897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c165336b60 con 0x55c1655d2d00 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a6bfff700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655bf720 0x55c1655c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a710ea700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 0x55c1656281b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0a64007f10 tx=0x7f0a640092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0356c51a5f3676 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a4bfff700 1 -- 172.21.15.16:0/3022324897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6400e040 con 0x55c1655d2d00 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a4bfff700 1 -- 172.21.15.16:0/3022324897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f0a64015bb0 con 0x55c1655d2d00 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a5c0a1fb0 con 0x55c1655d2d00 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a4bfff700 1 -- 172.21.15.16:0/3022324897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a64015d70 con 0x55c1655d2d00 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a7ed07b80 10 client.16752 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:12.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c16538ecd0 con 0x55c1655d2d00 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a4bfff700 1 -- 172.21.15.16:0/3022324897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a5c0a1fb0 con 0x55c1655d2d00 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a4bfff700 1 -- 172.21.15.16:0/3022324897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a64017410 con 0x55c1655d2d00 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.862+0000 7f0a7ed07b80 10 client.16752 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a7ed07b80 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a7ed07b80 20 client.16752 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c16562a060 0x55c16562c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a7ed07b80 4 client.16752 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c165501ec0 con 0x55c16562a060 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c16562a060 0x55c16562c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a4bfff700 10 client.16752 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.863+0000 7f0a708e9700 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c16562a060 0x55c16562c480 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2cd3393f0c8f3c server_cookie=b690d4c6e2c21efb in_seq=0 out_seq=0 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a4bfff700 1 -- 172.21.15.16:0/3022324897 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c165501ec0 con 0x55c16562a060 2024-05-08T23:25:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a4bfff700 10 client.16752 handle_command_reply: tid=0 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 client.16752 shutdown 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 msgr2=0x55c1656281b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 0x55c1656281b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0a64007f10 tx=0x7f0a640092f0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 shutdown_connections 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c1655d2d00 0x55c1656281b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c16562a060 0x55c16562c480 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c1655b9890 0x55c1655c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 --2- 172.21.15.16:0/3022324897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1655bf720 0x55c1655c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a4bfff700 0 client.16752 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 >> 172.21.15.16:0/3022324897 conn(0x55c165517210 msgr2=0x55c1655bdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 shutdown_connections 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 1 -- 172.21.15.16:0/3022324897 wait complete. 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.864+0000 7f0a7ed07b80 20 client.16752 trim_cache size 0 max 16384 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:25:12.873 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a580a1290 msgr2=0x7f0a580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a580a1290 0x7f0a580a3750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0a54006010 tx=0x7f0a54019040 comp rx=0 tx=0).stop 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 msgr2=0x7f0a6c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 0x7f0a6c197840 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00d950 tx=0x7f0a5c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 shutdown_connections 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0a580a1290 0x7f0a580a3750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0a6c19bbe0 0x7f0a6c19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a6c0682e0 0x7f0a6c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 --2- 172.21.15.16:0/3168384131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a6c106290 0x7f0a6c197840 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 >> 172.21.15.16:0/3168384131 conn(0x7f0a6c076f40 msgr2=0x7f0a6c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 shutdown_connections 2024-05-08T23:25:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:12.865+0000 7f0a72b4d700 1 -- 172.21.15.16:0/3168384131 wait complete. 2024-05-08T23:25:12.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:25:12.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 -- 172.21.15.16:0/1450863877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171410d770 msgr2=0x7f171410fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 --2- 172.21.15.16:0/1450863877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171410d770 0x7f171410fb60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f170c00b0a0 tx=0x7f170c00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 -- 172.21.15.16:0/1450863877 shutdown_connections 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 --2- 172.21.15.16:0/1450863877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f171410d770 0x7f171410fb60 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 --2- 172.21.15.16:0/1450863877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17140ffbc0 0x7f17140fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 --2- 172.21.15.16:0/1450863877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1714100570 0x7f171410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 -- 172.21.15.16:0/1450863877 >> 172.21.15.16:0/1450863877 conn(0x7f17140fb810 msgr2=0x7f17140fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 -- 172.21.15.16:0/1450863877 shutdown_connections 2024-05-08T23:25:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.015+0000 7f171bb06700 1 -- 172.21.15.16:0/1450863877 wait complete. 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 Processor -- start 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 -- start start 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 0x7f17141995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1714100570 0x7f1714199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f171410d770 0x7f171419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419e470 con 0x7f17140ffbc0 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419e5e0 con 0x7f171410d770 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171bb06700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419e7a0 con 0x7f1714100570 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f17198a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 0x7f17141995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f17198a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 0x7f17141995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45494/0 (socket says 172.21.15.16:45494) 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f17198a2700 1 -- 172.21.15.16:0/3971824403 learned_addr learned my addr 172.21.15.16:0/3971824403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f171a0a3700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f171410d770 0x7f171419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.016+0000 7f17190a1700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1714100570 0x7f1714199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f17198a2700 1 -- 172.21.15.16:0/3971824403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1714100570 msgr2=0x7f1714199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f17198a2700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1714100570 0x7f1714199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f17198a2700 1 -- 172.21.15.16:0/3971824403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f171410d770 msgr2=0x7f171419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f17198a2700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f171410d770 0x7f171419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f17198a2700 1 -- 172.21.15.16:0/3971824403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c009d00 con 0x7f17140ffbc0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f17198a2700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 0x7f17141995a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f171000e9a0 tx=0x7f171000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710009790 con 0x7f17140ffbc0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171419ea80 con 0x7f17140ffbc0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1710010430 con 0x7f17140ffbc0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171000cea0 con 0x7f17140ffbc0 2024-05-08T23:25:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.017+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17141a62c0 con 0x7f17140ffbc0 2024-05-08T23:25:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.018+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1714108900 con 0x7f17140ffbc0 2024-05-08T23:25:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.020+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f17100105a0 con 0x7f17140ffbc0 2024-05-08T23:25:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.020+0000 7f170affd700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f17000a1290 0x7f17000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.020+0000 7f17190a1700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f17000a1290 0x7f17000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.020+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f1710014070 con 0x7f17140ffbc0 2024-05-08T23:25:13.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.021+0000 7f17190a1700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f17000a1290 0x7f17000a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f171419aaa0 tx=0x7f1704009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.023+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1710096510 con 0x7f17140ffbc0 2024-05-08T23:25:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.302+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1714069d30 con 0x7f17140ffbc0 2024-05-08T23:25:13.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.303+0000 7f170affd700 1 -- 172.21.15.16:0/3971824403 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f1710026090 con 0x7f17140ffbc0 2024-05-08T23:25:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:25:13.304 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:25:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f17000a1290 msgr2=0x7f17000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f17000a1290 0x7f17000a3750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f171419aaa0 tx=0x7f1704009260 comp rx=0 tx=0).stop 2024-05-08T23:25:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 msgr2=0x7f17141995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 0x7f17141995a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f171000e9a0 tx=0x7f171000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 shutdown_connections 2024-05-08T23:25:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f17000a1290 0x7f17000a3750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f171410d770 0x7f171419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17140ffbc0 0x7f17141995a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 --2- 172.21.15.16:0/3971824403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1714100570 0x7f1714199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 >> 172.21.15.16:0/3971824403 conn(0x7f17140fb810 msgr2=0x7f17140fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 shutdown_connections 2024-05-08T23:25:13.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.306+0000 7f171bb06700 1 -- 172.21.15.16:0/3971824403 wait complete. 2024-05-08T23:25:13.309 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:25:13.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:25:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.460+0000 7fa634c3e700 1 -- 172.21.15.16:0/1347221761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6301062a0 msgr2=0x7fa63010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.460+0000 7fa634c3e700 1 --2- 172.21.15.16:0/1347221761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6301062a0 0x7fa63010de20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa62800b0a0 tx=0x7fa62800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.460+0000 7fa634c3e700 1 -- 172.21.15.16:0/1347221761 shutdown_connections 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.460+0000 7fa634c3e700 1 --2- 172.21.15.16:0/1347221761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6301062a0 0x7fa63010de20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.460+0000 7fa634c3e700 1 --2- 172.21.15.16:0/1347221761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa630069420 0x7fa630069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.461+0000 7fa634c3e700 1 --2- 172.21.15.16:0/1347221761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa630069dd0 0x7fa630105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.461+0000 7fa634c3e700 1 -- 172.21.15.16:0/1347221761 >> 172.21.15.16:0/1347221761 conn(0x7fa630076ba0 msgr2=0x7fa630076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.461+0000 7fa634c3e700 1 -- 172.21.15.16:0/1347221761 shutdown_connections 2024-05-08T23:25:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.461+0000 7fa634c3e700 1 -- 172.21.15.16:0/1347221761 wait complete. 2024-05-08T23:25:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 Processor -- start 2024-05-08T23:25:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 -- start start 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 0x7fa630197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6301062a0 0x7fa6301979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63019bd40 0x7fa63019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63019c7c0 con 0x7fa630069420 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63019c930 con 0x7fa63019bd40 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa634c3e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63019caf0 con 0x7fa6301062a0 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa62e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 0x7fa630197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa62e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 0x7fa630197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45512/0 (socket says 172.21.15.16:45512) 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa62e59c700 1 -- 172.21.15.16:0/2598649647 learned_addr learned my addr 172.21.15.16:0/2598649647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63019bd40 0x7fa63019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.462+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6301062a0 0x7fa6301979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa62e59c700 1 -- 172.21.15.16:0/2598649647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6301062a0 msgr2=0x7fa6301979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6301062a0 0x7fa6301979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa62e59c700 1 -- 172.21.15.16:0/2598649647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63019bd40 msgr2=0x7fa63019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63019bd40 0x7fa63019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa62e59c700 1 -- 172.21.15.16:0/2598649647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628009d00 con 0x7fa630069420 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 0x7fa630197460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e9a0 tx=0x7fa61800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa6277fe700 1 -- 172.21.15.16:0/2598649647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618009790 con 0x7fa630069420 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa6277fe700 1 -- 172.21.15.16:0/2598649647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61800cbd0 con 0x7fa630069420 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa6277fe700 1 -- 172.21.15.16:0/2598649647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618018680 con 0x7fa630069420 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6301a3ff0 con 0x7fa630069420 2024-05-08T23:25:13.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.463+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6301a4480 con 0x7fa630069420 2024-05-08T23:25:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.465+0000 7fa6277fe700 1 -- 172.21.15.16:0/2598649647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa61800cd40 con 0x7fa630069420 2024-05-08T23:25:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.466+0000 7fa6277fe700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa61c0a1290 0x7fa61c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.466+0000 7fa6277fe700 1 -- 172.21.15.16:0/2598649647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa618014070 con 0x7fa630069420 2024-05-08T23:25:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.466+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa61c0a1290 0x7fa61c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa61c0a1290 0x7fa61c0a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa620005950 tx=0x7fa6200058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 Processor -- start 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 -- start start 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc99e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc998350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc998cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc8ef150 con 0x559fcc99de00 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc8ecb10 con 0x559fcc99e720 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa640df8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc8e8400 con 0x559fcc998890 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa62e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc99e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa62e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc99e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45518/0 (socket says 172.21.15.16:45518) 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.467+0000 7fa62e59c700 1 -- 172.21.15.16:0/885349636 learned_addr learned my addr 172.21.15.16:0/885349636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc998cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc998350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62e59c700 1 -- 172.21.15.16:0/885349636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 msgr2=0x559fcc998cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62e59c700 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc998cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62e59c700 1 -- 172.21.15.16:0/885349636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 msgr2=0x559fcc998350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62e59c700 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc998350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62e59c700 1 -- 172.21.15.16:0/885349636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fcc7693b0 con 0x559fcc99de00 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa62e59c700 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc99e1e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa6180a1fb0 tx=0x7fa618099f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b721b2932e2223e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa624ff9700 1 -- 172.21.15.16:0/885349636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61809a060 con 0x559fcc99de00 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa624ff9700 1 -- 172.21.15.16:0/885349636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61809de90 con 0x559fcc99de00 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.468+0000 7fa624ff9700 1 -- 172.21.15.16:0/885349636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618099550 con 0x559fcc99de00 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 -- 172.21.15.16:0/885349636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 msgr2=0x559fcc99e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc99e1e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa6180a1fb0 tx=0x7fa618099f40 comp rx=0 tx=0).stop 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 -- 172.21.15.16:0/885349636 shutdown_connections 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc998cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc99e1e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 --2- 172.21.15.16:0/885349636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc998350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 -- 172.21.15.16:0/885349636 >> 172.21.15.16:0/885349636 conn(0x559fcc8f6210 msgr2=0x559fcc992df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 -- 172.21.15.16:0/885349636 shutdown_connections 2024-05-08T23:25:13.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 -- 172.21.15.16:0/885349636 wait complete. 2024-05-08T23:25:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.469+0000 7fa640df8b80 1 -- start start 2024-05-08T23:25:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 Processor -- start 2024-05-08T23:25:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc9a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99de00 0x559fcc9a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 0x559fcc9a2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc884560 con 0x559fcc99e720 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc884200 con 0x559fcc998890 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc76b6a0 con 0x559fcc99de00 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa640df8b80 10 client.0 _ll_register_callbacks cb 0x559fcc88db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 0x559fcc9a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 0x559fcc9a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45524/0 (socket says 172.21.15.16:45524) 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 -- 172.21.15.16:0/2842762012 learned_addr learned my addr 172.21.15.16:0/2842762012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc9a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99de00 0x559fcc9a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 -- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99de00 msgr2=0x559fcc9a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99de00 0x559fcc9a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 -- 172.21.15.16:0/2842762012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 msgr2=0x559fcc9a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc9a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62ed9d700 1 -- 172.21.15.16:0/2842762012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6301a4480 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.470+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc9a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 0x559fcc9a2650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa628015000 tx=0x7fa6280095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf938937dff29aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa615ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa615ffb700 1 -- 172.21.15.16:0/2842762012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800e040 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa615ffb700 1 -- 172.21.15.16:0/2842762012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa62800be40 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fcc841270 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa615ffb700 1 -- 172.21.15.16:0/2842762012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa628007c20 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa640df8b80 10 client.16770 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559fcc715b60 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa615ffb700 1 -- 172.21.15.16:0/2842762012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559fcc841270 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.471+0000 7fa615ffb700 1 -- 172.21.15.16:0/2842762012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa628023020 con 0x559fcc99e720 2024-05-08T23:25:13.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa640df8b80 10 client.16770 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa640df8b80 10 client.16770 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa640df8b80 20 client.16770 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa640df8b80 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcc9b22e0 0x559fcc9b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa640df8b80 4 client.16770 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fcc967810 con 0x559fcc9b22e0 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.472+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcc9b22e0 0x559fcc9b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa62e59c700 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcc9b22e0 0x559fcc9b4700 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bc122423a5e07f9 server_cookie=a36f07918efa5964 in_seq=0 out_seq=0 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa615ffb700 10 client.16770 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa615ffb700 1 -- 172.21.15.16:0/2842762012 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559fcc967810 con 0x559fcc9b22e0 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa615ffb700 10 client.16770 handle_command_reply: tid=0 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa640df8b80 1 client.16770 shutdown 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 msgr2=0x559fcc9a2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.473+0000 7fa640df8b80 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 0x559fcc9a2650 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa628015000 tx=0x7fa6280095c0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 shutdown_connections 2024-05-08T23:25:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99e720 0x559fcc9a2650 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcc9b22e0 0x559fcc9b4700 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc9a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 --2- 172.21.15.16:0/2842762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99de00 0x559fcc9a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa615ffb700 0 client.16770 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 >> 172.21.15.16:0/2842762012 conn(0x559fcc8f6210 msgr2=0x559fcc9925a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 shutdown_connections 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 1 -- 172.21.15.16:0/2842762012 wait complete. 2024-05-08T23:25:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.474+0000 7fa640df8b80 20 client.16770 trim_cache size 0 max 16384 2024-05-08T23:25:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 Processor -- start 2024-05-08T23:25:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 -- start start 2024-05-08T23:25:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc998350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99e720 0x559fcc992d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc994e90 con 0x559fcc99de00 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc9ae160 con 0x559fcc998890 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa640df8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc98b100 con 0x559fcc99e720 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa62dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa62dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45534/0 (socket says 172.21.15.16:45534) 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.488+0000 7fa62dd9b700 1 -- 172.21.15.16:0/1978302912 learned_addr learned my addr 172.21.15.16:0/1978302912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99e720 0x559fcc992d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62e59c700 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc998350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62dd9b700 1 -- 172.21.15.16:0/1978302912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99e720 msgr2=0x559fcc992d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99e720 0x559fcc992d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62dd9b700 1 -- 172.21.15.16:0/1978302912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 msgr2=0x559fcc998350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc998350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62dd9b700 1 -- 172.21.15.16:0/1978302912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fcc8ae560 con 0x559fcc99de00 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9adc20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa62000f840 tx=0x7fa620012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0a00f585633a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa6257fa700 1 -- 172.21.15.16:0/1978302912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620011c10 con 0x559fcc99de00 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa6257fa700 1 -- 172.21.15.16:0/1978302912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa620011d70 con 0x559fcc99de00 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.489+0000 7fa6257fa700 1 -- 172.21.15.16:0/1978302912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6200206a0 con 0x559fcc99de00 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- 172.21.15.16:0/1978302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 msgr2=0x559fcc9adc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9adc20 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa62000f840 tx=0x7fa620012000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- 172.21.15.16:0/1978302912 shutdown_connections 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc99e720 0x559fcc992d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc998890 0x559fcc998350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1978302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9adc20 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- 172.21.15.16:0/1978302912 >> 172.21.15.16:0/1978302912 conn(0x559fcc8f6210 msgr2=0x559fcc998c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- 172.21.15.16:0/1978302912 shutdown_connections 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- 172.21.15.16:0/1978302912 wait complete. 2024-05-08T23:25:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- start start 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 Processor -- start 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc9a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9a7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc9a7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc9a7bd0 con 0x559fcc99de00 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc9a7d40 con 0x559fcc99e720 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc9a7f00 con 0x559fcc998890 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.490+0000 7fa640df8b80 10 client.0 _ll_register_callbacks cb 0x559fcc88db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc9a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9a7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45544/0 (socket says 172.21.15.16:45544) 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62dd9b700 1 -- 172.21.15.16:0/1904931037 learned_addr learned my addr 172.21.15.16:0/1904931037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62ed9d700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc9a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 -- 172.21.15.16:0/1904931037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 msgr2=0x559fcc9a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc9a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 -- 172.21.15.16:0/1904931037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 msgr2=0x559fcc9a7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9a7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 -- 172.21.15.16:0/1904931037 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62000f9f0 con 0x559fcc998890 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa62e59c700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc9a9360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x559fcc9a4cc0 tx=0x7fa61801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4cb0b50c996aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa615ffb700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa615ffb700 1 -- 172.21.15.16:0/1904931037 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61809a0e0 con 0x559fcc998890 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa61809de90 con 0x559fcc998890 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.491+0000 7fa615ffb700 1 -- 172.21.15.16:0/1904931037 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa618099550 con 0x559fcc998890 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa615ffb700 1 -- 172.21.15.16:0/1904931037 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa618099710 con 0x559fcc998890 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 10 client.26512 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559fcc76dcd0 con 0x559fcc998890 2024-05-08T23:25:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa615ffb700 1 -- 172.21.15.16:0/1904931037 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa61807a080 con 0x559fcc998890 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa615ffb700 1 -- 172.21.15.16:0/1904931037 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa618095e50 con 0x559fcc998890 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 10 client.26512 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 10 client.26512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 20 client.26512 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcca08ab0 0x559fcca0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 4 client.26512 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559fcc8e0ec0 con 0x559fcca08ab0 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.492+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcca08ab0 0x559fcca0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa62dd9b700 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcca08ab0 0x559fcca0aed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44a6ab94a1fbe371 server_cookie=19922ad8b02dc99c in_seq=0 out_seq=0 2024-05-08T23:25:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa615ffb700 10 client.26512 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa615ffb700 1 -- 172.21.15.16:0/1904931037 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559fcc8e0ec0 con 0x559fcca08ab0 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa615ffb700 10 client.26512 handle_command_reply: tid=0 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 client.26512 shutdown 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 msgr2=0x559fcc9a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc9a9360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x559fcc9a4cc0 tx=0x7fa61801f000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 shutdown_connections 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559fcca08ab0 0x559fcca0aed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559fcc99e720 0x559fcc9a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa615ffb700 0 client.26512 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559fcc998890 0x559fcc9a9360 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 --2- 172.21.15.16:0/1904931037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559fcc99de00 0x559fcc9a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 >> 172.21.15.16:0/1904931037 conn(0x559fcc8f6210 msgr2=0x559fcc9a0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 shutdown_connections 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.493+0000 7fa640df8b80 1 -- 172.21.15.16:0/1904931037 wait complete. 2024-05-08T23:25:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa640df8b80 20 client.26512 trim_cache size 0 max 16384 2024-05-08T23:25:13.501 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:25:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa61c0a1290 msgr2=0x7fa61c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa61c0a1290 0x7fa61c0a3750 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa620005950 tx=0x7fa6200058e0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 msgr2=0x7fa630197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 0x7fa630197460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa61800e9a0 tx=0x7fa61800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 shutdown_connections 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa61c0a1290 0x7fa61c0a3750 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa63019bd40 0x7fa63019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa630069420 0x7fa630197460 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 --2- 172.21.15.16:0/2598649647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6301062a0 0x7fa6301979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 >> 172.21.15.16:0/2598649647 conn(0x7fa630076ba0 msgr2=0x7fa6300ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 shutdown_connections 2024-05-08T23:25:13.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.494+0000 7fa634c3e700 1 -- 172.21.15.16:0/2598649647 wait complete. 2024-05-08T23:25:13.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:25:13.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:13 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3971824403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 -- 172.21.15.16:0/3055990601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a8068c90 msgr2=0x7f81a8105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 --2- 172.21.15.16:0/3055990601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a8068c90 0x7f81a8105b60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f819c0099c0 tx=0x7f819c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 -- 172.21.15.16:0/3055990601 shutdown_connections 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 --2- 172.21.15.16:0/3055990601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a8106290 0x7f81a810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 --2- 172.21.15.16:0/3055990601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81a80682e0 0x7f81a80686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 --2- 172.21.15.16:0/3055990601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a8068c90 0x7f81a8105b60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 -- 172.21.15.16:0/3055990601 >> 172.21.15.16:0/3055990601 conn(0x7f81a8076f40 msgr2=0x7f81a8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 -- 172.21.15.16:0/3055990601 shutdown_connections 2024-05-08T23:25:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.649+0000 7f81ade9a700 1 -- 172.21.15.16:0/3055990601 wait complete. 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 Processor -- start 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 -- start start 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 0x7f81a80ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81a8106290 0x7f81a81003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81008e0 0x7f81a81a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a81a5a90 con 0x7f81a80682e0 2024-05-08T23:25:13.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a81a5c00 con 0x7f81a81008e0 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.650+0000 7f81ade9a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a81a5dc0 con 0x7f81a8106290 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 0x7f81a80ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 0x7f81a80ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45570/0 (socket says 172.21.15.16:45570) 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 -- 172.21.15.16:0/130601680 learned_addr learned my addr 172.21.15.16:0/130601680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81a8106290 0x7f81a81003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a7fff700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81008e0 0x7f81a81a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 -- 172.21.15.16:0/130601680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81a8106290 msgr2=0x7f81a81003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81a8106290 0x7f81a81003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 -- 172.21.15.16:0/130601680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81008e0 msgr2=0x7f81a81a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81008e0 0x7f81a81a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 -- 172.21.15.16:0/130601680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c0095f0 con 0x7f81a80682e0 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a7fff700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81008e0 0x7f81a81a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:13.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a77fe700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 0x7f81a80ffe60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9a0 tx=0x7f819800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a4ff9700 1 -- 172.21.15.16:0/130601680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198009790 con 0x7f81a80682e0 2024-05-08T23:25:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a81a60a0 con 0x7f81a80682e0 2024-05-08T23:25:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a4ff9700 1 -- 172.21.15.16:0/130601680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f819800cbd0 con 0x7f81a80682e0 2024-05-08T23:25:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81a4ff9700 1 -- 172.21.15.16:0/130601680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198018680 con 0x7f81a80682e0 2024-05-08T23:25:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.651+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a81a65b0 con 0x7f81a80682e0 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.654+0000 7f81a4ff9700 1 -- 172.21.15.16:0/130601680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f819800cd40 con 0x7f81a80682e0 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.654+0000 7f81a4ff9700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f81900a1290 0x7f81900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.654+0000 7f81a4ff9700 1 -- 172.21.15.16:0/130601680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f8198014070 con 0x7f81a80682e0 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.654+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f81900a1290 0x7f81900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.655+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f81900a1290 0x7f81900a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81a8101590 tx=0x7f819c00b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.655+0000 7f81ba054b80 1 Processor -- start 2024-05-08T23:25:13.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.655+0000 7f81ba054b80 1 -- start start 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b3e00 0x564b4d2b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 0x564b4d2ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2ae890 0x564b4d2aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81ba054b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d205150 con 0x564b4d2b4720 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81ba054b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d202b10 con 0x564b4d2b3e00 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81ba054b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d1fe400 con 0x564b4d2ae890 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 0x564b4d2ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 0x564b4d2ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45576/0 (socket says 172.21.15.16:45576) 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 -- 172.21.15.16:0/1179633169 learned_addr learned my addr 172.21.15.16:0/1179633169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a77fe700 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b3e00 0x564b4d2b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a7fff700 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2ae890 0x564b4d2aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 -- 172.21.15.16:0/1179633169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2ae890 msgr2=0x564b4d2aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2ae890 0x564b4d2aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 -- 172.21.15.16:0/1179633169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b3e00 msgr2=0x564b4d2b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b3e00 0x564b4d2b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 -- 172.21.15.16:0/1179633169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4d07f3b0 con 0x564b4d2b4720 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.656+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 0x564b4d2ae350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f819c015000 tx=0x7f819c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d0657a922cb597 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.657+0000 7f818dffb700 1 -- 172.21.15.16:0/1179633169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c02a070 con 0x564b4d2b4720 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.657+0000 7f818dffb700 1 -- 172.21.15.16:0/1179633169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f819c01d040 con 0x564b4d2b4720 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.657+0000 7f818dffb700 1 -- 172.21.15.16:0/1179633169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c003ce0 con 0x564b4d2b4720 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.657+0000 7f81ba054b80 1 -- 172.21.15.16:0/1179633169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 msgr2=0x564b4d2ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.657+0000 7f81ba054b80 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 0x564b4d2ae350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f819c015000 tx=0x7f819c01e000 comp rx=0 tx=0).stop 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.657+0000 7f81ba054b80 1 -- 172.21.15.16:0/1179633169 shutdown_connections 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2ae890 0x564b4d2aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b3e00 0x564b4d2b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 --2- 172.21.15.16:0/1179633169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b4720 0x564b4d2ae350 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 -- 172.21.15.16:0/1179633169 >> 172.21.15.16:0/1179633169 conn(0x564b4d20c210 msgr2=0x564b4d2a8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 -- 172.21.15.16:0/1179633169 shutdown_connections 2024-05-08T23:25:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 -- 172.21.15.16:0/1179633169 wait complete. 2024-05-08T23:25:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 -- start start 2024-05-08T23:25:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.658+0000 7f81ba054b80 1 Processor -- start 2024-05-08T23:25:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 0x564b4d2ba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b3e00 0x564b4d2b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b4720 0x564b4d2b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d19a560 con 0x564b4d2ae890 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d19a200 con 0x564b4d2b4720 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d0816a0 con 0x564b4d2b3e00 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81ba054b80 10 client.0 _ll_register_callbacks cb 0x564b4d1a3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 0x564b4d2ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 0x564b4d2ba750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45580/0 (socket says 172.21.15.16:45580) 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 -- 172.21.15.16:0/4248984870 learned_addr learned my addr 172.21.15.16:0/4248984870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b3e00 0x564b4d2b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a7fff700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b4720 0x564b4d2b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 -- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b3e00 msgr2=0x564b4d2b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b3e00 0x564b4d2b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 -- 172.21.15.16:0/4248984870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b4720 msgr2=0x564b4d2b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b4720 0x564b4d2b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 -- 172.21.15.16:0/4248984870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a810e650 con 0x564b4d2ae890 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a7fff700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b4720 0x564b4d2b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f81a77fe700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 0x564b4d2ba750 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f819800c5b0 tx=0x7f819809a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c31c4a47e056066 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.659+0000 7f8186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f8186ffd700 1 -- 172.21.15.16:0/4248984870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81980b1070 con 0x564b4d2ae890 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f8186ffd700 1 -- 172.21.15.16:0/4248984870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f81980a2590 con 0x564b4d2ae890 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b4d157270 con 0x564b4d2ae890 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f8186ffd700 1 -- 172.21.15.16:0/4248984870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81980caea0 con 0x564b4d2ae890 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 10 client.16788 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b4d02bb60 con 0x564b4d2ae890 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f8186ffd700 1 -- 172.21.15.16:0/4248984870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564b4d157270 con 0x564b4d2ae890 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f8186ffd700 1 -- 172.21.15.16:0/4248984870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8198099600 con 0x564b4d2ae890 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 10 client.16788 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 10 client.16788 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 20 client.16788 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d2c80c0 0x564b4d2ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 4 client.16788 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.660+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b4d27d810 con 0x564b4d2c80c0 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.661+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d2c80c0 0x564b4d2ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.661+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d2c80c0 0x564b4d2ca4b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1f6880c98e43b4 server_cookie=eb778d89be0009e9 in_seq=0 out_seq=0 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.661+0000 7f8186ffd700 10 client.16788 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f8186ffd700 1 -- 172.21.15.16:0/4248984870 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b4d27d810 con 0x564b4d2c80c0 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f8186ffd700 10 client.16788 handle_command_reply: tid=0 2024-05-08T23:25:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 client.16788 shutdown 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 msgr2=0x564b4d2ba750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 0x564b4d2ba750 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f819800c5b0 tx=0x7f819809a1c0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 shutdown_connections 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d2c80c0 0x564b4d2ca4b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2b4720 0x564b4d2b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2ae890 0x564b4d2ba750 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 --2- 172.21.15.16:0/4248984870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b3e00 0x564b4d2b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f8186ffd700 0 client.16788 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 >> 172.21.15.16:0/4248984870 conn(0x564b4d20c210 msgr2=0x564b4d2a85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 shutdown_connections 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.662+0000 7f81ba054b80 1 -- 172.21.15.16:0/4248984870 wait complete. 2024-05-08T23:25:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.663+0000 7f81ba054b80 20 client.16788 trim_cache size 0 max 16384 2024-05-08T23:25:13.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:13 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3971824403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:25:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 Processor -- start 2024-05-08T23:25:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 -- start start 2024-05-08T23:25:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d2c4c50 con 0x564b4d2b3e00 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d2a3a90 con 0x564b4d2ae890 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81ba054b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d2a5dc0 con 0x564b4d2b4720 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2c4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45594/0 (socket says 172.21.15.16:45594) 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 -- 172.21.15.16:0/3502347205 learned_addr learned my addr 172.21.15.16:0/3502347205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a77fe700 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a7fff700 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 -- 172.21.15.16:0/3502347205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 msgr2=0x564b4d2a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 -- 172.21.15.16:0/3502347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 msgr2=0x564b4d2ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.677+0000 7f81a6ffd700 1 -- 172.21.15.16:0/3502347205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4d1c4560 con 0x564b4d2b3e00 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2c4200 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f819c005690 tx=0x7f819c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fcf61bbb11864d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f818e7fc700 1 -- 172.21.15.16:0/3502347205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c02a040 con 0x564b4d2b3e00 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f818e7fc700 1 -- 172.21.15.16:0/3502347205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f819c038860 con 0x564b4d2b3e00 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f818e7fc700 1 -- 172.21.15.16:0/3502347205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00bcd0 con 0x564b4d2b3e00 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 -- 172.21.15.16:0/3502347205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 msgr2=0x564b4d2c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2c4200 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f819c005690 tx=0x7f819c003680 comp rx=0 tx=0).stop 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 -- 172.21.15.16:0/3502347205 shutdown_connections 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 --2- 172.21.15.16:0/3502347205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2c4200 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.678+0000 7f81ba054b80 1 -- 172.21.15.16:0/3502347205 >> 172.21.15.16:0/3502347205 conn(0x564b4d20c210 msgr2=0x564b4d2aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 -- 172.21.15.16:0/3502347205 shutdown_connections 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 -- 172.21.15.16:0/3502347205 wait complete. 2024-05-08T23:25:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 -- start start 2024-05-08T23:25:13.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 Processor -- start 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d2bdb10 con 0x564b4d2b3e00 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d2bdc80 con 0x564b4d2ae890 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4d2bde40 con 0x564b4d2b4720 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.679+0000 7f81ba054b80 10 client.0 _ll_register_callbacks cb 0x564b4d1a3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a7fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a7fff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2bd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:42930/0 (socket says 172.21.15.16:42930) 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a7fff700 1 -- 172.21.15.16:0/489747632 learned_addr learned my addr 172.21.15.16:0/489747632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a77fe700 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 -- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 msgr2=0x564b4d2bd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2bd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 -- 172.21.15.16:0/489747632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 msgr2=0x564b4d2bf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2bf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 -- 172.21.15.16:0/489747632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b4d07f3b0 con 0x564b4d2b3e00 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81a6ffd700 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2bd090 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f819c0098e0 tx=0x7f819c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5af892e9806970 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f8186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f8186ffd700 1 -- 172.21.15.16:0/489747632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c02a070 con 0x564b4d2b3e00 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f8186ffd700 1 -- 172.21.15.16:0/489747632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f819c003be0 con 0x564b4d2b3e00 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.680+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f819c01d040 con 0x564b4d2b3e00 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f8186ffd700 1 -- 172.21.15.16:0/489747632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f819c003da0 con 0x564b4d2b3e00 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 10 client.16794 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b4d083cd0 con 0x564b4d2b3e00 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f8186ffd700 1 -- 172.21.15.16:0/489747632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f819c01d040 con 0x564b4d2b3e00 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f8186ffd700 1 -- 172.21.15.16:0/489747632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f819c03f070 con 0x564b4d2b3e00 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 10 client.16794 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 10 client.16794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 20 client.16794 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d31e700 0x564b4d320b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 4 client.16794 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b4d1f6ec0 con 0x564b4d31e700 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.681+0000 7f81a77fe700 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d31e700 0x564b4d320b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f8186ffd700 10 client.16794 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81a77fe700 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d31e700 0x564b4d320b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8450b7678d8c16 server_cookie=ae979f428e55d100 in_seq=0 out_seq=0 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f8186ffd700 1 -- 172.21.15.16:0/489747632 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b4d1f6ec0 con 0x564b4d31e700 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f8186ffd700 10 client.16794 handle_command_reply: tid=0 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 client.16794 shutdown 2024-05-08T23:25:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 msgr2=0x564b4d2bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2bd090 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f819c0098e0 tx=0x7f819c005750 comp rx=0 tx=0).stop 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 shutdown_connections 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b4d2b4720 0x564b4d2bd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x564b4d31e700 0x564b4d320b20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564b4d2ae890 0x564b4d2bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 --2- 172.21.15.16:0/489747632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4d2b3e00 0x564b4d2bd090 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f8186ffd700 0 client.16794 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 >> 172.21.15.16:0/489747632 conn(0x564b4d20c210 msgr2=0x564b4d2aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 shutdown_connections 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.682+0000 7f81ba054b80 1 -- 172.21.15.16:0/489747632 wait complete. 2024-05-08T23:25:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ba054b80 20 client.16794 trim_cache size 0 max 16384 2024-05-08T23:25:13.690 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:25:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f81900a1290 msgr2=0x7f81900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f81900a1290 0x7f81900a3750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81a8101590 tx=0x7f819c00b540 comp rx=0 tx=0).stop 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 msgr2=0x7f81a80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 0x7f81a80ffe60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9a0 tx=0x7f819800ed60 comp rx=0 tx=0).stop 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 shutdown_connections 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f81900a1290 0x7f81900a3750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81a81008e0 0x7f81a81a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a80682e0 0x7f81a80ffe60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 --2- 172.21.15.16:0/130601680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81a8106290 0x7f81a81003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 >> 172.21.15.16:0/130601680 conn(0x7f81a8076f40 msgr2=0x7f81a80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 shutdown_connections 2024-05-08T23:25:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.683+0000 7f81ade9a700 1 -- 172.21.15.16:0/130601680 wait complete. 2024-05-08T23:25:13.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:25:13.714 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:13 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3971824403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:25:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 -- 172.21.15.16:0/2099714446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c101160 msgr2=0x7fb53c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 --2- 172.21.15.16:0/2099714446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c101160 0x7fb53c101540 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb538009990 tx=0x7fb538009ca0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 -- 172.21.15.16:0/2099714446 shutdown_connections 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 --2- 172.21.15.16:0/2099714446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb53c106280 0x7fb53c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 --2- 172.21.15.16:0/2099714446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c101160 0x7fb53c101540 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 --2- 172.21.15.16:0/2099714446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb53c101b10 0x7fb53c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 -- 172.21.15.16:0/2099714446 >> 172.21.15.16:0/2099714446 conn(0x7fb53c0fc9e0 msgr2=0x7fb53c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.836+0000 7fb543930700 1 -- 172.21.15.16:0/2099714446 shutdown_connections 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 -- 172.21.15.16:0/2099714446 wait complete. 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 Processor -- start 2024-05-08T23:25:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 -- start start 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb53c101160 0x7fb53c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 0x7fb53c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb53c073d20 0x7fb53c1a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1a5af0 con 0x7fb53c106280 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1a5c30 con 0x7fb53c073d20 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.837+0000 7fb543930700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c1a5df0 con 0x7fb53c101160 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 0x7fb53c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 0x7fb53c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45622/0 (socket says 172.21.15.16:45622) 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 -- 172.21.15.16:0/2476726964 learned_addr learned my addr 172.21.15.16:0/2476726964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb53c101160 0x7fb53c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb541ecd700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb53c073d20 0x7fb53c1a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 -- 172.21.15.16:0/2476726964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb53c101160 msgr2=0x7fb53c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb53c101160 0x7fb53c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 -- 172.21.15.16:0/2476726964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb53c073d20 msgr2=0x7fb53c1a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb53c073d20 0x7fb53c1a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 -- 172.21.15.16:0/2476726964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380095f0 con 0x7fb53c106280 2024-05-08T23:25:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 0x7fb53c0737e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00d980 tx=0x7fb52c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb5327fc700 1 -- 172.21.15.16:0/2476726964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00f890 con 0x7fb53c106280 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb5327fc700 1 -- 172.21.15.16:0/2476726964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb52c00fe80 con 0x7fb53c106280 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c1a60d0 con 0x7fb53c106280 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb5327fc700 1 -- 172.21.15.16:0/2476726964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00e3b0 con 0x7fb53c106280 2024-05-08T23:25:13.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.838+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1a65e0 con 0x7fb53c106280 2024-05-08T23:25:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.841+0000 7fb5327fc700 1 -- 172.21.15.16:0/2476726964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb52c00f9f0 con 0x7fb53c106280 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.841+0000 7fb5327fc700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5280a1290 0x7fb5280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.842+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5280a1290 0x7fb5280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.842+0000 7fb5327fc700 1 -- 172.21.15.16:0/2476726964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb52c0cca90 con 0x7fb53c106280 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.842+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5280a1290 0x7fb5280a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb53800b2e0 tx=0x7fb538005040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.842+0000 7fb54faeab80 1 Processor -- start 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 -- start start 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea09350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745e960150 con 0x55745ea0f720 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745e95db10 con 0x55745ea09890 2024-05-08T23:25:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb54faeab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745e959400 con 0x55745ea0ee00 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea09350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea09350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45636/0 (socket says 172.21.15.16:45636) 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 -- 172.21.15.16:0/2697709772 learned_addr learned my addr 172.21.15.16:0/2697709772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb541ecd700 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 -- 172.21.15.16:0/2697709772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 msgr2=0x55745ea0f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea0f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 -- 172.21.15.16:0/2697709772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 msgr2=0x55745ea09cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea09cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 -- 172.21.15.16:0/2697709772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55745e7da3b0 con 0x55745ea0f720 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb541ecd700 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea09cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.843+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea09350 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0997f0 tx=0x7fb52c0994a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9685a1f5e4ae691 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb5277fe700 1 -- 172.21.15.16:0/2697709772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c099f60 con 0x55745ea0f720 2024-05-08T23:25:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb5277fe700 1 -- 172.21.15.16:0/2697709772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb52c099860 con 0x55745ea0f720 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb5277fe700 1 -- 172.21.15.16:0/2697709772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c099ac0 con 0x55745ea0f720 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 -- 172.21.15.16:0/2697709772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 msgr2=0x55745ea09350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea09350 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0997f0 tx=0x7fb52c0994a0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 -- 172.21.15.16:0/2697709772 shutdown_connections 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea09cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea0f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2697709772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea09350 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.844+0000 7fb54faeab80 1 -- 172.21.15.16:0/2697709772 >> 172.21.15.16:0/2697709772 conn(0x55745e967210 msgr2=0x55745ea03df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 -- 172.21.15.16:0/2697709772 shutdown_connections 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 -- 172.21.15.16:0/2697709772 wait complete. 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 -- start start 2024-05-08T23:25:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 Processor -- start 2024-05-08T23:25:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 0x55745ea12aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea12fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea0f720 0x55745ea13520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745e8f5560 con 0x55745ea09890 2024-05-08T23:25:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745e8f5200 con 0x55745ea0f720 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.845+0000 7fb54faeab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745e7dc6a0 con 0x55745ea0ee00 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb54faeab80 10 client.0 _ll_register_callbacks cb 0x55745e8feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb5416cc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 0x55745ea12aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea12fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea12fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:42964/0 (socket says 172.21.15.16:42964) 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb5416cc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 0x55745ea12aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45650/0 (socket says 172.21.15.16:45650) 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 -- 172.21.15.16:0/2563953018 learned_addr learned my addr 172.21.15.16:0/2563953018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb541ecd700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea0f720 0x55745ea13520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 -- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea0f720 msgr2=0x55745ea13520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea0f720 0x55745ea13520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 -- 172.21.15.16:0/2563953018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 msgr2=0x55745ea12aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 0x55745ea12aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 -- 172.21.15.16:0/2563953018 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1a65e0 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 0x55745ea12aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb540ecb700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea12fe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0ccd70 tx=0x7fb52c099f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da358cdffa94c911 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.846+0000 7fb524ff9700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb524ff9700 1 -- 172.21.15.16:0/2563953018 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0a1ce0 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55745e8b2270 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb524ff9700 1 -- 172.21.15.16:0/2563953018 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb52c098d50 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb524ff9700 1 -- 172.21.15.16:0/2563953018 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb52c098f10 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 10 client.26542 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55745e786b60 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb524ff9700 1 -- 172.21.15.16:0/2563953018 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55745e8b2270 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb524ff9700 1 -- 172.21.15.16:0/2563953018 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb52c0a1850 con 0x55745ea0ee00 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 10 client.26542 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 20 client.26542 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea22c60 0x55745ea25050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 4 client.26542 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.847+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55745e9d8810 con 0x55745ea22c60 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.848+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea22c60 0x55745ea25050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.848+0000 7fb5416cc700 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea22c60 0x55745ea25050 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2514dcac64f4770d server_cookie=56cc5d41aacd1ab6 in_seq=0 out_seq=0 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.848+0000 7fb524ff9700 10 client.26542 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb524ff9700 1 -- 172.21.15.16:0/2563953018 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55745e9d8810 con 0x55745ea22c60 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb524ff9700 10 client.26542 handle_command_reply: tid=0 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 client.26542 shutdown 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 msgr2=0x55745ea12fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea12fe0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0ccd70 tx=0x7fb52c099f60 comp rx=0 tx=0).stop 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 shutdown_connections 2024-05-08T23:25:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea22c60 0x55745ea25050 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea0f720 0x55745ea13520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea09890 0x55745ea12aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 --2- 172.21.15.16:0/2563953018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea12fe0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb524ff9700 0 client.26542 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 >> 172.21.15.16:0/2563953018 conn(0x55745e967210 msgr2=0x55745ea035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 shutdown_connections 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 1 -- 172.21.15.16:0/2563953018 wait complete. 2024-05-08T23:25:13.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.849+0000 7fb54faeab80 20 client.26542 trim_cache size 0 max 16384 2024-05-08T23:25:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 Processor -- start 2024-05-08T23:25:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 -- start start 2024-05-08T23:25:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea03d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea13340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745ea2a260 con 0x55745ea0f720 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745ea10ff0 con 0x55745ea09890 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.863+0000 7fb54faeab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745ea04a90 con 0x55745ea0ee00 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea13340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea13340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45658/0 (socket says 172.21.15.16:45658) 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 -- 172.21.15.16:0/4229642073 learned_addr learned my addr 172.21.15.16:0/4229642073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb5416cc700 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb540ecb700 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea03d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 -- 172.21.15.16:0/4229642073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 msgr2=0x55745ea03d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea03d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 -- 172.21.15.16:0/4229642073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 msgr2=0x55745ea1f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 -- 172.21.15.16:0/4229642073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55745e91f560 con 0x55745ea0f720 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb540ecb700 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea03d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb541ecd700 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea13340 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb534007630 tx=0x7fb53400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f2fa06b62eebd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb527fff700 1 -- 172.21.15.16:0/4229642073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400e040 con 0x55745ea0f720 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb527fff700 1 -- 172.21.15.16:0/4229642073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb53400f040 con 0x55745ea0f720 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.864+0000 7fb527fff700 1 -- 172.21.15.16:0/4229642073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534017670 con 0x55745ea0f720 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 -- 172.21.15.16:0/4229642073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 msgr2=0x55745ea13340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea13340 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb534007630 tx=0x7fb53400b040 comp rx=0 tx=0).stop 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 -- 172.21.15.16:0/4229642073 shutdown_connections 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0f720 0x55745ea13340 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 --2- 172.21.15.16:0/4229642073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0ee00 0x55745ea03d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 -- 172.21.15.16:0/4229642073 >> 172.21.15.16:0/4229642073 conn(0x55745e967210 msgr2=0x55745ea00210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 -- 172.21.15.16:0/4229642073 shutdown_connections 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 -- 172.21.15.16:0/4229642073 wait complete. 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.865+0000 7fb54faeab80 1 -- start start 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 Processor -- start 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 0x55745ea0a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0f720 0x55745ea0af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745ea0b610 con 0x55745ea0ee00 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745ea0b780 con 0x55745ea09890 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55745ea0b940 con 0x55745ea0f720 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb54faeab80 10 client.0 _ll_register_callbacks cb 0x55745e8feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 0x55745ea0a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb540ecb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 0x55745ea0a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45674/0 (socket says 172.21.15.16:45674) 2024-05-08T23:25:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb540ecb700 1 -- 172.21.15.16:0/154655939 learned_addr learned my addr 172.21.15.16:0/154655939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb541ecd700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0f720 0x55745ea0af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.866+0000 7fb5416cc700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb540ecb700 1 -- 172.21.15.16:0/154655939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0f720 msgr2=0x55745ea0af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb540ecb700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0f720 0x55745ea0af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb540ecb700 1 -- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 msgr2=0x55745ea1a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb540ecb700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb540ecb700 1 -- 172.21.15.16:0/154655939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5340092f0 con 0x55745ea0ee00 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb540ecb700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 0x55745ea0a9a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb52c09a8c0 tx=0x7fb52c0109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf67b32804bda0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb524ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb524ff9700 1 -- 172.21.15.16:0/154655939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c0b9070 con 0x55745ea0ee00 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb524ff9700 1 -- 172.21.15.16:0/154655939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fb52c0a1ce0 con 0x55745ea0ee00 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb52c099860 con 0x55745ea0ee00 2024-05-08T23:25:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb524ff9700 1 -- 172.21.15.16:0/154655939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb52c09d690 con 0x55745ea0ee00 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb54faeab80 10 client.16812 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55745e7decd0 con 0x55745ea0ee00 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.867+0000 7fb524ff9700 1 -- 172.21.15.16:0/154655939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb52c099860 con 0x55745ea0ee00 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb524ff9700 1 -- 172.21.15.16:0/154655939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb52c095650 con 0x55745ea0ee00 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb54faeab80 10 client.16812 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb54faeab80 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb54faeab80 20 client.16812 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb54faeab80 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea79430 0x55745ea7b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb54faeab80 4 client.16812 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55745e951ec0 con 0x55745ea79430 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb5416cc700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea79430 0x55745ea7b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb524ff9700 10 client.16812 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.868+0000 7fb5416cc700 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea79430 0x55745ea7b820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b860fbd1ece80b05 server_cookie=f1691f9e9397165 in_seq=0 out_seq=0 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb524ff9700 1 -- 172.21.15.16:0/154655939 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55745e951ec0 con 0x55745ea79430 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb524ff9700 10 client.16812 handle_command_reply: tid=0 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 client.16812 shutdown 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 msgr2=0x55745ea0a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 0x55745ea0a9a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb52c09a8c0 tx=0x7fb52c0109d0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 shutdown_connections 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55745ea0f720 0x55745ea0af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55745ea79430 0x55745ea7b820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55745ea09890 0x55745ea1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 --2- 172.21.15.16:0/154655939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55745ea0ee00 0x55745ea0a9a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb524ff9700 0 client.16812 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 >> 172.21.15.16:0/154655939 conn(0x55745e967210 msgr2=0x55745ea1f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 shutdown_connections 2024-05-08T23:25:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 1 -- 172.21.15.16:0/154655939 wait complete. 2024-05-08T23:25:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.869+0000 7fb54faeab80 20 client.16812 trim_cache size 0 max 16384 2024-05-08T23:25:13.877 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5280a1290 msgr2=0x7fb5280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5280a1290 0x7fb5280a3750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb53800b2e0 tx=0x7fb538005040 comp rx=0 tx=0).stop 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 msgr2=0x7fb53c0737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 0x7fb53c0737e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00d980 tx=0x7fb52c00dc90 comp rx=0 tx=0).stop 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 shutdown_connections 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5280a1290 0x7fb5280a3750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb53c073d20 0x7fb53c1a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb53c101160 0x7fb53c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 --2- 172.21.15.16:0/2476726964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb53c106280 0x7fb53c0737e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 >> 172.21.15.16:0/2476726964 conn(0x7fb53c0fc9e0 msgr2=0x7fb53c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 shutdown_connections 2024-05-08T23:25:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:13.870+0000 7fb543930700 1 -- 172.21.15.16:0/2476726964 wait complete. 2024-05-08T23:25:14.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:14 smithi016 ceph-mon[42541]: pgmap v173: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 30 op/s 2024-05-08T23:25:14.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:14 smithi189 ceph-mon[46919]: pgmap v173: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 30 op/s 2024-05-08T23:25:14.714 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:14 smithi155 ceph-mon[47105]: pgmap v173: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 63 KiB/s wr, 30 op/s 2024-05-08T23:25:14.885 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:25:14.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:25:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 -- 172.21.15.16:0/287106681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 msgr2=0x7fab3c106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 --2- 172.21.15.16:0/287106681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c106370 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fab380099c0 tx=0x7fab38009cd0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 -- 172.21.15.16:0/287106681 shutdown_connections 2024-05-08T23:25:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 --2- 172.21.15.16:0/287106681 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab3c100ab0 0x7fab3c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 --2- 172.21.15.16:0/287106681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c106370 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 --2- 172.21.15.16:0/287106681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab3c1000f0 0x7fab3c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 -- 172.21.15.16:0/287106681 >> 172.21.15.16:0/287106681 conn(0x7fab3c076ba0 msgr2=0x7fab3c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 -- 172.21.15.16:0/287106681 shutdown_connections 2024-05-08T23:25:15.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.035+0000 7fab43c74700 1 -- 172.21.15.16:0/287106681 wait complete. 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 Processor -- start 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 -- start start 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab3c1000f0 0x7fab3c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab3c100ab0 0x7fab3c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19c170 con 0x7fab3c105f90 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19c2e0 con 0x7fab3c1000f0 2024-05-08T23:25:15.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab43c74700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19c4a0 con 0x7fab3c100ab0 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.036+0000 7fab42211700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab4120f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab3c100ab0 0x7fab3c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45688/0 (socket says 172.21.15.16:45688) 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 -- 172.21.15.16:0/1433917124 learned_addr learned my addr 172.21.15.16:0/1433917124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab41a10700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab3c1000f0 0x7fab3c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 -- 172.21.15.16:0/1433917124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab3c100ab0 msgr2=0x7fab3c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab3c100ab0 0x7fab3c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 -- 172.21.15.16:0/1433917124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab3c1000f0 msgr2=0x7fab3c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab3c1000f0 0x7fab3c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 -- 172.21.15.16:0/1433917124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380095f0 con 0x7fab3c105f90 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab42211700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c19bb30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fab3400ef30 tx=0x7fab3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab32ffd700 1 -- 172.21.15.16:0/1433917124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34018070 con 0x7fab3c105f90 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab32ffd700 1 -- 172.21.15.16:0/1433917124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3400f040 con 0x7fab3c105f90 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c19c720 con 0x7fab3c105f90 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab32ffd700 1 -- 172.21.15.16:0/1433917124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400cbc0 con 0x7fab3c105f90 2024-05-08T23:25:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.037+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1a3fc0 con 0x7fab3c105f90 2024-05-08T23:25:15.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.040+0000 7fab32ffd700 1 -- 172.21.15.16:0/1433917124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab34007500 con 0x7fab3c105f90 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.040+0000 7fab32ffd700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab280a1310 0x7fab280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.040+0000 7fab32ffd700 1 -- 172.21.15.16:0/1433917124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fab340ccaa0 con 0x7fab3c105f90 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.040+0000 7fab41a10700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab280a1310 0x7fab280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.041+0000 7fab41a10700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab280a1310 0x7fab280a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b580 tx=0x7fab38005c00 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.041+0000 7fab4fe2eb80 1 Processor -- start 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.041+0000 7fab4fe2eb80 1 -- start start 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f8647260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f8641d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f85981d0 con 0x5605f8641910 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f8595b90 con 0x5605f86477a0 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f8591480 con 0x5605f8646e80 2024-05-08T23:25:15.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f8641d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f8641d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45700/0 (socket says 172.21.15.16:45700) 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 -- 172.21.15.16:0/1160205749 learned_addr learned my addr 172.21.15.16:0/1160205749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab41a10700 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f8647260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab4120f700 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 -- 172.21.15.16:0/1160205749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 msgr2=0x5605f8647260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f8647260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 -- 172.21.15.16:0/1160205749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 msgr2=0x5605f86413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 -- 172.21.15.16:0/1160205749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f8412430 con 0x5605f8641910 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.042+0000 7fab42211700 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f8641d50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fab340a17c0 tx=0x7fab34099260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbe4c836e2bc780 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab27fff700 1 -- 172.21.15.16:0/1160205749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3409a000 con 0x5605f8641910 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab27fff700 1 -- 172.21.15.16:0/1160205749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab34099900 con 0x5605f8641910 2024-05-08T23:25:15.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab27fff700 1 -- 172.21.15.16:0/1160205749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34099b60 con 0x5605f8641910 2024-05-08T23:25:15.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1160205749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 msgr2=0x5605f8641d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f8641d50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fab340a17c0 tx=0x7fab34099260 comp rx=0 tx=0).stop 2024-05-08T23:25:15.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1160205749 shutdown_connections 2024-05-08T23:25:15.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f8641d50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f8647260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1160205749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.043+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1160205749 >> 172.21.15.16:0/1160205749 conn(0x5605f859f290 msgr2=0x5605f863be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.044+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1160205749 shutdown_connections 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.044+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1160205749 wait complete. 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.044+0000 7fab4fe2eb80 1 -- start start 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.044+0000 7fab4fe2eb80 1 Processor -- start 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8641910 0x5605f864ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f864b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 0x5605f864b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f852d5e0 con 0x5605f86477a0 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f852d280 con 0x5605f8641910 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f8414720 con 0x5605f8646e80 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 10 client.0 _ll_register_callbacks cb 0x5605f8536bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 0x5605f864b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 0x5605f864b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45702/0 (socket says 172.21.15.16:45702) 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 -- 172.21.15.16:0/2067609427 learned_addr learned my addr 172.21.15.16:0/2067609427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab41a10700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8641910 0x5605f864ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4120f700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f864b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 -- 172.21.15.16:0/2067609427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 msgr2=0x5605f864b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f864b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 -- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8641910 msgr2=0x5605f864ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8641910 0x5605f864ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 -- 172.21.15.16:0/2067609427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1a3fc0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab42211700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 0x5605f864b5b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fab340c9f60 tx=0x7fab34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487a12cf6aeaef6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab257fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab257fa700 1 -- 172.21.15.16:0/2067609427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340a1cf0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab257fa700 1 -- 172.21.15.16:0/2067609427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fab3400cd80 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.045+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605f84ea2f0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab257fa700 1 -- 172.21.15.16:0/2067609427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab34098df0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 10 client.16824 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5605f83bebe0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab257fa700 1 -- 172.21.15.16:0/2067609427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5605f84ea2f0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab257fa700 1 -- 172.21.15.16:0/2067609427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab34099cb0 con 0x5605f86477a0 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 10 client.16824 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 20 client.16824 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f865ae10 0x5605f865d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 4 client.16824 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605f8610890 con 0x5605f865ae10 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.046+0000 7fab41a10700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f865ae10 0x5605f865d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.047+0000 7fab257fa700 10 client.16824 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.047+0000 7fab41a10700 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f865ae10 0x5605f865d230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2826f90b8cb58c4f server_cookie=9ae1f76666f0756 in_seq=0 out_seq=0 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.047+0000 7fab257fa700 1 -- 172.21.15.16:0/2067609427 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605f8610890 con 0x5605f865ae10 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.047+0000 7fab257fa700 10 client.16824 handle_command_reply: tid=0 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 client.16824 shutdown 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 msgr2=0x5605f864b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 0x5605f864b5b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fab340c9f60 tx=0x7fab34004ad0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 shutdown_connections 2024-05-08T23:25:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f86477a0 0x5605f864b5b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f865ae10 0x5605f865d230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8641910 0x5605f864ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/2067609427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8646e80 0x5605f864b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 >> 172.21.15.16:0/2067609427 conn(0x5605f859f290 msgr2=0x5605f863b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab257fa700 0 client.16824 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 shutdown_connections 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/2067609427 wait complete. 2024-05-08T23:25:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.048+0000 7fab4fe2eb80 20 client.16824 trim_cache size 0 max 16384 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.066+0000 7fab4fe2eb80 1 Processor -- start 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.066+0000 7fab4fe2eb80 1 -- start start 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f86413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8646e80 0x5605f863bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f86477a0 0x5605f864b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f864ded0 con 0x5605f8641910 2024-05-08T23:25:15.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f864c1a0 con 0x5605f8646e80 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f8638e40 con 0x5605f86477a0 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f86413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f86413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45710/0 (socket says 172.21.15.16:45710) 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 -- 172.21.15.16:0/1992313365 learned_addr learned my addr 172.21.15.16:0/1992313365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab42211700 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f86477a0 0x5605f864b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab4120f700 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8646e80 0x5605f863bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 -- 172.21.15.16:0/1992313365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f86477a0 msgr2=0x5605f864b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f86477a0 0x5605f864b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 -- 172.21.15.16:0/1992313365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8646e80 msgr2=0x5605f863bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8646e80 0x5605f863bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 -- 172.21.15.16:0/1992313365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f85575e0 con 0x5605f8641910 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.067+0000 7fab41a10700 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f86413d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fab380099c0 tx=0x7fab38020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9ce3b123deb852 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab30da4700 1 -- 172.21.15.16:0/1992313365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38003680 con 0x5605f8641910 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab30da4700 1 -- 172.21.15.16:0/1992313365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3801e040 con 0x5605f8641910 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab30da4700 1 -- 172.21.15.16:0/1992313365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38034780 con 0x5605f8641910 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1992313365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 msgr2=0x5605f86413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f86413d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fab380099c0 tx=0x7fab38020000 comp rx=0 tx=0).stop 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1992313365 shutdown_connections 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f86477a0 0x5605f864b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8641910 0x5605f86413d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/1992313365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f8646e80 0x5605f863bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1992313365 >> 172.21.15.16:0/1992313365 conn(0x5605f859f290 msgr2=0x5605f8649420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1992313365 shutdown_connections 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.068+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/1992313365 wait complete. 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 -- start start 2024-05-08T23:25:15.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 Processor -- start 2024-05-08T23:25:15.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8641910 0x5605f864f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 0x5605f864f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f864fff0 con 0x5605f8646e80 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f86b1840 con 0x5605f86477a0 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f86b1a00 con 0x5605f8641910 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4fe2eb80 10 client.0 _ll_register_callbacks cb 0x5605f8536bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4120f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 0x5605f864f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4120f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 0x5605f864f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45720/0 (socket says 172.21.15.16:45720) 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.069+0000 7fab4120f700 1 -- 172.21.15.16:0/3212251844 learned_addr learned my addr 172.21.15.16:0/3212251844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab41a10700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8641910 0x5605f864f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab42211700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4120f700 1 -- 172.21.15.16:0/3212251844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8641910 msgr2=0x5605f864f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4120f700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8641910 0x5605f864f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4120f700 1 -- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 msgr2=0x5605f86b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4120f700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4120f700 1 -- 172.21.15.16:0/3212251844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c10d550 con 0x5605f8646e80 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab41a10700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8641910 0x5605f864f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab42211700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4120f700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 0x5605f864f850 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fab3c1987a0 tx=0x7fab2c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2734879f36cda09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab257fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab257fa700 1 -- 172.21.15.16:0/3212251844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c004830 con 0x5605f8646e80 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab257fa700 1 -- 172.21.15.16:0/3212251844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fab2c004990 con 0x5605f8646e80 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab34099900 con 0x5605f8646e80 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab257fa700 1 -- 172.21.15.16:0/3212251844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab2c004b50 con 0x5605f8646e80 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4fe2eb80 10 client.16830 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5605f8416d50 con 0x5605f8646e80 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.070+0000 7fab257fa700 1 -- 172.21.15.16:0/3212251844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab34099900 con 0x5605f8646e80 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab257fa700 1 -- 172.21.15.16:0/3212251844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab2c004cd0 con 0x5605f8646e80 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab4fe2eb80 10 client.16830 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab4fe2eb80 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab4fe2eb80 20 client.16830 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f86b32e0 0x5605f86b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab4fe2eb80 4 client.16830 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5605f8589f40 con 0x5605f86b32e0 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.071+0000 7fab41a10700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f86b32e0 0x5605f86b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.072+0000 7fab257fa700 10 client.16830 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.072+0000 7fab41a10700 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f86b32e0 0x5605f86b5700 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1071e9650bcb83 server_cookie=571e1b7916c43bef in_seq=0 out_seq=0 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.073+0000 7fab257fa700 1 -- 172.21.15.16:0/3212251844 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5605f8589f40 con 0x5605f86b32e0 2024-05-08T23:25:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.073+0000 7fab257fa700 10 client.16830 handle_command_reply: tid=0 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.073+0000 7fab4fe2eb80 1 client.16830 shutdown 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 msgr2=0x5605f864f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 0x5605f864f850 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fab3c1987a0 tx=0x7fab2c00bdf0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 shutdown_connections 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5605f86b32e0 0x5605f86b5700 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5605f86477a0 0x5605f86b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605f8641910 0x5605f864f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 --2- 172.21.15.16:0/3212251844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5605f8646e80 0x5605f864f850 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 >> 172.21.15.16:0/3212251844 conn(0x5605f859f290 msgr2=0x5605f86484c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab257fa700 0 client.16830 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 shutdown_connections 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 1 -- 172.21.15.16:0/3212251844 wait complete. 2024-05-08T23:25:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab4fe2eb80 20 client.16830 trim_cache size 0 max 16384 2024-05-08T23:25:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:25:15.080 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:25:15.080 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "9956f930-95d9-42a6-b41a-f439d3959987", 2024-05-08T23:25:15.080 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:25:15.080 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab280a1310 msgr2=0x7fab280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab280a1310 0x7fab280a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab3800b580 tx=0x7fab38005c00 comp rx=0 tx=0).stop 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 msgr2=0x7fab3c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c19bb30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fab3400ef30 tx=0x7fab3400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 shutdown_connections 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab280a1310 0x7fab280a37d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab3c105f90 0x7fab3c19bb30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab3c1000f0 0x7fab3c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.074+0000 7fab43c74700 1 --2- 172.21.15.16:0/1433917124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab3c100ab0 0x7fab3c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.075+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 >> 172.21.15.16:0/1433917124 conn(0x7fab3c076ba0 msgr2=0x7fab3c0694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.075+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 shutdown_connections 2024-05-08T23:25:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.075+0000 7fab43c74700 1 -- 172.21.15.16:0/1433917124 wait complete. 2024-05-08T23:25:15.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.227+0000 7fc4cc504700 1 -- 172.21.15.16:0/3503081588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 msgr2=0x7fc4c410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.227+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3503081588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c410de10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b0d0 tx=0x7fc4bc00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 -- 172.21.15.16:0/3503081588 shutdown_connections 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3503081588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c410de10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3503081588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 0x7fc4c40686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3503081588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c4068c90 0x7fc4c4105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 -- 172.21.15.16:0/3503081588 >> 172.21.15.16:0/3503081588 conn(0x7fc4c4076f40 msgr2=0x7fc4c4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 -- 172.21.15.16:0/3503081588 shutdown_connections 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 -- 172.21.15.16:0/3503081588 wait complete. 2024-05-08T23:25:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 Processor -- start 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 -- start start 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 0x7fc4c40ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.228+0000 7fc4cc504700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c41003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4cc504700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41008e0 0x7fc4c41a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4cc504700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41a5b90 con 0x7fc4c4106290 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4cc504700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41a5d00 con 0x7fc4c41008e0 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4cc504700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41a5ec0 con 0x7fc4c40682e0 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c41003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c41003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45734/0 (socket says 172.21.15.16:45734) 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/3526487720 learned_addr learned my addr 172.21.15.16:0/3526487720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41008e0 0x7fc4c41a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 0x7fc4c40ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/3526487720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 msgr2=0x7fc4c40ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 0x7fc4c40ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/3526487720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41008e0 msgr2=0x7fc4c41a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41008e0 0x7fc4c41a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/3526487720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc009d00 con 0x7fc4c4106290 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 0x7fc4c40ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c41003a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d980 tx=0x7fc4b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4bb7fe700 1 -- 172.21.15.16:0/3526487720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4009930 con 0x7fc4c4106290 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4bb7fe700 1 -- 172.21.15.16:0/3526487720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4b4010460 con 0x7fc4c4106290 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c41a61a0 con 0x7fc4c4106290 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4bb7fe700 1 -- 172.21.15.16:0/3526487720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400f620 con 0x7fc4c4106290 2024-05-08T23:25:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.229+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c41a66b0 con 0x7fc4c4106290 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.232+0000 7fc4bb7fe700 1 -- 172.21.15.16:0/3526487720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc4b40105d0 con 0x7fc4c4106290 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.233+0000 7fc4bb7fe700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc4b00a1290 0x7fc4b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.233+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc4b00a1290 0x7fc4b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.233+0000 7fc4bb7fe700 1 -- 172.21.15.16:0/3526487720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc4b40ccaa0 con 0x7fc4c4106290 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.233+0000 7fc4d86beb80 1 Processor -- start 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.233+0000 7fc4d86beb80 1 -- start start 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e17e00 0x5652b2e181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e12cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2d69150 con 0x5652b2e18720 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2d66b10 con 0x5652b2e17e00 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2d62400 con 0x5652b2e12890 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc4b00a1290 0x7fc4b00a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005950 tx=0x7fc4c00058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e12350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45740/0 (socket says 172.21.15.16:45740) 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2031757885 learned_addr learned my addr 172.21.15.16:0/2031757885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e12cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e17e00 0x5652b2e181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2031757885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 msgr2=0x5652b2e12cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e12cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2031757885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e17e00 msgr2=0x5652b2e181e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e17e00 0x5652b2e181e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2031757885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652b2be33b0 con 0x5652b2e18720 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e12cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e17e00 0x5652b2e181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e12350 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40a1760 tx=0x7fc4b4099240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1730b430a04940f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4b8ff9700 1 -- 172.21.15.16:0/2031757885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b409d640 con 0x5652b2e18720 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4b8ff9700 1 -- 172.21.15.16:0/2031757885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4b40a1c90 con 0x5652b2e18720 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.234+0000 7fc4b8ff9700 1 -- 172.21.15.16:0/2031757885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4099cd0 con 0x5652b2e18720 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2031757885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 msgr2=0x5652b2e12350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e12350 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40a1760 tx=0x7fc4b4099240 comp rx=0 tx=0).stop 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2031757885 shutdown_connections 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e12cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e17e00 0x5652b2e181e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2031757885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e12350 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2031757885 >> 172.21.15.16:0/2031757885 conn(0x5652b2d70210 msgr2=0x5652b2e0cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2031757885 shutdown_connections 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.235+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2031757885 wait complete. 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 -- start start 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 Processor -- start 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e1ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 0x5652b2e1bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e18720 0x5652b2e1c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2cfe560 con 0x5652b2e17e00 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2cfe200 con 0x5652b2e18720 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2be56a0 con 0x5652b2e12890 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4d86beb80 10 client.0 _ll_register_callbacks cb 0x5652b2d07b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4c9a9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 0x5652b2e1bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4c9a9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 0x5652b2e1bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45752/0 (socket says 172.21.15.16:45752) 2024-05-08T23:25:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2160509702 learned_addr learned my addr 172.21.15.16:0/2160509702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e1ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.236+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e18720 0x5652b2e1c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2160509702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 msgr2=0x5652b2e1ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e1ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e18720 msgr2=0x5652b2e1c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e18720 0x5652b2e1c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4c9a9f700 1 -- 172.21.15.16:0/2160509702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c41a66b0 con 0x5652b2e17e00 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 0x5652b2e1bf90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4010880 tx=0x7fc4b40c9e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b86598bb4d6cb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4adffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4adffb700 1 -- 172.21.15.16:0/2160509702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4098d00 con 0x5652b2e17e00 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4adffb700 1 -- 172.21.15.16:0/2160509702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc4b4098e60 con 0x5652b2e17e00 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652b2cbb270 con 0x5652b2e17e00 2024-05-08T23:25:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4adffb700 1 -- 172.21.15.16:0/2160509702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b4099a70 con 0x5652b2e17e00 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4d86beb80 10 client.16842 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652b2b8fb60 con 0x5652b2e17e00 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.237+0000 7fc4adffb700 1 -- 172.21.15.16:0/2160509702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5652b2cbb270 con 0x5652b2e17e00 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4adffb700 1 -- 172.21.15.16:0/2160509702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc4b4095600 con 0x5652b2e17e00 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4d86beb80 10 client.16842 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4d86beb80 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4d86beb80 20 client.16842 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e2bc50 0x5652b2e2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4d86beb80 4 client.16842 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652b2de1810 con 0x5652b2e2bc50 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e2bc50 0x5652b2e2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e2bc50 0x5652b2e2e040 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c17a077762e8d6 server_cookie=19bc868e695a2cbe in_seq=0 out_seq=0 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.238+0000 7fc4adffb700 10 client.16842 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4adffb700 1 -- 172.21.15.16:0/2160509702 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652b2de1810 con 0x5652b2e2bc50 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4adffb700 10 client.16842 handle_command_reply: tid=0 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 client.16842 shutdown 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 msgr2=0x5652b2e1bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 0x5652b2e1bf90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4010880 tx=0x7fc4b40c9e60 comp rx=0 tx=0).stop 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 shutdown_connections 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e2bc50 0x5652b2e2e040 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e18720 0x5652b2e1c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e12890 0x5652b2e1ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2160509702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e17e00 0x5652b2e1bf90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4adffb700 0 client.16842 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 >> 172.21.15.16:0/2160509702 conn(0x5652b2d70210 msgr2=0x5652b2e0c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 shutdown_connections 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2160509702 wait complete. 2024-05-08T23:25:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.239+0000 7fc4d86beb80 20 client.16842 trim_cache size 0 max 16384 2024-05-08T23:25:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 Processor -- start 2024-05-08T23:25:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 -- start start 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e17290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2e1d0c0 con 0x5652b2e18720 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2e09dc0 con 0x5652b2e12890 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2e07800 con 0x5652b2e17e00 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4caaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4caaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e230a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45756/0 (socket says 172.21.15.16:45756) 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4caaa1700 1 -- 172.21.15.16:0/1947189381 learned_addr learned my addr 172.21.15.16:0/1947189381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.258+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e17290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4caaa1700 1 -- 172.21.15.16:0/1947189381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 msgr2=0x5652b2e17290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e17290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4caaa1700 1 -- 172.21.15.16:0/1947189381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 msgr2=0x5652b2e0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4caaa1700 1 -- 172.21.15.16:0/1947189381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652b2d28560 con 0x5652b2e18720 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e230a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc007be0 tx=0x7fc4bc007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c5b65f757d5702 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4b97fa700 1 -- 172.21.15.16:0/1947189381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00e040 con 0x5652b2e18720 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4b97fa700 1 -- 172.21.15.16:0/1947189381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4bc0074e0 con 0x5652b2e18720 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.259+0000 7fc4b97fa700 1 -- 172.21.15.16:0/1947189381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc025470 con 0x5652b2e18720 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- 172.21.15.16:0/1947189381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 msgr2=0x5652b2e230a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e230a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc007be0 tx=0x7fc4bc007c10 comp rx=0 tx=0).stop 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- 172.21.15.16:0/1947189381 shutdown_connections 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e230a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/1947189381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e17290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- 172.21.15.16:0/1947189381 >> 172.21.15.16:0/1947189381 conn(0x5652b2d70210 msgr2=0x5652b2e16b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- 172.21.15.16:0/1947189381 shutdown_connections 2024-05-08T23:25:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- 172.21.15.16:0/1947189381 wait complete. 2024-05-08T23:25:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- start start 2024-05-08T23:25:15.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 Processor -- start 2024-05-08T23:25:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e1f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e1fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2e201a0 con 0x5652b2e18720 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2e20310 con 0x5652b2e12890 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652b2e204d0 con 0x5652b2e17e00 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.260+0000 7fc4d86beb80 10 client.0 _ll_register_callbacks cb 0x5652b2d07b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e1fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e1fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45762/0 (socket says 172.21.15.16:45762) 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 -- 172.21.15.16:0/2115129563 learned_addr learned my addr 172.21.15.16:0/2115129563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e1f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 -- 172.21.15.16:0/2115129563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 msgr2=0x5652b2e1f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e1f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 -- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 msgr2=0x5652b2e232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 -- 172.21.15.16:0/2115129563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c410e650 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4c9a9f700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e1f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4caaa1700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e1fc20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b0d0 tx=0x7fc4bc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1e37eda15bb6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4adffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4adffb700 1 -- 172.21.15.16:0/2115129563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00e050 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4adffb700 1 -- 172.21.15.16:0/2115129563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fc4bc009460 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4b40a1c90 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.261+0000 7fc4adffb700 1 -- 172.21.15.16:0/2115129563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4bc0052b0 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 10 client.16848 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652b2be7cd0 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4adffb700 1 -- 172.21.15.16:0/2115129563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4b40a1c90 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4adffb700 1 -- 172.21.15.16:0/2115129563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc4bc00b100 con 0x5652b2e18720 2024-05-08T23:25:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 10 client.16848 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 20 client.16848 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e83390 0x5652b2e85780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 4 client.16848 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652b2d5aec0 con 0x5652b2e83390 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.262+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e83390 0x5652b2e85780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4adffb700 10 client.16848 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4ca2a0700 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e83390 0x5652b2e85780 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec3cc8836467e5c server_cookie=9ee17666ccc55d4a in_seq=0 out_seq=0 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4adffb700 1 -- 172.21.15.16:0/2115129563 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652b2d5aec0 con 0x5652b2e83390 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4adffb700 10 client.16848 handle_command_reply: tid=0 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 client.16848 shutdown 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 msgr2=0x5652b2e1fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e1fc20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b0d0 tx=0x7fc4bc003a00 comp rx=0 tx=0).stop 2024-05-08T23:25:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 shutdown_connections 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652b2e18720 0x5652b2e1fc20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5652b2e83390 0x5652b2e85780 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5652b2e12890 0x5652b2e232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 --2- 172.21.15.16:0/2115129563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652b2e17e00 0x5652b2e1f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4adffb700 0 client.16848 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 >> 172.21.15.16:0/2115129563 conn(0x5652b2d70210 msgr2=0x5652b2e17410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 shutdown_connections 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 1 -- 172.21.15.16:0/2115129563 wait complete. 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.263+0000 7fc4d86beb80 20 client.16848 trim_cache size 0 max 16384 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:25:15.270 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: "9956f930-95d9-42a6-b41a-f439d3959987": { 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "9956f930-95d9-42a6-b41a-f439d3959987", 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:25:15.271 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc4b00a1290 msgr2=0x7fc4b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc4b00a1290 0x7fc4b00a3750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005950 tx=0x7fc4c00058e0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 msgr2=0x7fc4c41003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c41003a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d980 tx=0x7fc4b400dc90 comp rx=0 tx=0).stop 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 shutdown_connections 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc4b00a1290 0x7fc4b00a3750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4c41008e0 0x7fc4c41a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4c40682e0 0x7fc4c40ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 --2- 172.21.15.16:0/3526487720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c4106290 0x7fc4c41003a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:15.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 >> 172.21.15.16:0/3526487720 conn(0x7fc4c4076f40 msgr2=0x7fc4c40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 shutdown_connections 2024-05-08T23:25:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:15.264+0000 7fc4cc504700 1 -- 172.21.15.16:0/3526487720 wait complete. 2024-05-08T23:25:15.278 INFO:tasks.cephfs.filesystem:scrub status for tag:9956f930-95d9-42a6-b41a-f439d3959987 - {'path': '/', 'tag': '9956f930-95d9-42a6-b41a-f439d3959987', 'options': 'recursive,force'} 2024-05-08T23:25:15.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:15 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:25:15.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:15 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:15.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:15 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:25:15.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:15 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:15.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:15 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:25:15.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:15 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:15.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:15 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:25:15.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:15 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:15.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:15 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:25:15.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:15 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:15.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:15 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:25:15.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:15 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:16.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:16 smithi189 ceph-mon[46919]: pgmap v174: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:16.715 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:16 smithi155 ceph-mon[47105]: pgmap v174: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:16.865 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:16 smithi016 ceph-mon[42541]: pgmap v174: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:18 smithi016 ceph-mon[42541]: pgmap v175: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:18.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:18 smithi189 ceph-mon[46919]: pgmap v175: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:18.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:18 smithi155 ceph-mon[47105]: pgmap v175: 129 pgs: 129 active+clean; 6.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:20.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:20 smithi016 ceph-mon[42541]: pgmap v176: 129 pgs: 129 active+clean; 6.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:20.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:20 smithi189 ceph-mon[46919]: pgmap v176: 129 pgs: 129 active+clean; 6.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:20.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:20 smithi155 ceph-mon[47105]: pgmap v176: 129 pgs: 129 active+clean; 6.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:21.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:21 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:25:21.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:21 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:25:21.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:21 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:25:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:22 smithi016 ceph-mon[42541]: pgmap v177: 129 pgs: 129 active+clean; 6.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:22.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:22 smithi189 ceph-mon[46919]: pgmap v177: 129 pgs: 129 active+clean; 6.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:22.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:22 smithi155 ceph-mon[47105]: pgmap v177: 129 pgs: 129 active+clean; 6.2 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:24.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:24 smithi016 ceph-mon[42541]: pgmap v178: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:24.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:24 smithi189 ceph-mon[46919]: pgmap v178: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:24.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:24 smithi155 ceph-mon[47105]: pgmap v178: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:26.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:26 smithi016 ceph-mon[42541]: pgmap v179: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:26.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:26 smithi189 ceph-mon[46919]: pgmap v179: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:26.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:26 smithi155 ceph-mon[47105]: pgmap v179: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:28.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:28 smithi016 ceph-mon[42541]: pgmap v180: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:28.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:28 smithi189 ceph-mon[46919]: pgmap v180: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:28.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:28 smithi155 ceph-mon[47105]: pgmap v180: 129 pgs: 129 active+clean; 6.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:30.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:30 smithi016 ceph-mon[42541]: pgmap v181: 129 pgs: 129 active+clean; 6.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:30.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:30 smithi189 ceph-mon[46919]: pgmap v181: 129 pgs: 129 active+clean; 6.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:30.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:30 smithi155 ceph-mon[47105]: pgmap v181: 129 pgs: 129 active+clean; 6.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:32 smithi016 ceph-mon[42541]: pgmap v182: 129 pgs: 129 active+clean; 6.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:32.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:32 smithi189 ceph-mon[46919]: pgmap v182: 129 pgs: 129 active+clean; 6.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:32 smithi155 ceph-mon[47105]: pgmap v182: 129 pgs: 129 active+clean; 6.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:34 smithi016 ceph-mon[42541]: pgmap v183: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-05-08T23:25:34.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:34 smithi189 ceph-mon[46919]: pgmap v183: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-05-08T23:25:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:34 smithi155 ceph-mon[47105]: pgmap v183: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-05-08T23:25:37.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:37 smithi016 ceph-mon[42541]: pgmap v184: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:37.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:37 smithi189 ceph-mon[46919]: pgmap v184: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:37.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:37 smithi155 ceph-mon[47105]: pgmap v184: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:39.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:39 smithi016 ceph-mon[42541]: pgmap v185: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:39.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:39 smithi189 ceph-mon[46919]: pgmap v185: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:39.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:39 smithi155 ceph-mon[47105]: pgmap v185: 129 pgs: 129 active+clean; 6.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:25:41.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:41 smithi016 ceph-mon[42541]: pgmap v186: 129 pgs: 129 active+clean; 6.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:41.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:41 smithi189 ceph-mon[46919]: pgmap v186: 129 pgs: 129 active+clean; 6.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:41.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:41 smithi155 ceph-mon[47105]: pgmap v186: 129 pgs: 129 active+clean; 6.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:43.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:43 smithi016 ceph-mon[42541]: pgmap v187: 129 pgs: 129 active+clean; 6.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:43 smithi189 ceph-mon[46919]: pgmap v187: 129 pgs: 129 active+clean; 6.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:43.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:43 smithi155 ceph-mon[47105]: pgmap v187: 129 pgs: 129 active+clean; 6.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:45.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:25:45.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:45 smithi016 ceph-mon[42541]: pgmap v188: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:45.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:45 smithi189 ceph-mon[46919]: pgmap v188: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:45.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 -- 172.21.15.16:0/3384004153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a8105f90 msgr2=0x7fa1a8106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/3384004153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a8105f90 0x7fa1a8106370 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa1980099c0 tx=0x7fa198009cd0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 -- 172.21.15.16:0/3384004153 shutdown_connections 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/3384004153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8100ab0 0x7fa1a810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/3384004153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a8105f90 0x7fa1a8106370 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/3384004153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a81000f0 0x7fa1a8100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 -- 172.21.15.16:0/3384004153 >> 172.21.15.16:0/3384004153 conn(0x7fa1a8076ba0 msgr2=0x7fa1a8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 -- 172.21.15.16:0/3384004153 shutdown_connections 2024-05-08T23:25:45.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.428+0000 7fa1ad90f700 1 -- 172.21.15.16:0/3384004153 wait complete. 2024-05-08T23:25:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.429+0000 7fa1ad90f700 1 Processor -- start 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 -- start start 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 0x7fa1a8197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a8100ab0 0x7fa1a8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8105f90 0x7fa1a819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a819c170 con 0x7fa1a81000f0 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a819c2e0 con 0x7fa1a8105f90 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1ad90f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a819c4a0 con 0x7fa1a8100ab0 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 0x7fa1a8197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 0x7fa1a8197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50014/0 (socket says 172.21.15.16:50014) 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/6046645 learned_addr learned my addr 172.21.15.16:0/6046645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a77fe700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8105f90 0x7fa1a819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a8100ab0 0x7fa1a8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/6046645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a8100ab0 msgr2=0x7fa1a8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a8100ab0 0x7fa1a8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/6046645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8105f90 msgr2=0x7fa1a819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8105f90 0x7fa1a819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/6046645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980095f0 con 0x7fa1a81000f0 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a8100ab0 0x7fa1a8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.430+0000 7fa1a77fe700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8105f90 0x7fa1a819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.431+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 0x7fa1a8197250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa198000c00 tx=0x7fa19800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.431+0000 7fa1ac90d700 1 -- 172.21.15.16:0/6046645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19801c070 con 0x7fa1a81000f0 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.431+0000 7fa1ac90d700 1 -- 172.21.15.16:0/6046645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1980047d0 con 0x7fa1a81000f0 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.431+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a819c720 con 0x7fa1a81000f0 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.431+0000 7fa1ac90d700 1 -- 172.21.15.16:0/6046645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198005290 con 0x7fa1a81000f0 2024-05-08T23:25:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.431+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a819c990 con 0x7fa1a81000f0 2024-05-08T23:25:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.433+0000 7fa1ac90d700 1 -- 172.21.15.16:0/6046645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa198020070 con 0x7fa1a81000f0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.434+0000 7fa1ac90d700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1900a1290 0x7fa1900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.434+0000 7fa1ac90d700 1 -- 172.21.15.16:0/6046645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa1980cdd60 con 0x7fa1a81000f0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.434+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1900a1290 0x7fa1900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.434+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1900a1290 0x7fa1900a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a81987a0 tx=0x7fa19c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 Processor -- start 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 -- start start 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c0ca0 0x559d491c1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d49113150 con 0x559d491c61f0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d49110b10 con 0x559d491c6b10 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d4910c400 con 0x559d491c0ca0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50022/0 (socket says 172.21.15.16:50022) 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/4153585229 learned_addr learned my addr 172.21.15.16:0/4153585229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/4153585229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c0ca0 msgr2=0x559d491c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c0ca0 0x559d491c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/4153585229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 msgr2=0x559d491c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/4153585229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d48f8d3b0 con 0x559d491c61f0 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.435+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.436+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491c65d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa1980975d0 tx=0x7fa19809a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e810690f7a9d282 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.436+0000 7fa18dffb700 1 -- 172.21.15.16:0/4153585229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19809e990 con 0x559d491c61f0 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.436+0000 7fa18dffb700 1 -- 172.21.15.16:0/4153585229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1980a2f50 con 0x559d491c61f0 2024-05-08T23:25:45.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.436+0000 7fa18dffb700 1 -- 172.21.15.16:0/4153585229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19809afe0 con 0x559d491c61f0 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.436+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/4153585229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 msgr2=0x559d491c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.436+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491c65d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa1980975d0 tx=0x7fa19809a720 comp rx=0 tx=0).stop 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/4153585229 shutdown_connections 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c0ca0 0x559d491c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491c65d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/4153585229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/4153585229 >> 172.21.15.16:0/4153585229 conn(0x559d4911a210 msgr2=0x559d491b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/4153585229 shutdown_connections 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/4153585229 wait complete. 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 -- start start 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.437+0000 7fa1b9ac9b80 1 Processor -- start 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d490a8560 con 0x559d491c0ca0 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d490a8200 con 0x559d491c6b10 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d48f8f6a0 con 0x559d491c61f0 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1b9ac9b80 10 client.0 _ll_register_callbacks cb 0x559d490b1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50030/0 (socket says 172.21.15.16:50030) 2024-05-08T23:25:45.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/2005403164 learned_addr learned my addr 172.21.15.16:0/2005403164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a77fe700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/2005403164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 msgr2=0x559d491c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 msgr2=0x559d491ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/2005403164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a819c990 con 0x559d491c0ca0 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.438+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491ccaa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa1980a2a20 tx=0x7fa19809ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160de2987af7e1db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa186ffd700 1 -- 172.21.15.16:0/2005403164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19809a250 con 0x559d491c0ca0 2024-05-08T23:25:45.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa186ffd700 1 -- 172.21.15.16:0/2005403164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa1980b3440 con 0x559d491c0ca0 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d49065270 con 0x559d491c0ca0 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa186ffd700 1 -- 172.21.15.16:0/2005403164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa19809adc0 con 0x559d491c0ca0 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 10 client.16860 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d48f39b60 con 0x559d491c0ca0 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa186ffd700 1 -- 172.21.15.16:0/2005403164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa19807d020 con 0x559d491c0ca0 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa186ffd700 1 -- 172.21.15.16:0/2005403164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa1980053f0 con 0x559d491c0ca0 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 10 client.16860 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 20 client.16860 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d491d9c60 0x559d491dc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 4 client.16860 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.439+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d4918b810 con 0x559d491d9c60 2024-05-08T23:25:45.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.440+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d491d9c60 0x559d491dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.440+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d491d9c60 0x559d491dc050 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d9a71e3b6fa5e1 server_cookie=b6db83f52ffd8e39 in_seq=0 out_seq=0 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.440+0000 7fa186ffd700 10 client.16860 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa186ffd700 1 -- 172.21.15.16:0/2005403164 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d4918b810 con 0x559d491d9c60 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa186ffd700 10 client.16860 handle_command_reply: tid=0 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 client.16860 shutdown 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 msgr2=0x559d491ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491ccaa0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa1980a2a20 tx=0x7fa19809ac20 comp rx=0 tx=0).stop 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 shutdown_connections 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d491d9c60 0x559d491dc050 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491ccaa0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa186ffd700 0 client.16860 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/2005403164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 >> 172.21.15.16:0/2005403164 conn(0x559d4911a210 msgr2=0x559d491b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 shutdown_connections 2024-05-08T23:25:45.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/2005403164 wait complete. 2024-05-08T23:25:45.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.441+0000 7fa1b9ac9b80 20 client.16860 trim_cache size 0 max 16384 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.459+0000 7fa1b9ac9b80 1 Processor -- start 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.459+0000 7fa1b9ac9b80 1 -- start start 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.459+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.459+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.459+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d491b8e60 con 0x559d491c0ca0 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d491b3dc0 con 0x559d491c6b10 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d491b8890 con 0x559d491c61f0 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491d6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50032/0 (socket says 172.21.15.16:50032) 2024-05-08T23:25:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/3336116966 learned_addr learned my addr 172.21.15.16:0/3336116966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/3336116966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 msgr2=0x559d491b6d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491b6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/3336116966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 msgr2=0x559d491cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 -- 172.21.15.16:0/3336116966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d490d2560 con 0x559d491c0ca0 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491d6270 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa198005000 tx=0x7fa19809a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de39896b2df73003 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa18e7fc700 1 -- 172.21.15.16:0/3336116966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19800f990 con 0x559d491c0ca0 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa18e7fc700 1 -- 172.21.15.16:0/3336116966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa19807d030 con 0x559d491c0ca0 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.460+0000 7fa18e7fc700 1 -- 172.21.15.16:0/3336116966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1980662c0 con 0x559d491c0ca0 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/3336116966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 msgr2=0x559d491d6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491d6270 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa198005000 tx=0x7fa19809a6a0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/3336116966 shutdown_connections 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c6b10 0x559d491cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c0ca0 0x559d491d6270 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/3336116966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c61f0 0x559d491b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/3336116966 >> 172.21.15.16:0/3336116966 conn(0x559d4911a210 msgr2=0x559d491d5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/3336116966 shutdown_connections 2024-05-08T23:25:45.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/3336116966 wait complete. 2024-05-08T23:25:45.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.461+0000 7fa1b9ac9b80 1 -- start start 2024-05-08T23:25:45.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 Processor -- start 2024-05-08T23:25:45.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c0ca0 0x559d491d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c6b10 0x559d491c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d491c24c0 con 0x559d491c61f0 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d491c2630 con 0x559d491c0ca0 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d491c27f0 con 0x559d491c6b10 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1b9ac9b80 10 client.0 _ll_register_callbacks cb 0x559d490b1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50048/0 (socket says 172.21.15.16:50048) 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 -- 172.21.15.16:0/1375821645 learned_addr learned my addr 172.21.15.16:0/1375821645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c0ca0 0x559d491d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a77fe700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c6b10 0x559d491c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 -- 172.21.15.16:0/1375821645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c6b10 msgr2=0x559d491c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c6b10 0x559d491c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 -- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c0ca0 msgr2=0x559d491d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.462+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c0ca0 0x559d491d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1a67fc700 1 -- 172.21.15.16:0/1375821645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d48f39b60 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1a67fc700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491d15f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x559d491c6f70 tx=0x7fa19c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1af0181ea961d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa186ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa186ffd700 1 -- 172.21.15.16:0/1375821645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c012dc0 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa186ffd700 1 -- 172.21.15.16:0/1375821645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fa19c025070 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1980a2f50 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa186ffd700 1 -- 172.21.15.16:0/1375821645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa19c004030 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 10 client.16866 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d48f91cd0 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa186ffd700 1 -- 172.21.15.16:0/1375821645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1980a2f50 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa186ffd700 1 -- 172.21.15.16:0/1375821645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa19c028780 con 0x559d491c61f0 2024-05-08T23:25:45.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 10 client.16866 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 20 client.16866 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d49231360 0x559d49233780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 4 client.16866 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.463+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559d49104ec0 con 0x559d49231360 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.464+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d49231360 0x559d49233780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.464+0000 7fa186ffd700 10 client.16866 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.464+0000 7fa1a6ffd700 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d49231360 0x559d49233780 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9348a977c1bbc74f server_cookie=ae4408382fe1c821 in_seq=0 out_seq=0 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.464+0000 7fa186ffd700 1 -- 172.21.15.16:0/1375821645 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559d49104ec0 con 0x559d49231360 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.464+0000 7fa186ffd700 10 client.16866 handle_command_reply: tid=0 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.464+0000 7fa1b9ac9b80 1 client.16866 shutdown 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 msgr2=0x559d491d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491d15f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x559d491c6f70 tx=0x7fa19c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 shutdown_connections 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d491c6b10 0x559d491c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559d49231360 0x559d49233780 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559d491c0ca0 0x559d491d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 --2- 172.21.15.16:0/1375821645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d491c61f0 0x559d491d15f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa186ffd700 0 client.16866 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 >> 172.21.15.16:0/1375821645 conn(0x559d4911a210 msgr2=0x559d491c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 shutdown_connections 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 1 -- 172.21.15.16:0/1375821645 wait complete. 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1b9ac9b80 20 client.16866 trim_cache size 0 max 16384 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:25:45.470 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:25:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:25:45.471 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:25:45.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1900a1290 msgr2=0x7fa1900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1900a1290 0x7fa1900a3750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1a81987a0 tx=0x7fa19c009260 comp rx=0 tx=0).stop 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 msgr2=0x7fa1a8197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 0x7fa1a8197250 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa198000c00 tx=0x7fa19800f740 comp rx=0 tx=0).stop 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 shutdown_connections 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa1900a1290 0x7fa1900a3750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1a8105f90 0x7fa1a819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1a81000f0 0x7fa1a8197250 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 --2- 172.21.15.16:0/6046645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a8100ab0 0x7fa1a8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.465+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 >> 172.21.15.16:0/6046645 conn(0x7fa1a8076ba0 msgr2=0x7fa1a8069400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.466+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 shutdown_connections 2024-05-08T23:25:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.466+0000 7fa1ad90f700 1 -- 172.21.15.16:0/6046645 wait complete. 2024-05-08T23:25:45.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:45 smithi155 ceph-mon[47105]: pgmap v188: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:45.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:25:45.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:25:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 -- 172.21.15.16:0/3303242934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 msgr2=0x7f9950100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 --2- 172.21.15.16:0/3303242934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950100320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9940009990 tx=0x7f9940009ca0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 -- 172.21.15.16:0/3303242934 shutdown_connections 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 --2- 172.21.15.16:0/3303242934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 0x7f995010cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 --2- 172.21.15.16:0/3303242934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950100320 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 --2- 172.21.15.16:0/3303242934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99501008f0 0x7f9950104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 -- 172.21.15.16:0/3303242934 >> 172.21.15.16:0/3303242934 conn(0x7f99500fb7e0 msgr2=0x7f99500fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 -- 172.21.15.16:0/3303242934 shutdown_connections 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.616+0000 7f99566bb700 1 -- 172.21.15.16:0/3303242934 wait complete. 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.617+0000 7f99566bb700 1 Processor -- start 2024-05-08T23:25:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.617+0000 7f99566bb700 1 -- start start 2024-05-08T23:25:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f99566bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f99566bb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 0x7f99501999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f99566bb700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995019dd50 0x7f995019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f99566bb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019e7d0 con 0x7f99500fff40 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f99566bb700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019e940 con 0x7f995019dd50 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f99566bb700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019eb00 con 0x7f9950105060 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50066/0 (socket says 172.21.15.16:50066) 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 -- 172.21.15.16:0/3691691396 learned_addr learned my addr 172.21.15.16:0/3691691396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994f7fe700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 0x7f99501999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f9954c58700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995019dd50 0x7f995019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 -- 172.21.15.16:0/3691691396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 msgr2=0x7f99501999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 0x7f99501999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 -- 172.21.15.16:0/3691691396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995019dd50 msgr2=0x7f995019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995019dd50 0x7f995019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 -- 172.21.15.16:0/3691691396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400095f0 con 0x7f99500fff40 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f9954c58700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995019dd50 0x7f995019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994f7fe700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 0x7f99501999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994ffff700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950199470 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9940004730 tx=0x7f9940009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.618+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994001d070 con 0x7f99500fff40 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.619+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f994000bea0 con 0x7f99500fff40 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.619+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400173f0 con 0x7f99500fff40 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.619+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99501a6000 con 0x7f99500fff40 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.619+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99501a63d0 con 0x7f99500fff40 2024-05-08T23:25:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.620+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99501030a0 con 0x7f99500fff40 2024-05-08T23:25:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.621+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9940022470 con 0x7f99500fff40 2024-05-08T23:25:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.622+0000 7f994d7fa700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f99380a1290 0x7f99380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.622+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f99400ceb70 con 0x7f99500fff40 2024-05-08T23:25:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.622+0000 7f994f7fe700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f99380a1290 0x7f99380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.622+0000 7f994f7fe700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f99380a1290 0x7f99380a3750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f995019a9c0 tx=0x7f994400b5f0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.625+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9940097d10 con 0x7f99500fff40 2024-05-08T23:25:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.893+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f995019a810 con 0x7f99500fff40 2024-05-08T23:25:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.894+0000 7f994d7fa700 1 -- 172.21.15.16:0/3691691396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f994009b330 con 0x7f99500fff40 2024-05-08T23:25:45.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:25:45.893 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f99380a1290 msgr2=0x7f99380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f99380a1290 0x7f99380a3750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f995019a9c0 tx=0x7f994400b5f0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 msgr2=0x7f9950199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950199470 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9940004730 tx=0x7f9940009570 comp rx=0 tx=0).stop 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 shutdown_connections 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f99380a1290 0x7f99380a3750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f995019dd50 0x7f995019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99500fff40 0x7f9950199470 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 --2- 172.21.15.16:0/3691691396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9950105060 0x7f99501999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 >> 172.21.15.16:0/3691691396 conn(0x7f99500fb7e0 msgr2=0x7f99500fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 shutdown_connections 2024-05-08T23:25:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:45.896+0000 7f99566bb700 1 -- 172.21.15.16:0/3691691396 wait complete. 2024-05-08T23:25:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:25:45.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 -- 172.21.15.16:0/1166036354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 msgr2=0x7f57f0106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 --2- 172.21.15.16:0/1166036354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f0106370 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f57e00099c0 tx=0x7f57e0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 -- 172.21.15.16:0/1166036354 shutdown_connections 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 --2- 172.21.15.16:0/1166036354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0100ab0 0x7f57f010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 --2- 172.21.15.16:0/1166036354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f0106370 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 --2- 172.21.15.16:0/1166036354 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57f01000f0 0x7f57f0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 -- 172.21.15.16:0/1166036354 >> 172.21.15.16:0/1166036354 conn(0x7f57f0076ba0 msgr2=0x7f57f0076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 -- 172.21.15.16:0/1166036354 shutdown_connections 2024-05-08T23:25:46.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.049+0000 7f57f5095700 1 -- 172.21.15.16:0/1166036354 wait complete. 2024-05-08T23:25:46.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 Processor -- start 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 -- start start 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f01000f0 0x7f57f0101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57f0100ab0 0x7f57f0102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f01025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01a5630 con 0x7f57f0105f90 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01a57a0 con 0x7f57f0100ab0 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57f5095700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01a5960 con 0x7f57f01000f0 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57ef59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f01025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57ef59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f01025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50090/0 (socket says 172.21.15.16:50090) 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57ef59e700 1 -- 172.21.15.16:0/3052991363 learned_addr learned my addr 172.21.15.16:0/3052991363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57ee59c700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57f0100ab0 0x7f57f0102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.050+0000 7f57eed9d700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f01000f0 0x7f57f0101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57ef59e700 1 -- 172.21.15.16:0/3052991363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f01000f0 msgr2=0x7f57f0101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57ef59e700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f01000f0 0x7f57f0101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57ef59e700 1 -- 172.21.15.16:0/3052991363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57f0100ab0 msgr2=0x7f57f0102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57ef59e700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57f0100ab0 0x7f57f0102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57ef59e700 1 -- 172.21.15.16:0/3052991363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00095f0 con 0x7f57f0105f90 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57ef59e700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f01025a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ef30 tx=0x7f57e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57d7fff700 1 -- 172.21.15.16:0/3052991363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8018070 con 0x7f57f0105f90 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f01a5be0 con 0x7f57f0105f90 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57d7fff700 1 -- 172.21.15.16:0/3052991363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57e800f040 con 0x7f57f0105f90 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57d7fff700 1 -- 172.21.15.16:0/3052991363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800cbc0 con 0x7f57f0105f90 2024-05-08T23:25:46.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.051+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f01a60a0 con 0x7f57f0105f90 2024-05-08T23:25:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.054+0000 7f57d7fff700 1 -- 172.21.15.16:0/3052991363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f57e800cd80 con 0x7f57f0105f90 2024-05-08T23:25:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.054+0000 7f57d7fff700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57d80a1290 0x7f57d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.055+0000 7f57eed9d700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57d80a1290 0x7f57d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.055+0000 7f57d7fff700 1 -- 172.21.15.16:0/3052991363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f57e80ccb30 con 0x7f57f0105f90 2024-05-08T23:25:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.055+0000 7f57eed9d700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57d80a1290 0x7f57d80a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009990 tx=0x7f57e0019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.055+0000 7f580124fb80 1 Processor -- start 2024-05-08T23:25:46.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.055+0000 7f580124fb80 1 -- start start 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f580124fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264ae00 0x5638a264b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f580124fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 0x5638a2645350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f580124fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a2645cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f580124fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a259c150 con 0x5638a264b720 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f580124fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a2599b10 con 0x5638a2645890 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f580124fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a2595400 con 0x5638a264ae00 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 0x5638a2645350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 0x5638a2645350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50104/0 (socket says 172.21.15.16:50104) 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 -- 172.21.15.16:0/1287558829 learned_addr learned my addr 172.21.15.16:0/1287558829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ef59e700 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a2645cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57eed9d700 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264ae00 0x5638a264b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 -- 172.21.15.16:0/1287558829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264ae00 msgr2=0x5638a264b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264ae00 0x5638a264b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 -- 172.21.15.16:0/1287558829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 msgr2=0x5638a2645cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a2645cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 -- 172.21.15.16:0/1287558829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638a24163b0 con 0x5638a264b720 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.056+0000 7f57ee59c700 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 0x5638a2645350 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009d20 tx=0x7f57e4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90a4e74c32357d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f57d57fa700 1 -- 172.21.15.16:0/1287558829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4019630 con 0x5638a264b720 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f57d57fa700 1 -- 172.21.15.16:0/1287558829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57e4019c20 con 0x5638a264b720 2024-05-08T23:25:46.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f57d57fa700 1 -- 172.21.15.16:0/1287558829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400eb00 con 0x5638a264b720 2024-05-08T23:25:46.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 -- 172.21.15.16:0/1287558829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 msgr2=0x5638a2645350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 0x5638a2645350 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009d20 tx=0x7f57e4011720 comp rx=0 tx=0).stop 2024-05-08T23:25:46.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 -- 172.21.15.16:0/1287558829 shutdown_connections 2024-05-08T23:25:46.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a2645cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264ae00 0x5638a264b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 --2- 172.21.15.16:0/1287558829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264b720 0x5638a2645350 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 -- 172.21.15.16:0/1287558829 >> 172.21.15.16:0/1287558829 conn(0x5638a25a3210 msgr2=0x5638a263fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.057+0000 7f580124fb80 1 -- 172.21.15.16:0/1287558829 shutdown_connections 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 -- 172.21.15.16:0/1287558829 wait complete. 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 -- start start 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 Processor -- start 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a264ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a264efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a264f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a2531560 con 0x5638a264ae00 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a2531200 con 0x5638a264b720 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a24186a0 con 0x5638a2645890 2024-05-08T23:25:46.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f580124fb80 10 client.0 _ll_register_callbacks cb 0x5638a253ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a264efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a264efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50112/0 (socket says 172.21.15.16:50112) 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.058+0000 7f57ee59c700 1 -- 172.21.15.16:0/2835524631 learned_addr learned my addr 172.21.15.16:0/2835524631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a264ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ef59e700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a264f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ee59c700 1 -- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 msgr2=0x5638a264ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ee59c700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a264ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ee59c700 1 -- 172.21.15.16:0/2835524631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 msgr2=0x5638a264f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ee59c700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a264f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ee59c700 1 -- 172.21.15.16:0/2835524631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e4009980 con 0x5638a264ae00 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57ee59c700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a264efb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f57e401c040 tx=0x7f57e40199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923a8270b5aba78e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2835524631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4005450 con 0x5638a264ae00 2024-05-08T23:25:46.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2835524631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f57e4005ce0 con 0x5638a264ae00 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638a24ee270 con 0x5638a264ae00 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2835524631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e40055b0 con 0x5638a264ae00 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f580124fb80 10 client.16884 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.059+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5638a23c2b60 con 0x5638a264ae00 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2835524631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638a24ee270 con 0x5638a264ae00 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2835524631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57e4004b20 con 0x5638a264ae00 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f580124fb80 10 client.16884 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f580124fb80 10 client.16884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f580124fb80 20 client.16884 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f580124fb80 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a265f110 0x5638a2661530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f580124fb80 4 client.16884 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638a2614810 con 0x5638a265f110 2024-05-08T23:25:46.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.060+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a265f110 0x5638a2661530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f57d2ff5700 10 client.16884 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a265f110 0x5638a2661530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2ee7d02a2eb1e8a server_cookie=a67d45fcbb092874 in_seq=0 out_seq=0 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2835524631 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638a2614810 con 0x5638a265f110 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f57d2ff5700 10 client.16884 handle_command_reply: tid=0 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f580124fb80 1 client.16884 shutdown 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 msgr2=0x5638a264efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f580124fb80 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a264efb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f57e401c040 tx=0x7f57e40199b0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.061+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 shutdown_connections 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a265f110 0x5638a2661530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a264f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a264ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 --2- 172.21.15.16:0/2835524631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a264efb0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f57d2ff5700 0 client.16884 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 >> 172.21.15.16:0/2835524631 conn(0x5638a25a3210 msgr2=0x5638a263f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 shutdown_connections 2024-05-08T23:25:46.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 1 -- 172.21.15.16:0/2835524631 wait complete. 2024-05-08T23:25:46.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.062+0000 7f580124fb80 20 client.16884 trim_cache size 0 max 16384 2024-05-08T23:25:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 Processor -- start 2024-05-08T23:25:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 -- start start 2024-05-08T23:25:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a265ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a263fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264b720 0x5638a264efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a26515e0 con 0x5638a264ae00 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a263cdc0 con 0x5638a2645890 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f580124fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a2666740 con 0x5638a264b720 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a263fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a263fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50116/0 (socket says 172.21.15.16:50116) 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 -- 172.21.15.16:0/235086007 learned_addr learned my addr 172.21.15.16:0/235086007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ef59e700 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264b720 0x5638a264efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57eed9d700 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a265ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 -- 172.21.15.16:0/235086007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264b720 msgr2=0x5638a264efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264b720 0x5638a264efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 -- 172.21.15.16:0/235086007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 msgr2=0x5638a265ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a265ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.076+0000 7f57ee59c700 1 -- 172.21.15.16:0/235086007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638a255b560 con 0x5638a264ae00 2024-05-08T23:25:46.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.077+0000 7f57ee59c700 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a263fd20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009c70 tx=0x7f57e4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8953d08399b39c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.077+0000 7f57d5ffb700 1 -- 172.21.15.16:0/235086007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40037a0 con 0x5638a264ae00 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.077+0000 7f57d5ffb700 1 -- 172.21.15.16:0/235086007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57e4003e70 con 0x5638a264ae00 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.077+0000 7f57d5ffb700 1 -- 172.21.15.16:0/235086007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e402b3e0 con 0x5638a264ae00 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 -- 172.21.15.16:0/235086007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 msgr2=0x5638a263fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a263fd20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009c70 tx=0x7f57e4011b20 comp rx=0 tx=0).stop 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 -- 172.21.15.16:0/235086007 shutdown_connections 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a264b720 0x5638a264efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a2645890 0x5638a265ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 --2- 172.21.15.16:0/235086007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a263fd20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 -- 172.21.15.16:0/235086007 >> 172.21.15.16:0/235086007 conn(0x5638a25a3210 msgr2=0x5638a263c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 -- 172.21.15.16:0/235086007 shutdown_connections 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 -- 172.21.15.16:0/235086007 wait complete. 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 -- start start 2024-05-08T23:25:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.078+0000 7f580124fb80 1 Processor -- start 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a2655350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a2655890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a2655dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a26b4610 con 0x5638a264ae00 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a26b4750 con 0x5638a264b720 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638a26b4890 con 0x5638a2645890 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f580124fb80 10 client.0 _ll_register_callbacks cb 0x5638a253ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a2655890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a2655890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50124/0 (socket says 172.21.15.16:50124) 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 -- 172.21.15.16:0/2243919735 learned_addr learned my addr 172.21.15.16:0/2243919735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ef59e700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a2655dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a2655350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 -- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 msgr2=0x5638a2655350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a2655350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 -- 172.21.15.16:0/2243919735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 msgr2=0x5638a2655dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a2655dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 -- 172.21.15.16:0/2243919735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638a23c2b60 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a2655350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57ee59c700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a2655890 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009d20 tx=0x7f57e4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e824995c551228d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.079+0000 7f57d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2243919735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4005720 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57e4019c20 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2243919735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f57e4000e40 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2243919735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e40037a0 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 10 client.16890 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5638a241acd0 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2243919735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57e4019c20 con 0x5638a264ae00 2024-05-08T23:25:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2243919735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57e400e590 con 0x5638a264ae00 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 10 client.16890 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 10 client.16890 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 20 client.16890 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a26b5c20 0x5638a26b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 4 client.16890 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.080+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5638a258dec0 con 0x5638a26b5c20 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.081+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a26b5c20 0x5638a26b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.081+0000 7f57eed9d700 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a26b5c20 0x5638a26b8010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2ea833c98f59fe2 server_cookie=36108da21946222 in_seq=0 out_seq=0 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.081+0000 7f57d2ff5700 10 client.16890 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.081+0000 7f57d2ff5700 1 -- 172.21.15.16:0/2243919735 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5638a258dec0 con 0x5638a26b5c20 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.081+0000 7f57d2ff5700 10 client.16890 handle_command_reply: tid=0 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.081+0000 7f580124fb80 1 client.16890 shutdown 2024-05-08T23:25:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 msgr2=0x5638a2655890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a2655890 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009d20 tx=0x7f57e4005e50 comp rx=0 tx=0).stop 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 shutdown_connections 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5638a26b5c20 0x5638a26b8010 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638a264b720 0x5638a2655dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638a2645890 0x5638a2655350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57d2ff5700 0 client.16890 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 --2- 172.21.15.16:0/2243919735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638a264ae00 0x5638a2655890 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 >> 172.21.15.16:0/2243919735 conn(0x5638a25a3210 msgr2=0x5638a264bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 shutdown_connections 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 1 -- 172.21.15.16:0/2243919735 wait complete. 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f580124fb80 20 client.16890 trim_cache size 0 max 16384 2024-05-08T23:25:46.087 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:25:46.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57d80a1290 msgr2=0x7f57d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57d80a1290 0x7f57d80a3750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009990 tx=0x7f57e0019040 comp rx=0 tx=0).stop 2024-05-08T23:25:46.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 msgr2=0x7f57f01025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f01025a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ef30 tx=0x7f57e800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 shutdown_connections 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57d80a1290 0x7f57d80a3750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0105f90 0x7f57f01025a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f01000f0 0x7f57f0101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 --2- 172.21.15.16:0/3052991363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57f0100ab0 0x7f57f0102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.082+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 >> 172.21.15.16:0/3052991363 conn(0x7f57f0076ba0 msgr2=0x7f57f0069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.083+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 shutdown_connections 2024-05-08T23:25:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.083+0000 7f57f5095700 1 -- 172.21.15.16:0/3052991363 wait complete. 2024-05-08T23:25:46.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:25:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 -- 172.21.15.16:0/84481553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 msgr2=0x7f885010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 --2- 172.21.15.16:0/84481553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f885010de20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f88480099c0 tx=0x7f8848009cd0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 -- 172.21.15.16:0/84481553 shutdown_connections 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 --2- 172.21.15.16:0/84481553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f885010de20 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 --2- 172.21.15.16:0/84481553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 0x7f8850069bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 --2- 172.21.15.16:0/84481553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885006a190 0x7f8850105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 -- 172.21.15.16:0/84481553 >> 172.21.15.16:0/84481553 conn(0x7f8850076f10 msgr2=0x7f8850077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 -- 172.21.15.16:0/84481553 shutdown_connections 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.243+0000 7f8855954700 1 -- 172.21.15.16:0/84481553 wait complete. 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 Processor -- start 2024-05-08T23:25:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 -- start start 2024-05-08T23:25:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 0x7f8850197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f88501977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885019bb60 0x7f885019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019c5e0 con 0x7f88501062a0 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019c750 con 0x7f88500697e0 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f8855954700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019c910 con 0x7f885019bb60 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.244+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f88501977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 0x7f8850197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884f7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885019bb60 0x7f885019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f88501977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50152/0 (socket says 172.21.15.16:50152) 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884e7fc700 1 -- 172.21.15.16:0/2831449581 learned_addr learned my addr 172.21.15.16:0/2831449581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 -- 172.21.15.16:0/2831449581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885019bb60 msgr2=0x7f885019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885019bb60 0x7f885019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 -- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 msgr2=0x7f88501977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f88501977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 -- 172.21.15.16:0/2831449581 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8840009c40 con 0x7f88500697e0 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884e7fc700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f88501977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f884effd700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 0x7f8850197280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f884000efc0 tx=0x7f884000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f8854952700 1 -- 172.21.15.16:0/2831449581 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840018070 con 0x7f88500697e0 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f8854952700 1 -- 172.21.15.16:0/2831449581 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884000f040 con 0x7f88500697e0 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88480095f0 con 0x7f88500697e0 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f8854952700 1 -- 172.21.15.16:0/2831449581 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840004280 con 0x7f88500697e0 2024-05-08T23:25:46.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.245+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88501a41b0 con 0x7f88500697e0 2024-05-08T23:25:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.248+0000 7f8854952700 1 -- 172.21.15.16:0/2831449581 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f884000ca20 con 0x7f88500697e0 2024-05-08T23:25:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.248+0000 7f8854952700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88380a1310 0x7f88380a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.248+0000 7f8854952700 1 -- 172.21.15.16:0/2831449581 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f88400ccad0 con 0x7f88500697e0 2024-05-08T23:25:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.248+0000 7f884e7fc700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88380a1310 0x7f88380a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.249+0000 7f884e7fc700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88380a1310 0x7f88380a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88501987d0 tx=0x7f8844009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 Processor -- start 2024-05-08T23:25:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 -- start start 2024-05-08T23:25:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d69720 0x55bc35d63350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d63cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35cba150 con 0x55bc35d68e00 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35cb7b10 con 0x55bc35d69720 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35cb3400 con 0x55bc35d63890 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f884effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f884effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50168/0 (socket says 172.21.15.16:50168) 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f884effd700 1 -- 172.21.15.16:0/4245004788 learned_addr learned my addr 172.21.15.16:0/4245004788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f884f7fe700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d63cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.251+0000 7f884e7fc700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d69720 0x55bc35d63350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884effd700 1 -- 172.21.15.16:0/4245004788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 msgr2=0x55bc35d63cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884effd700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d63cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884effd700 1 -- 172.21.15.16:0/4245004788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d69720 msgr2=0x55bc35d63350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884effd700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d69720 0x55bc35d63350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884effd700 1 -- 172.21.15.16:0/4245004788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc35b343b0 con 0x55bc35d68e00 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884e7fc700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d69720 0x55bc35d63350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884f7fe700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d63cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f884effd700 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d691e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f88400963a0 tx=0x7f8840099780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1db8eb643826e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f8835ffb700 1 -- 172.21.15.16:0/4245004788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840084040 con 0x55bc35d68e00 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f8835ffb700 1 -- 172.21.15.16:0/4245004788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88400b7e90 con 0x55bc35d68e00 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.252+0000 7f8835ffb700 1 -- 172.21.15.16:0/4245004788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88400a1d20 con 0x55bc35d68e00 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 -- 172.21.15.16:0/4245004788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 msgr2=0x55bc35d691e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d691e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f88400963a0 tx=0x7f8840099780 comp rx=0 tx=0).stop 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 -- 172.21.15.16:0/4245004788 shutdown_connections 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d63cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d691e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/4245004788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d69720 0x55bc35d63350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 -- 172.21.15.16:0/4245004788 >> 172.21.15.16:0/4245004788 conn(0x55bc35cc1210 msgr2=0x55bc35d5ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 -- 172.21.15.16:0/4245004788 shutdown_connections 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 -- 172.21.15.16:0/4245004788 wait complete. 2024-05-08T23:25:46.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.253+0000 7f8861b0eb80 1 -- start start 2024-05-08T23:25:46.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 Processor -- start 2024-05-08T23:25:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 0x55bc35d6cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d6d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35d6d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35c4f560 con 0x55bc35d63890 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35c4f200 con 0x55bc35d68e00 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35b366a0 con 0x55bc35d69720 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f8861b0eb80 10 client.0 _ll_register_callbacks cb 0x55bc35c58b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f884effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 0x55bc35d6cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f884effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 0x55bc35d6cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50172/0 (socket says 172.21.15.16:50172) 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f884effd700 1 -- 172.21.15.16:0/1146278309 learned_addr learned my addr 172.21.15.16:0/1146278309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f884e7fc700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d6d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.254+0000 7f884f7fe700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35d6d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f884effd700 1 -- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 msgr2=0x55bc35d6d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f884effd700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35d6d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f884effd700 1 -- 172.21.15.16:0/1146278309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 msgr2=0x55bc35d6d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f884effd700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d6d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f884effd700 1 -- 172.21.15.16:0/1146278309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8844017070 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f884effd700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 0x55bc35d6cb70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f88400ca4e0 tx=0x7f8840099dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87095c78d30ac7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f882effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f882effd700 1 -- 172.21.15.16:0/1146278309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840084040 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f882effd700 1 -- 172.21.15.16:0/1146278309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f884009a030 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc35c0c270 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f882effd700 1 -- 172.21.15.16:0/1146278309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88400b2250 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f8861b0eb80 10 client.16902 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f88501a41b0 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.255+0000 7f882effd700 1 -- 172.21.15.16:0/1146278309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f884007c080 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f882effd700 1 -- 172.21.15.16:0/1146278309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88400a1890 con 0x55bc35d63890 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f8861b0eb80 10 client.16902 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f8861b0eb80 10 client.16902 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f8861b0eb80 20 client.16902 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:46.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35d7cd50 0x55bc35d7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f8861b0eb80 4 client.16902 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc35d32810 con 0x55bc35d7cd50 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f884e7fc700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35d7cd50 0x55bc35d7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f884e7fc700 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35d7cd50 0x55bc35d7f140 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b4f47f4d6bf8643 server_cookie=acdb20bc991aaa7c in_seq=0 out_seq=0 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.256+0000 7f882effd700 10 client.16902 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f882effd700 1 -- 172.21.15.16:0/1146278309 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc35d32810 con 0x55bc35d7cd50 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f882effd700 10 client.16902 handle_command_reply: tid=0 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 client.16902 shutdown 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 msgr2=0x55bc35d6cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 0x55bc35d6cb70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f88400ca4e0 tx=0x7f8840099dc0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 shutdown_connections 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35d6d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d63890 0x55bc35d6cb70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35d7cd50 0x55bc35d7f140 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/1146278309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d6d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f882effd700 0 client.16902 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 >> 172.21.15.16:0/1146278309 conn(0x55bc35cc1210 msgr2=0x55bc35d5d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 shutdown_connections 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.257+0000 7f8861b0eb80 1 -- 172.21.15.16:0/1146278309 wait complete. 2024-05-08T23:25:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.258+0000 7f8861b0eb80 20 client.16902 trim_cache size 0 max 16384 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 Processor -- start 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 -- start start 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d5dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d69720 0x55bc35d6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35d5f680 con 0x55bc35d69720 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35d636d0 con 0x55bc35d68e00 2024-05-08T23:25:46.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.272+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35d84320 con 0x55bc35d63890 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d69720 0x55bc35d6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d5dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884effd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d69720 0x55bc35d6d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50184/0 (socket says 172.21.15.16:50184) 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d5dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:48904/0 (socket says 172.21.15.16:48904) 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884effd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d691e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:44438/0 (socket says 172.21.15.16:44438) 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884f7fe700 1 -- 172.21.15.16:0/3142822840 learned_addr learned my addr 172.21.15.16:0/3142822840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 -- 172.21.15.16:0/3142822840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 msgr2=0x55bc35d691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 -- 172.21.15.16:0/3142822840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d69720 msgr2=0x55bc35d6d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d69720 0x55bc35d6d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 -- 172.21.15.16:0/3142822840 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc35ae0b60 con 0x55bc35d68e00 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f884e7fc700 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d5dd20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8844004140 tx=0x7f8844014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d0e301fe8bb0b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f88367fc700 1 -- 172.21.15.16:0/3142822840 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844024040 con 0x55bc35d68e00 2024-05-08T23:25:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f88367fc700 1 -- 172.21.15.16:0/3142822840 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88440298b0 con 0x55bc35d68e00 2024-05-08T23:25:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.273+0000 7f88367fc700 1 -- 172.21.15.16:0/3142822840 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844005020 con 0x55bc35d68e00 2024-05-08T23:25:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3142822840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 msgr2=0x55bc35d5dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d5dd20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8844004140 tx=0x7f8844014000 comp rx=0 tx=0).stop 2024-05-08T23:25:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3142822840 shutdown_connections 2024-05-08T23:25:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d69720 0x55bc35d6d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d63890 0x55bc35d691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3142822840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d68e00 0x55bc35d5dd20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3142822840 >> 172.21.15.16:0/3142822840 conn(0x55bc35cc1210 msgr2=0x55bc35d5a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3142822840 shutdown_connections 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.274+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3142822840 wait complete. 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.275+0000 7f8861b0eb80 1 -- start start 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.275+0000 7f8861b0eb80 1 Processor -- start 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.275+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d63890 0x55bc35d72150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.275+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d72690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.275+0000 7f8861b0eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35dd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.275+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35d72f50 con 0x55bc35d68e00 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35dd2730 con 0x55bc35d63890 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f8861b0eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc35dd2870 con 0x55bc35d69720 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f8861b0eb80 10 client.0 _ll_register_callbacks cb 0x55bc35c58b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d72690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d72690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50190/0 (socket says 172.21.15.16:50190) 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 -- 172.21.15.16:0/3286560966 learned_addr learned my addr 172.21.15.16:0/3286560966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884effd700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d63890 0x55bc35d72150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884f7fe700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35dd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 -- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 msgr2=0x55bc35dd21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35dd21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 -- 172.21.15.16:0/3286560966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d63890 msgr2=0x55bc35d72150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d63890 0x55bc35d72150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 -- 172.21.15.16:0/3286560966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8844003680 con 0x55bc35d68e00 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884f7fe700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35dd21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f884e7fc700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d72690 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f884402b000 tx=0x7f8844004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123613546f862842 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f882effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.276+0000 7f882effd700 1 -- 172.21.15.16:0/3286560966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844024040 con 0x55bc35d68e00 2024-05-08T23:25:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f882effd700 1 -- 172.21.15.16:0/3286560966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f8844029620 con 0x55bc35d68e00 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88400b7e90 con 0x55bc35d68e00 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f882effd700 1 -- 172.21.15.16:0/3286560966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8844033480 con 0x55bc35d68e00 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 10 client.16908 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bc35c79560 con 0x55bc35d68e00 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f882effd700 1 -- 172.21.15.16:0/3286560966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88400b7e90 con 0x55bc35d68e00 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f882effd700 1 -- 172.21.15.16:0/3286560966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f884401e6e0 con 0x55bc35d68e00 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 10 client.16908 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 20 client.16908 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35dd3fc0 0x55bc35dd63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 4 client.16908 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc35cabec0 con 0x55bc35dd3fc0 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.277+0000 7f884effd700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35dd3fc0 0x55bc35dd63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f884effd700 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35dd3fc0 0x55bc35dd63e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c0a68b4ee984854 server_cookie=9bf7fc6d7bdd6bae in_seq=0 out_seq=0 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f882effd700 10 client.16908 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f882effd700 1 -- 172.21.15.16:0/3286560966 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc35cabec0 con 0x55bc35dd3fc0 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f882effd700 10 client.16908 handle_command_reply: tid=0 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 client.16908 shutdown 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 msgr2=0x55bc35d72690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d72690 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f884402b000 tx=0x7f8844004310 comp rx=0 tx=0).stop 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 shutdown_connections 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc35d69720 0x55bc35dd21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55bc35dd3fc0 0x55bc35dd63e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55bc35d63890 0x55bc35d72150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 --2- 172.21.15.16:0/3286560966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc35d68e00 0x55bc35d72690 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 >> 172.21.15.16:0/3286560966 conn(0x55bc35cc1210 msgr2=0x55bc35d6b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f882effd700 0 client.16908 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:25:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 shutdown_connections 2024-05-08T23:25:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.278+0000 7f8861b0eb80 1 -- 172.21.15.16:0/3286560966 wait complete. 2024-05-08T23:25:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8861b0eb80 20 client.16908 trim_cache size 0 max 16384 2024-05-08T23:25:46.284 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:25:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88380a1310 msgr2=0x7f88380a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88380a1310 0x7f88380a37d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f88501987d0 tx=0x7f8844009260 comp rx=0 tx=0).stop 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 msgr2=0x7f8850197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 0x7f8850197280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f884000efc0 tx=0x7f884000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 shutdown_connections 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f88380a1310 0x7f88380a37d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885019bb60 0x7f885019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500697e0 0x7f8850197280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 --2- 172.21.15.16:0/2831449581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88501062a0 0x7f88501977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 >> 172.21.15.16:0/2831449581 conn(0x7f8850076f10 msgr2=0x7f88500fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 shutdown_connections 2024-05-08T23:25:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.279+0000 7f8855954700 1 -- 172.21.15.16:0/2831449581 wait complete. 2024-05-08T23:25:46.291 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3691691396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:25:46.291 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:46 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:25:46.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:25:46.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3691691396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:25:46.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:46 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:25:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 -- 172.21.15.16:0/447426411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0180068050 msgr2=0x7f0180068430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 --2- 172.21.15.16:0/447426411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0180068050 0x7f0180068430 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0170009990 tx=0x7f0170009ca0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 -- 172.21.15.16:0/447426411 shutdown_connections 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 --2- 172.21.15.16:0/447426411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f018010d940 0x7f018010fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 --2- 172.21.15.16:0/447426411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0180068050 0x7f0180068430 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 --2- 172.21.15.16:0/447426411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068970 0x7f018010d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 -- 172.21.15.16:0/447426411 >> 172.21.15.16:0/447426411 conn(0x7f0180076f40 msgr2=0x7f0180077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 -- 172.21.15.16:0/447426411 shutdown_connections 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.441+0000 7f018648b700 1 -- 172.21.15.16:0/447426411 wait complete. 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 Processor -- start 2024-05-08T23:25:46.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 -- start start 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 0x7f0180199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0180068970 0x7f0180199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018010d940 0x7f018019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018019e430 con 0x7f0180068050 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018019e5a0 con 0x7f018010d940 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f018648b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018019e760 con 0x7f0180068970 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f017ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 0x7f0180199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f017ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 0x7f0180199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51516/0 (socket says 172.21.15.16:51516) 2024-05-08T23:25:46.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f017ffff700 1 -- 172.21.15.16:0/1330050819 learned_addr learned my addr 172.21.15.16:0/1330050819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f017f7fe700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0180068970 0x7f0180199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.442+0000 7f0184a28700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018010d940 0x7f018019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017ffff700 1 -- 172.21.15.16:0/1330050819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0180068970 msgr2=0x7f0180199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017ffff700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0180068970 0x7f0180199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017ffff700 1 -- 172.21.15.16:0/1330050819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018010d940 msgr2=0x7f018019ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017ffff700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018010d940 0x7f018019ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017ffff700 1 -- 172.21.15.16:0/1330050819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01700095f0 con 0x7f0180068050 2024-05-08T23:25:46.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017ffff700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 0x7f0180199510 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0170004730 tx=0x7f0170004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017d7fa700 1 -- 172.21.15.16:0/1330050819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017001d070 con 0x7f0180068050 2024-05-08T23:25:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017d7fa700 1 -- 172.21.15.16:0/1330050819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f017000bed0 con 0x7f0180068050 2024-05-08T23:25:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018019e9e0 con 0x7f0180068050 2024-05-08T23:25:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f017d7fa700 1 -- 172.21.15.16:0/1330050819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170022de0 con 0x7f0180068050 2024-05-08T23:25:46.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.443+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01801a6280 con 0x7f0180068050 2024-05-08T23:25:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.446+0000 7f017d7fa700 1 -- 172.21.15.16:0/1330050819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0170022570 con 0x7f0180068050 2024-05-08T23:25:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.447+0000 7f017d7fa700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01680a1290 0x7f01680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.447+0000 7f017f7fe700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01680a1290 0x7f01680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.447+0000 7f017d7fa700 1 -- 172.21.15.16:0/1330050819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f01700ce3a0 con 0x7f0180068050 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.447+0000 7f017f7fe700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01680a1290 0x7f01680a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f018019aa60 tx=0x7f0174009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 Processor -- start 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 -- start start 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f16441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1595150 con 0x5574f1643e00 2024-05-08T23:25:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1592b10 con 0x5574f163e890 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0192645b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f158e400 con 0x5574f1644720 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f017ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f16441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f017ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f16441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51520/0 (socket says 172.21.15.16:51520) 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f017ffff700 1 -- 172.21.15.16:0/3603761488 learned_addr learned my addr 172.21.15.16:0/3603761488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f017f7fe700 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.449+0000 7f0184a28700 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017ffff700 1 -- 172.21.15.16:0/3603761488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 msgr2=0x5574f163e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017ffff700 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017ffff700 1 -- 172.21.15.16:0/3603761488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 msgr2=0x5574f163ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017ffff700 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017ffff700 1 -- 172.21.15.16:0/3603761488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f140f3b0 con 0x5574f1643e00 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017f7fe700 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f017ffff700 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f16441e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0170097820 tx=0x7f017009aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac5fadb0c98f16b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f01667fc700 1 -- 172.21.15.16:0/3603761488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009b320 con 0x5574f1643e00 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f01667fc700 1 -- 172.21.15.16:0/3603761488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f017009f150 con 0x5574f1643e00 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.450+0000 7f01667fc700 1 -- 172.21.15.16:0/3603761488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009b580 con 0x5574f1643e00 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 -- 172.21.15.16:0/3603761488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 msgr2=0x5574f16441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f16441e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0170097820 tx=0x7f017009aee0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 -- 172.21.15.16:0/3603761488 shutdown_connections 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f16441e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 --2- 172.21.15.16:0/3603761488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 -- 172.21.15.16:0/3603761488 >> 172.21.15.16:0/3603761488 conn(0x5574f159c210 msgr2=0x5574f1638df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 -- 172.21.15.16:0/3603761488 shutdown_connections 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 -- 172.21.15.16:0/3603761488 wait complete. 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.451+0000 7f0192645b80 1 -- start start 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 Processor -- start 2024-05-08T23:25:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f1647aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f1647fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f1648520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f152a560 con 0x5574f1643e00 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f152a200 con 0x5574f163e890 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f14116a0 con 0x5574f1644720 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0192645b80 10 client.0 _ll_register_callbacks cb 0x5574f1533b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f017f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f1647fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f017ffff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f1647aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f017f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f1647fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51536/0 (socket says 172.21.15.16:51536) 2024-05-08T23:25:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f017f7fe700 1 -- 172.21.15.16:0/497939355 learned_addr learned my addr 172.21.15.16:0/497939355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.452+0000 7f0184a28700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f1648520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017ffff700 1 -- 172.21.15.16:0/497939355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 msgr2=0x5574f1648520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017ffff700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f1648520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017ffff700 1 -- 172.21.15.16:0/497939355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 msgr2=0x5574f1647fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017ffff700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f1647fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017ffff700 1 -- 172.21.15.16:0/497939355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0174017070 con 0x5574f163e890 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017f7fe700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f1647fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f017ffff700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f1647aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0170005950 tx=0x7f01700cb720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9c3fcbafd0956a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f015f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f015f7fe700 1 -- 172.21.15.16:0/497939355 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009aa10 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f015f7fe700 1 -- 172.21.15.16:0/497939355 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f01700b3e40 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574f14e7270 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f015f7fe700 1 -- 172.21.15.16:0/497939355 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017009b320 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f0192645b80 10 client.26540 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f01801a6280 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.453+0000 7f015f7fe700 1 -- 172.21.15.16:0/497939355 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01700a7020 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f015f7fe700 1 -- 172.21.15.16:0/497939355 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01700b3020 con 0x5574f163e890 2024-05-08T23:25:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f0192645b80 10 client.26540 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f0192645b80 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f0192645b80 20 client.26540 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f0192645b80 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16579b0 0x5574f1659da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f0192645b80 4 client.26540 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574f160d810 con 0x5574f16579b0 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f017f7fe700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16579b0 0x5574f1659da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.454+0000 7f015f7fe700 10 client.26540 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f017f7fe700 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16579b0 0x5574f1659da0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b5094f2c1ad1ec server_cookie=d3c46199b4b96e3a in_seq=0 out_seq=0 2024-05-08T23:25:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f015f7fe700 1 -- 172.21.15.16:0/497939355 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574f160d810 con 0x5574f16579b0 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f015f7fe700 10 client.26540 handle_command_reply: tid=0 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 client.26540 shutdown 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 msgr2=0x5574f1647aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f1647aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0170005950 tx=0x7f01700cb720 comp rx=0 tx=0).stop 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 shutdown_connections 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f1648520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16579b0 0x5574f1659da0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f1647aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f0192645b80 1 --2- 172.21.15.16:0/497939355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1643e00 0x5574f1647fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.455+0000 7f015f7fe700 0 client.26540 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.456+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 >> 172.21.15.16:0/497939355 conn(0x5574f159c210 msgr2=0x5574f16385a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.456+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 shutdown_connections 2024-05-08T23:25:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.456+0000 7f0192645b80 1 -- 172.21.15.16:0/497939355 wait complete. 2024-05-08T23:25:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.456+0000 7f0192645b80 20 client.26540 trim_cache size 0 max 16384 2024-05-08T23:25:46.467 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3691691396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:25:46.467 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:46 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 Processor -- start 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 -- start start 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 0x5574f1659da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 0x5574f1638d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f163aad0 con 0x5574f1644720 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f165ef40 con 0x5574f163e890 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.470+0000 7f0192645b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1633800 con 0x5574f1643e00 2024-05-08T23:25:46.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 0x5574f1638d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f017f7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 0x5574f1659da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 0x5574f1638d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51552/0 (socket says 172.21.15.16:51552) 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f017f7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 0x5574f1659da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:35436/0 (socket says 172.21.15.16:35436) 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 -- 172.21.15.16:0/2699712043 learned_addr learned my addr 172.21.15.16:0/2699712043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f017ffff700 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 -- 172.21.15.16:0/2699712043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 msgr2=0x5574f1659da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 0x5574f1659da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 -- 172.21.15.16:0/2699712043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 msgr2=0x5574f163e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 -- 172.21.15.16:0/2699712043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f13bbb60 con 0x5574f1644720 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f017f7fe700 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 0x5574f1659da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f017ffff700 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.471+0000 7f0184a28700 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 0x5574f1638d20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0178009500 tx=0x7f0178009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137d07129cc2d87d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0166ffd700 1 -- 172.21.15.16:0/2699712043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017800e040 con 0x5574f1644720 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0166ffd700 1 -- 172.21.15.16:0/2699712043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f017800f040 con 0x5574f1644720 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0166ffd700 1 -- 172.21.15.16:0/2699712043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178017e10 con 0x5574f1644720 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 -- 172.21.15.16:0/2699712043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 msgr2=0x5574f1638d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 0x5574f1638d20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0178009500 tx=0x7f0178009920 comp rx=0 tx=0).stop 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 -- 172.21.15.16:0/2699712043 shutdown_connections 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f1644720 0x5574f1638d20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f163e890 0x5574f163e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 --2- 172.21.15.16:0/2699712043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1643e00 0x5574f1659da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.472+0000 7f0192645b80 1 -- 172.21.15.16:0/2699712043 >> 172.21.15.16:0/2699712043 conn(0x5574f159c210 msgr2=0x5574f163ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.473+0000 7f0192645b80 1 -- 172.21.15.16:0/2699712043 shutdown_connections 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.473+0000 7f0192645b80 1 -- 172.21.15.16:0/2699712043 wait complete. 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.473+0000 7f0192645b80 1 -- start start 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.473+0000 7f0192645b80 1 Processor -- start 2024-05-08T23:25:46.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 0x5574f164f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f1643e00 0x5574f163f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f16405a0 con 0x5574f163e890 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1640710 con 0x5574f1643e00 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f16408d0 con 0x5574f1644720 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0192645b80 10 client.0 _ll_register_callbacks cb 0x5574f1533b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 0x5574f164f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 0x5574f164f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51564/0 (socket says 172.21.15.16:51564) 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 -- 172.21.15.16:0/3379314101 learned_addr learned my addr 172.21.15.16:0/3379314101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017f7fe700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f1643e00 0x5574f163f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0184a28700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 -- 172.21.15.16:0/3379314101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 msgr2=0x5574f163fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 -- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f1643e00 msgr2=0x5574f163f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f1643e00 0x5574f163f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 -- 172.21.15.16:0/3379314101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01801a6280 con 0x5574f163e890 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f0184a28700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017f7fe700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f1643e00 0x5574f163f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.474+0000 7f017ffff700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 0x5574f164f270 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f01700cbdb0 tx=0x7f017009b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a21f00e2ba78e9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f015f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f015f7fe700 1 -- 172.21.15.16:0/3379314101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01700b7070 con 0x5574f163e890 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f015f7fe700 1 -- 172.21.15.16:0/3379314101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f01700b3e40 con 0x5574f163e890 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f017009f150 con 0x5574f163e890 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f015f7fe700 1 -- 172.21.15.16:0/3379314101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01700a3280 con 0x5574f163e890 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 10 client.16926 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f0174004040 con 0x5574f163e890 2024-05-08T23:25:46.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f015f7fe700 1 -- 172.21.15.16:0/3379314101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01700ae030 con 0x5574f163e890 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f015f7fe700 1 -- 172.21.15.16:0/3379314101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01700b3040 con 0x5574f163e890 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 10 client.16926 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 20 client.16926 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16ae100 0x5574f16b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 4 client.16926 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.475+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574f1586ec0 con 0x5574f16ae100 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.476+0000 7f017f7fe700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16ae100 0x5574f16b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.476+0000 7f015f7fe700 10 client.16926 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.476+0000 7f017f7fe700 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16ae100 0x5574f16b04f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78ed7e9c7ca432e server_cookie=3bb9bdb84fc70165 in_seq=0 out_seq=0 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.476+0000 7f015f7fe700 1 -- 172.21.15.16:0/3379314101 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574f1586ec0 con 0x5574f16ae100 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.476+0000 7f015f7fe700 10 client.16926 handle_command_reply: tid=0 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.476+0000 7f0192645b80 1 client.16926 shutdown 2024-05-08T23:25:46.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 msgr2=0x5574f164f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 0x5574f164f270 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f01700cbdb0 tx=0x7f017009b830 comp rx=0 tx=0).stop 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 shutdown_connections 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5574f1644720 0x5574f163fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5574f163e890 0x5574f164f270 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5574f16ae100 0x5574f16b04f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 --2- 172.21.15.16:0/3379314101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5574f1643e00 0x5574f163f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f015f7fe700 0 client.16926 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 >> 172.21.15.16:0/3379314101 conn(0x5574f159c210 msgr2=0x5574f16462a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 shutdown_connections 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 1 -- 172.21.15.16:0/3379314101 wait complete. 2024-05-08T23:25:46.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f0192645b80 20 client.16926 trim_cache size 0 max 16384 2024-05-08T23:25:46.485 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:25:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01680a1290 msgr2=0x7f01680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f018648b700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01680a1290 0x7f01680a3750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f018019aa60 tx=0x7f0174009260 comp rx=0 tx=0).stop 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 msgr2=0x7f0180199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.477+0000 7f018648b700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 0x7f0180199510 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0170004730 tx=0x7f0170004760 comp rx=0 tx=0).stop 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 shutdown_connections 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01680a1290 0x7f01680a3750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f018010d940 0x7f018019ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0180068050 0x7f0180199510 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 --2- 172.21.15.16:0/1330050819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0180068970 0x7f0180199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 >> 172.21.15.16:0/1330050819 conn(0x7f0180076f40 msgr2=0x7f018010f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 shutdown_connections 2024-05-08T23:25:46.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:46.478+0000 7f018648b700 1 -- 172.21.15.16:0/1330050819 wait complete. 2024-05-08T23:25:47.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:47 smithi189 ceph-mon[46919]: pgmap v189: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:25:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:25:47.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:25:47.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:47 smithi155 ceph-mon[47105]: pgmap v189: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:47.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:25:47.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:25:47.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:25:47.493 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:25:47.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:25:47.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:47 smithi016 ceph-mon[42541]: pgmap v189: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:47.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:25:47.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:25:47.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:25:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 -- 172.21.15.16:0/1763226425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068970 msgr2=0x7fe11810d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 --2- 172.21.15.16:0/1763226425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068970 0x7fe11810d400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe108009990 tx=0x7fe108009ca0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 -- 172.21.15.16:0/1763226425 shutdown_connections 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 --2- 172.21.15.16:0/1763226425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 0x7fe11810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 --2- 172.21.15.16:0/1763226425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068050 0x7fe118068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 --2- 172.21.15.16:0/1763226425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068970 0x7fe11810d400 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 -- 172.21.15.16:0/1763226425 >> 172.21.15.16:0/1763226425 conn(0x7fe118076f40 msgr2=0x7fe118077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 -- 172.21.15.16:0/1763226425 shutdown_connections 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.645+0000 7fe12024a700 1 -- 172.21.15.16:0/1763226425 wait complete. 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 Processor -- start 2024-05-08T23:25:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 -- start start 2024-05-08T23:25:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 0x7fe118102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068970 0x7fe1181025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 0x7fe118102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181a9b50 con 0x7fe118068050 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181a9c90 con 0x7fe118068970 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe12024a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181a9e00 con 0x7fe11810d940 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 0x7fe118102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 0x7fe118102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51572/0 (socket says 172.21.15.16:51572) 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 -- 172.21.15.16:0/2977372440 learned_addr learned my addr 172.21.15.16:0/2977372440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068970 0x7fe1181025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 0x7fe118102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 -- 172.21.15.16:0/2977372440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 msgr2=0x7fe118102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 0x7fe118102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 -- 172.21.15.16:0/2977372440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068970 msgr2=0x7fe1181025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068970 0x7fe1181025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11dfe6700 1 -- 172.21.15.16:0/2977372440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080095f0 con 0x7fe118068050 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068970 0x7fe1181025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.646+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 0x7fe118102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.647+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 0x7fe118102070 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9a0 tx=0x7fe11400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.647+0000 7fe10effd700 1 -- 172.21.15.16:0/2977372440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114009790 con 0x7fe118068050 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.647+0000 7fe10effd700 1 -- 172.21.15.16:0/2977372440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe11400cbd0 con 0x7fe118068050 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.647+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1181aa0e0 con 0x7fe118068050 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.647+0000 7fe10effd700 1 -- 172.21.15.16:0/2977372440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114018680 con 0x7fe118068050 2024-05-08T23:25:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.647+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1181aa640 con 0x7fe118068050 2024-05-08T23:25:47.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.651+0000 7fe10effd700 1 -- 172.21.15.16:0/2977372440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe11400cd40 con 0x7fe118068050 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.651+0000 7fe10effd700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1040a1310 0x7fe1040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.651+0000 7fe10effd700 1 -- 172.21.15.16:0/2977372440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe114014070 con 0x7fe118068050 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.651+0000 7fe12c404b80 1 Processor -- start 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.651+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1040a1310 0x7fe1040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 -- start start 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135555e80 0x558135556260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581355567a0 0x5581355503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135550d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581354a71d0 con 0x558135550910 2024-05-08T23:25:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581354a4b90 con 0x558135555e80 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe12c404b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581354a0480 con 0x5581355567a0 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135550d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135550d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51576/0 (socket says 172.21.15.16:51576) 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1182611864 learned_addr learned my addr 172.21.15.16:0/1182611864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1040a1310 0x7fe1040a37d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1181037a0 tx=0x7fe108005960 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581355567a0 0x5581355503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135555e80 0x558135556260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1182611864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581355567a0 msgr2=0x5581355503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581355567a0 0x5581355503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1182611864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135555e80 msgr2=0x558135556260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135555e80 0x558135556260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.652+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1182611864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558135321430 con 0x558135550910 2024-05-08T23:25:47.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.653+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135550d50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe110009ca0 tx=0x7fe110007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad30c1a20352775 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.653+0000 7fe103fff700 1 -- 172.21.15.16:0/1182611864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110014600 con 0x558135550910 2024-05-08T23:25:47.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.653+0000 7fe103fff700 1 -- 172.21.15.16:0/1182611864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe110007e40 con 0x558135550910 2024-05-08T23:25:47.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.653+0000 7fe103fff700 1 -- 172.21.15.16:0/1182611864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110013a70 con 0x558135550910 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 -- 172.21.15.16:0/1182611864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 msgr2=0x558135550d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135550d50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe110009ca0 tx=0x7fe110007500 comp rx=0 tx=0).stop 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 -- 172.21.15.16:0/1182611864 shutdown_connections 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135550d50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135555e80 0x558135556260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1182611864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581355567a0 0x5581355503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 -- 172.21.15.16:0/1182611864 >> 172.21.15.16:0/1182611864 conn(0x5581354ae290 msgr2=0x55813554ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 -- 172.21.15.16:0/1182611864 shutdown_connections 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 -- 172.21.15.16:0/1182611864 wait complete. 2024-05-08T23:25:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 -- start start 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.654+0000 7fe12c404b80 1 Processor -- start 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135559be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x55813555a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581355567a0 0x55813555a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813543c5e0 con 0x558135550910 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813543c280 con 0x5581355567a0 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558135323720 con 0x558135555e80 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe12c404b80 10 client.0 _ll_register_callbacks cb 0x558135445bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135559be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135559be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51580/0 (socket says 172.21.15.16:51580) 2024-05-08T23:25:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 -- 172.21.15.16:0/1565388709 learned_addr learned my addr 172.21.15.16:0/1565388709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x55813555a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581355567a0 0x55813555a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 -- 172.21.15.16:0/1565388709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 msgr2=0x55813555a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x55813555a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 -- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581355567a0 msgr2=0x55813555a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581355567a0 0x55813555a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11dfe6700 1 -- 172.21.15.16:0/1565388709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe110009900 con 0x558135550910 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.655+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581355567a0 0x55813555a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135559be0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ca9a0 tx=0x7fe11409a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21185eed09d7c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe1017fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe1017fa700 1 -- 172.21.15.16:0/1565388709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140b6070 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe1017fa700 1 -- 172.21.15.16:0/1565388709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe1140a2780 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581353f92f0 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe1017fa700 1 -- 172.21.15.16:0/1565388709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1140986a0 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 10 client.16938 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5581352cdbe0 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe1017fa700 1 -- 172.21.15.16:0/1565388709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5581353f92f0 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe1017fa700 1 -- 172.21.15.16:0/1565388709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe1140a22f0 con 0x558135550910 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 10 client.16938 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 20 client.16938 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55813556a1b0 0x55813556c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 4 client.16938 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.656+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55813551f890 con 0x55813556a1b0 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.657+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55813556a1b0 0x55813556c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.657+0000 7fe11d7e5700 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55813556a1b0 0x55813556c5a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a402a8c0c53a5b3 server_cookie=8a5f6bc6d3c1283 in_seq=0 out_seq=0 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.657+0000 7fe1017fa700 10 client.16938 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe1017fa700 1 -- 172.21.15.16:0/1565388709 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55813551f890 con 0x55813556a1b0 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe1017fa700 10 client.16938 handle_command_reply: tid=0 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 client.16938 shutdown 2024-05-08T23:25:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 msgr2=0x558135559be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135559be0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe1140ca9a0 tx=0x7fe11409a3d0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 shutdown_connections 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55813556a1b0 0x55813556c5a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5581355567a0 0x55813555a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558135550910 0x558135559be0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1565388709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x55813555a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe1017fa700 0 client.16938 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 >> 172.21.15.16:0/1565388709 conn(0x5581354ae290 msgr2=0x55813554a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 shutdown_connections 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 1 -- 172.21.15.16:0/1565388709 wait complete. 2024-05-08T23:25:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.658+0000 7fe12c404b80 20 client.16938 trim_cache size 0 max 16384 2024-05-08T23:25:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 Processor -- start 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 -- start start 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x5581355503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x558135555550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x55813554ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813551f890 con 0x5581355567a0 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581355679a0 con 0x558135550910 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe12c404b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813555b1e0 con 0x558135555e80 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x55813554ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x55813554ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51588/0 (socket says 172.21.15.16:51588) 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 -- 172.21.15.16:0/3248613558 learned_addr learned my addr 172.21.15.16:0/3248613558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x5581355503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 -- 172.21.15.16:0/3248613558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 msgr2=0x558135555550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:25:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x558135555550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 -- 172.21.15.16:0/3248613558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 msgr2=0x5581355503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x5581355503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.677+0000 7fe11e7e7700 1 -- 172.21.15.16:0/3248613558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558135547560 con 0x5581355567a0 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x55813554ada0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe110009ca0 tx=0x7fe110006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e492fdd14a743c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe10cff9700 1 -- 172.21.15.16:0/3248613558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe110004a90 con 0x5581355567a0 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe10cff9700 1 -- 172.21.15.16:0/3248613558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe110004bf0 con 0x5581355567a0 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe10cff9700 1 -- 172.21.15.16:0/3248613558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100052a0 con 0x5581355567a0 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 -- 172.21.15.16:0/3248613558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 msgr2=0x55813554ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x55813554ada0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe110009ca0 tx=0x7fe110006040 comp rx=0 tx=0).stop 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 -- 172.21.15.16:0/3248613558 shutdown_connections 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x55813554ada0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x5581355503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 --2- 172.21.15.16:0/3248613558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x558135555550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.678+0000 7fe12c404b80 1 -- 172.21.15.16:0/3248613558 >> 172.21.15.16:0/3248613558 conn(0x5581354ae290 msgr2=0x558135558380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 -- 172.21.15.16:0/3248613558 shutdown_connections 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 -- 172.21.15.16:0/3248613558 wait complete. 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 -- start start 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 Processor -- start 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x55813555ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x558135560450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x5581355609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581355c0810 con 0x5581355567a0 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581355c0980 con 0x558135550910 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581355c0b40 con 0x558135555e80 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.679+0000 7fe12c404b80 10 client.0 _ll_register_callbacks cb 0x558135445bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x5581355609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x5581355609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51598/0 (socket says 172.21.15.16:51598) 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1291214826 learned_addr learned my addr 172.21.15.16:0/1291214826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1291214826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 msgr2=0x558135560450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x558135560450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 msgr2=0x55813555ff10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x55813555ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 -- 172.21.15.16:0/1291214826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558135321430 con 0x5581355567a0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe11e7e7700 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x5581355609b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe110017040 tx=0x7fe110007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904f7fe96571d591 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe1017fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe1017fa700 1 -- 172.21.15.16:0/1291214826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11002f680 con 0x5581355567a0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe110007e40 con 0x5581355567a0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe1017fa700 1 -- 172.21.15.16:0/1291214826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7fe11002f7e0 con 0x5581355567a0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe1017fa700 1 -- 172.21.15.16:0/1291214826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe11002f9a0 con 0x5581355567a0 2024-05-08T23:25:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe12c404b80 10 client.16944 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.680+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5581354665e0 con 0x5581355567a0 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe1017fa700 1 -- 172.21.15.16:0/1291214826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe110007e40 con 0x5581355567a0 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe1017fa700 1 -- 172.21.15.16:0/1291214826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe110026ba0 con 0x5581355567a0 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe12c404b80 10 client.16944 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe12c404b80 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe12c404b80 20 client.16944 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5581355c1e70 0x5581355c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe12c404b80 4 client.16944 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558135498f40 con 0x5581355c1e70 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.681+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5581355c1e70 0x5581355c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.682+0000 7fe1017fa700 10 client.16944 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.682+0000 7fe11dfe6700 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5581355c1e70 0x5581355c4260 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9318069bd0561f85 server_cookie=5e19043480637bcb in_seq=0 out_seq=0 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe1017fa700 1 -- 172.21.15.16:0/1291214826 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558135498f40 con 0x5581355c1e70 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe1017fa700 10 client.16944 handle_command_reply: tid=0 2024-05-08T23:25:47.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 client.16944 shutdown 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 msgr2=0x5581355609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x5581355609b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe110017040 tx=0x7fe110007490 comp rx=0 tx=0).stop 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 shutdown_connections 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581355567a0 0x5581355609b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5581355c1e70 0x5581355c4260 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558135550910 0x55813555ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 --2- 172.21.15.16:0/1291214826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558135555e80 0x558135560450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 >> 172.21.15.16:0/1291214826 conn(0x5581354ae290 msgr2=0x558135558c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe1017fa700 0 client.16944 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 shutdown_connections 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 1 -- 172.21.15.16:0/1291214826 wait complete. 2024-05-08T23:25:47.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.683+0000 7fe12c404b80 20 client.16944 trim_cache size 0 max 16384 2024-05-08T23:25:47.691 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:25:47.691 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:25:47.691 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "59b43e54-c0b9-4fd1-a9e8-e83f9d6e4239", 2024-05-08T23:25:47.691 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:25:47.691 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1040a1310 msgr2=0x7fe1040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1040a1310 0x7fe1040a37d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1181037a0 tx=0x7fe108005960 comp rx=0 tx=0).stop 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 msgr2=0x7fe118102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 0x7fe118102070 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9a0 tx=0x7fe11400ed60 comp rx=0 tx=0).stop 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 shutdown_connections 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe1040a1310 0x7fe1040a37d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe11810d940 0x7fe118102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118068050 0x7fe118102070 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 --2- 172.21.15.16:0/2977372440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe118068970 0x7fe1181025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 >> 172.21.15.16:0/2977372440 conn(0x7fe118076f40 msgr2=0x7fe11810f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 shutdown_connections 2024-05-08T23:25:47.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.684+0000 7fe12024a700 1 -- 172.21.15.16:0/2977372440 wait complete. 2024-05-08T23:25:47.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 -- 172.21.15.16:0/3720746040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8100a10 msgr2=0x7f95e8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 --2- 172.21.15.16:0/3720746040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8100a10 0x7f95e8104a30 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009990 tx=0x7f95d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 -- 172.21.15.16:0/3720746040 shutdown_connections 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 --2- 172.21.15.16:0/3720746040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95e8105160 0x7f95e810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 --2- 172.21.15.16:0/3720746040 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95e8100060 0x7f95e8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 --2- 172.21.15.16:0/3720746040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8100a10 0x7f95e8104a30 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 -- 172.21.15.16:0/3720746040 >> 172.21.15.16:0/3720746040 conn(0x7f95e8076f40 msgr2=0x7f95e8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 -- 172.21.15.16:0/3720746040 shutdown_connections 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 -- 172.21.15.16:0/3720746040 wait complete. 2024-05-08T23:25:47.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.881+0000 7f95edeec700 1 Processor -- start 2024-05-08T23:25:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 -- start start 2024-05-08T23:25:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95e8100060 0x7f95e81962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95e8105160 0x7f95e8196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 0x7f95e819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819b620 con 0x7f95e819aba0 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819b790 con 0x7f95e8100060 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95edeec700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819b950 con 0x7f95e8105160 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 0x7f95e819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 0x7f95e819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51610/0 (socket says 172.21.15.16:51610) 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 -- 172.21.15.16:0/3970446756 learned_addr learned my addr 172.21.15.16:0/3970446756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95e8100060 0x7f95e81962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95e8105160 0x7f95e8196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 -- 172.21.15.16:0/3970446756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95e8105160 msgr2=0x7f95e8196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95e8105160 0x7f95e8196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 -- 172.21.15.16:0/3970446756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95e8100060 msgr2=0x7f95e81962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95e8100060 0x7f95e81962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.882+0000 7f95e7fff700 1 -- 172.21.15.16:0/3970446756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00095f0 con 0x7f95e819aba0 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.883+0000 7f95e7fff700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 0x7f95e819b020 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f95e000e580 tx=0x7f95e000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.883+0000 7f95e4ff9700 1 -- 172.21.15.16:0/3970446756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004db0 con 0x7f95e819aba0 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.883+0000 7f95e4ff9700 1 -- 172.21.15.16:0/3970446756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95e0003e90 con 0x7f95e819aba0 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.883+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e8068050 con 0x7f95e819aba0 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.883+0000 7f95e4ff9700 1 -- 172.21.15.16:0/3970446756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004490 con 0x7f95e819aba0 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.883+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e8068530 con 0x7f95e819aba0 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.885+0000 7f95e4ff9700 1 -- 172.21.15.16:0/3970446756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f95e0004630 con 0x7f95e819aba0 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.886+0000 7f95e4ff9700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f95d40a1290 0x7f95d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.886+0000 7f95e77fe700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f95d40a1290 0x7f95d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.886+0000 7f95e4ff9700 1 -- 172.21.15.16:0/3970446756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f95e0016070 con 0x7f95e819aba0 2024-05-08T23:25:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.886+0000 7f95e77fe700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f95d40a1290 0x7f95d40a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006fd0 tx=0x7f95d8009190 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 Processor -- start 2024-05-08T23:25:47.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 -- start start 2024-05-08T23:25:47.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1d3720 0x564dbf1cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1cd890 0x564dbf1cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf124150 con 0x564dbf1d2e00 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf121b10 con 0x564dbf1d3720 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf11d400 con 0x564dbf1cd890 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.887+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51612/0 (socket says 172.21.15.16:51612) 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 -- 172.21.15.16:0/160403570 learned_addr learned my addr 172.21.15.16:0/160403570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1d3720 0x564dbf1cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e7fff700 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1cd890 0x564dbf1cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 -- 172.21.15.16:0/160403570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1cd890 msgr2=0x564dbf1cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1cd890 0x564dbf1cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 -- 172.21.15.16:0/160403570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1d3720 msgr2=0x564dbf1cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1d3720 0x564dbf1cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 -- 172.21.15.16:0/160403570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dbef9e3b0 con 0x564dbf1d2e00 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95e77fe700 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1d31e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f95d800ea20 tx=0x7f95d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e266ed961d5829ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95ddffb700 1 -- 172.21.15.16:0/160403570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800eee0 con 0x564dbf1d2e00 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95ddffb700 1 -- 172.21.15.16:0/160403570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95d8015910 con 0x564dbf1d2e00 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.888+0000 7f95ddffb700 1 -- 172.21.15.16:0/160403570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8013910 con 0x564dbf1d2e00 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/160403570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 msgr2=0x564dbf1d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1d31e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f95d800ea20 tx=0x7f95d8011000 comp rx=0 tx=0).stop 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/160403570 shutdown_connections 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1cd890 0x564dbf1cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1d31e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/160403570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1d3720 0x564dbf1cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/160403570 >> 172.21.15.16:0/160403570 conn(0x564dbf12b210 msgr2=0x564dbf1c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/160403570 shutdown_connections 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/160403570 wait complete. 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.889+0000 7f95fa0a6b80 1 -- start start 2024-05-08T23:25:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 Processor -- start 2024-05-08T23:25:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf0b9560 con 0x564dbf1d3720 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf0b9200 con 0x564dbf1cd890 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbefa06a0 con 0x564dbf1d2e00 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95fa0a6b80 10 client.0 _ll_register_callbacks cb 0x564dbf0c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.890+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1d75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51620/0 (socket says 172.21.15.16:51620) 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 -- 172.21.15.16:0/1460015986 learned_addr learned my addr 172.21.15.16:0/1460015986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e77fe700 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 -- 172.21.15.16:0/1460015986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 msgr2=0x564dbf1d70a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1d70a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 -- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 msgr2=0x564dbf1d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 -- 172.21.15.16:0/1460015986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d800e700 con 0x564dbf1d3720 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95e7fff700 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1d75e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f95e009b970 tx=0x7f95e009c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4b616aaca653be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95c6ffd700 1 -- 172.21.15.16:0/1460015986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00a40f0 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95c6ffd700 1 -- 172.21.15.16:0/1460015986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f95e009b160 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dbf076270 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95c6ffd700 1 -- 172.21.15.16:0/1460015986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95e009b320 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95fa0a6b80 10 client.16956 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.891+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564dbef4ab60 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95c6ffd700 1 -- 172.21.15.16:0/1460015986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95e0065080 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95c6ffd700 1 -- 172.21.15.16:0/1460015986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95e00afb40 con 0x564dbf1d3720 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95fa0a6b80 10 client.16956 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95fa0a6b80 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95fa0a6b80 20 client.16956 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf1e71e0 0x564dbf1e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95fa0a6b80 4 client.16956 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.892+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dbf19c810 con 0x564dbf1e71e0 2024-05-08T23:25:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.894+0000 7f95e77fe700 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf1e71e0 0x564dbf1e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.898+0000 7f95c6ffd700 10 client.16956 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.898+0000 7f95e77fe700 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf1e71e0 0x564dbf1e95d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf7b04034cb0705 server_cookie=a5476fe054e8ad79 in_seq=0 out_seq=0 2024-05-08T23:25:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95c6ffd700 1 -- 172.21.15.16:0/1460015986 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564dbf19c810 con 0x564dbf1e71e0 2024-05-08T23:25:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95c6ffd700 10 client.16956 handle_command_reply: tid=0 2024-05-08T23:25:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 client.16956 shutdown 2024-05-08T23:25:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 msgr2=0x564dbf1d75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1d75e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f95e009b970 tx=0x7f95e009c370 comp rx=0 tx=0).stop 2024-05-08T23:25:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 shutdown_connections 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1d75e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf1e71e0 0x564dbf1e95d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1460015986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95c6ffd700 0 client.16956 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 >> 172.21.15.16:0/1460015986 conn(0x564dbf12b210 msgr2=0x564dbf1c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 shutdown_connections 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.900+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1460015986 wait complete. 2024-05-08T23:25:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.901+0000 7f95fa0a6b80 20 client.16956 trim_cache size 0 max 16384 2024-05-08T23:25:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 Processor -- start 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 -- start start 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf1ee7e0 con 0x564dbf1d3720 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf1d9c20 con 0x564dbf1cd890 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf1c2800 con 0x564dbf1d2e00 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51632/0 (socket says 172.21.15.16:51632) 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 -- 172.21.15.16:0/1574969901 learned_addr learned my addr 172.21.15.16:0/1574969901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e77fe700 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 -- 172.21.15.16:0/1574969901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 msgr2=0x564dbf1c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 -- 172.21.15.16:0/1574969901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 msgr2=0x564dbf1d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e7fff700 1 -- 172.21.15.16:0/1574969901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dbf0e3560 con 0x564dbf1d3720 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.919+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95e7fff700 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1cf9b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f95e00986e0 tx=0x7f95e00ba170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3bf7e29930984c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95de7fc700 1 -- 172.21.15.16:0/1574969901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00a40f0 con 0x564dbf1d3720 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95de7fc700 1 -- 172.21.15.16:0/1574969901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564dbf076270 con 0x564dbf1d3720 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95de7fc700 1 -- 172.21.15.16:0/1574969901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00afc90 con 0x564dbf1d3720 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1574969901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 msgr2=0x564dbf1cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1cf9b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f95e00986e0 tx=0x7f95e00ba170 comp rx=0 tx=0).stop 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1574969901 shutdown_connections 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.920+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d3720 0x564dbf1cf9b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/1574969901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d2e00 0x564dbf1c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1574969901 >> 172.21.15.16:0/1574969901 conn(0x564dbf12b210 msgr2=0x564dbf1d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1574969901 shutdown_connections 2024-05-08T23:25:47.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/1574969901 wait complete. 2024-05-08T23:25:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 -- start start 2024-05-08T23:25:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 Processor -- start 2024-05-08T23:25:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1de380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d3720 0x564dbf1dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.921+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf1dca10 con 0x564dbf1d2e00 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf1dcb80 con 0x564dbf1cd890 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95fa0a6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbf1dcd40 con 0x564dbf1d3720 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95fa0a6b80 10 client.0 _ll_register_callbacks cb 0x564dbf0c2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1de380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1de380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51636/0 (socket says 172.21.15.16:51636) 2024-05-08T23:25:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 -- 172.21.15.16:0/2632318076 learned_addr learned my addr 172.21.15.16:0/2632318076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e77fe700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e7fff700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d3720 0x564dbf1dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 -- 172.21.15.16:0/2632318076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d3720 msgr2=0x564dbf1dc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d3720 0x564dbf1dc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 -- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 msgr2=0x564dbf1dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 -- 172.21.15.16:0/2632318076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dbef4ab60 con 0x564dbf1d2e00 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e77fe700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:25:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95e6ffd700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1de380 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f95e806a6d0 tx=0x7f95d00054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ad4cf25a7a1aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95c6ffd700 1 -- 172.21.15.16:0/2632318076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d001d030 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95c6ffd700 1 -- 172.21.15.16:0/2632318076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9358+0+0 (secure 0 0 0) 0x7f95d000b5b0 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.922+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95d8015910 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95c6ffd700 1 -- 172.21.15.16:0/2632318076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95d00056d0 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 10 client.16962 fetch_fsmap learned FSMap version 25 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564dbefa2cd0 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95c6ffd700 1 -- 172.21.15.16:0/2632318076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95d8015910 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95c6ffd700 1 -- 172.21.15.16:0/2632318076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95d0004840 con 0x564dbf1d2e00 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 10 client.16962 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:25:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:25:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 20 client.16962 populate_metadata read hostname 'smithi016' 2024-05-08T23:25:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf23e8c0 0x564dbf240ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:25:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 4 client.16962 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:25:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564dbf115ec0 con 0x564dbf23e8c0 2024-05-08T23:25:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.923+0000 7f95e77fe700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf23e8c0 0x564dbf240ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:25:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95e77fe700 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf23e8c0 0x564dbf240ce0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f586d2f0f948ff server_cookie=9199c77d478b43ac in_seq=0 out_seq=0 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95c6ffd700 10 client.16962 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95c6ffd700 1 -- 172.21.15.16:0/2632318076 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564dbf115ec0 con 0x564dbf23e8c0 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95c6ffd700 10 client.16962 handle_command_reply: tid=0 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 client.16962 shutdown 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 msgr2=0x564dbf1de380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1de380 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f95e806a6d0 tx=0x7f95d00054f0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 shutdown_connections 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564dbf1d3720 0x564dbf1dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564dbf23e8c0 0x564dbf240ce0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dbf1cd890 0x564dbf1dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 --2- 172.21.15.16:0/2632318076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564dbf1d2e00 0x564dbf1de380 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95c6ffd700 0 client.16962 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 >> 172.21.15.16:0/2632318076 conn(0x564dbf12b210 msgr2=0x564dbf1d3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 shutdown_connections 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 1 -- 172.21.15.16:0/2632318076 wait complete. 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.924+0000 7f95fa0a6b80 20 client.16962 trim_cache size 0 max 16384 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout: "59b43e54-c0b9-4fd1-a9e8-e83f9d6e4239": { 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "59b43e54-c0b9-4fd1-a9e8-e83f9d6e4239", 2024-05-08T23:25:47.933 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:25:47.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:25:47.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:25:47.934 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:25:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f95d40a1290 msgr2=0x7f95d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f95d40a1290 0x7f95d40a3750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006fd0 tx=0x7f95d8009190 comp rx=0 tx=0).stop 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 msgr2=0x7f95e819b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 0x7f95e819b020 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f95e000e580 tx=0x7f95e000e890 comp rx=0 tx=0).stop 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 shutdown_connections 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f95d40a1290 0x7f95d40a3750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e819aba0 0x7f95e819b020 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95e8100060 0x7f95e81962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 --2- 172.21.15.16:0/3970446756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95e8105160 0x7f95e8196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 >> 172.21.15.16:0/3970446756 conn(0x7f95e8076f40 msgr2=0x7f95e80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 shutdown_connections 2024-05-08T23:25:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:25:47.925+0000 7f95edeec700 1 -- 172.21.15.16:0/3970446756 wait complete. 2024-05-08T23:25:47.938 INFO:tasks.cephfs.filesystem:scrub status for tag:59b43e54-c0b9-4fd1-a9e8-e83f9d6e4239 - {'path': '/', 'tag': '59b43e54-c0b9-4fd1-a9e8-e83f9d6e4239', 'options': 'recursive,force'} 2024-05-08T23:25:48.888 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:189/600: open fd = 3 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:190/600: open fd = 3 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.889 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:191/600: open fd = 3 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:192/600: open fd = 3 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.890 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:193/600: open fd = 3 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:194/600: open fd = 3 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.891 INFO:tasks.workunit.client.0.smithi016.stdout:195/600: open fd = 3 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:196/600: open fd = 3 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:197/600: open fd = 3 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.892 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:198/600: open fd = 3 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:199/600: open fd = 3 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:200/600: open fd = 3 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.893 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:201/600: open fd = 3 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:202/600: open fd = 3 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:203/600: open fd = 3 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.894 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:204/600: open fd = 3 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:205/600: open fd = 3 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:206/600: open fd = 3 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.895 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:207/600: open fd = 3 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:208/600: open fd = 3 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.896 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.897 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.897 INFO:tasks.workunit.client.0.smithi016.stdout:209/600: open fd = 3 2024-05-08T23:25:48.897 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.897 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:210/600: open fd = 3 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:211/600: open fd = 3 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:212/600: open fd = 3 2024-05-08T23:25:48.898 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.899 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.899 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.899 INFO:tasks.workunit.client.0.smithi016.stdout:213/600: open fd = 3 2024-05-08T23:25:48.899 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.899 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:214/600: open fd = 3 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:215/600: open fd = 3 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:216/600: open fd = 3 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.900 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:217/600: open fd = 3 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:218/600: open fd = 3 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:219/600: open fd = 3 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.901 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:220/600: open fd = 3 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:221/600: open fd = 3 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:222/600: open fd = 3 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.902 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:223/600: open fd = 3 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:224/600: open fd = 3 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:225/600: open fd = 3 2024-05-08T23:25:48.903 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:226/600: open fd = 3 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:227/600: open fd = 3 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:228/600: open fd = 3 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.904 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:229/600: open fd = 3 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:230/600: open fd = 3 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:231/600: open fd = 3 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:232/600: open fd = 3 2024-05-08T23:25:48.905 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:233/600: open fd = 3 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:234/600: open fd = 3 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.906 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:235/600: open fd = 3 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:236/600: open fd = 3 2024-05-08T23:25:48.907 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:237/600: open fd = 3 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:238/600: open fd = 3 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:239/600: open fd = 3 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.908 INFO:tasks.workunit.client.0.smithi016.stdout:240/600: open fd = 3 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:241/600: open fd = 3 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:242/600: open fd = 3 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:243/600: open fd = 3 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.909 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.910 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.910 INFO:tasks.workunit.client.0.smithi016.stdout:244/600: open fd = 3 2024-05-08T23:25:48.910 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.910 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:245/600: open fd = 3 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:246/600: open fd = 3 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:247/600: open fd = 3 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:248/600: open fd = 3 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.911 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:249/600: open fd = 3 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:250/600: open fd = 3 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:48.912 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:25:49.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:49 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:25:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:49 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:49 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:25:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:49 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:49 smithi189 ceph-mon[46919]: pgmap v190: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:49 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:25:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:49 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:49 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:25:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:49 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:49 smithi155 ceph-mon[47105]: pgmap v190: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:49 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:25:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:49 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:49 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:25:49.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:49 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:25:49.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:49 smithi016 ceph-mon[42541]: pgmap v190: 129 pgs: 129 active+clean; 7.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:51 smithi189 ceph-mon[46919]: pgmap v191: 129 pgs: 129 active+clean; 7.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:51 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:25:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:51 smithi155 ceph-mon[47105]: pgmap v191: 129 pgs: 129 active+clean; 7.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:51 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:25:51.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:51 smithi016 ceph-mon[42541]: pgmap v191: 129 pgs: 129 active+clean; 7.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:25:51.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:51 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:25:53.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:53 smithi189 ceph-mon[46919]: pgmap v192: 129 pgs: 129 active+clean; 7.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:53 smithi155 ceph-mon[47105]: pgmap v192: 129 pgs: 129 active+clean; 7.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:53.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:53 smithi016 ceph-mon[42541]: pgmap v192: 129 pgs: 129 active+clean; 7.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:55 smithi189 ceph-mon[46919]: pgmap v193: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:55 smithi155 ceph-mon[47105]: pgmap v193: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:55.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:55 smithi016 ceph-mon[42541]: pgmap v193: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-05-08T23:25:57.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:57 smithi189 ceph-mon[46919]: pgmap v194: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:57.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:57 smithi155 ceph-mon[47105]: pgmap v194: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:57.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:57 smithi016 ceph-mon[42541]: pgmap v194: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:59.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:25:59 smithi189 ceph-mon[46919]: pgmap v195: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:59.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:25:59 smithi155 ceph-mon[47105]: pgmap v195: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:25:59.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:25:59 smithi016 ceph-mon[42541]: pgmap v195: 129 pgs: 129 active+clean; 7.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 10 op/s 2024-05-08T23:26:01.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:01 smithi189 ceph-mon[46919]: pgmap v196: 129 pgs: 129 active+clean; 7.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:26:01.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:01 smithi155 ceph-mon[47105]: pgmap v196: 129 pgs: 129 active+clean; 7.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:26:01.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:01 smithi016 ceph-mon[42541]: pgmap v196: 129 pgs: 129 active+clean; 7.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-08T23:26:03.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:03 smithi189 ceph-mon[46919]: pgmap v197: 129 pgs: 129 active+clean; 7.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:26:03.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:03 smithi155 ceph-mon[47105]: pgmap v197: 129 pgs: 129 active+clean; 7.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:26:03.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:03 smithi016 ceph-mon[42541]: pgmap v197: 129 pgs: 129 active+clean; 7.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2024-05-08T23:26:05.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:05 smithi189 ceph-mon[46919]: pgmap v198: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 85 KiB/s wr, 31 op/s 2024-05-08T23:26:05.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:05 smithi155 ceph-mon[47105]: pgmap v198: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 85 KiB/s wr, 31 op/s 2024-05-08T23:26:05.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:05 smithi016 ceph-mon[42541]: pgmap v198: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 85 KiB/s wr, 31 op/s 2024-05-08T23:26:06.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:06 smithi189 ceph-mon[46919]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:26:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:06 smithi155 ceph-mon[47105]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:26:06.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:06 smithi016 ceph-mon[42541]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T23:26:07.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:07 smithi189 ceph-mon[46919]: pgmap v200: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 86 KiB/s wr, 32 op/s 2024-05-08T23:26:07.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:07 smithi155 ceph-mon[47105]: pgmap v200: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 86 KiB/s wr, 32 op/s 2024-05-08T23:26:07.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:07 smithi016 ceph-mon[42541]: pgmap v200: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 86 KiB/s wr, 32 op/s 2024-05-08T23:26:09.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:09 smithi189 ceph-mon[46919]: pgmap v201: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 86 KiB/s wr, 32 op/s 2024-05-08T23:26:09.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:09 smithi155 ceph-mon[47105]: pgmap v201: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 86 KiB/s wr, 32 op/s 2024-05-08T23:26:09.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:09 smithi016 ceph-mon[42541]: pgmap v201: 129 pgs: 129 active+clean; 7.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 86 KiB/s wr, 32 op/s 2024-05-08T23:26:11.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:11 smithi189 ceph-mon[46919]: pgmap v202: 129 pgs: 129 active+clean; 7.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-05-08T23:26:11.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:11 smithi155 ceph-mon[47105]: pgmap v202: 129 pgs: 129 active+clean; 7.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-05-08T23:26:11.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:11 smithi016 ceph-mon[42541]: pgmap v202: 129 pgs: 129 active+clean; 7.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-05-08T23:26:13.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:13 smithi155 ceph-mon[47105]: pgmap v203: 129 pgs: 129 active+clean; 7.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-05-08T23:26:13.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:13 smithi016 ceph-mon[42541]: pgmap v203: 129 pgs: 129 active+clean; 7.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-05-08T23:26:13.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:13 smithi189 ceph-mon[46919]: pgmap v203: 129 pgs: 129 active+clean; 7.8 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 90 KiB/s wr, 32 op/s 2024-05-08T23:26:14.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:14 smithi155 ceph-mon[47105]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:26:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:14 smithi016 ceph-mon[42541]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:26:15.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:14 smithi189 ceph-mon[46919]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T23:26:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:15 smithi155 ceph-mon[47105]: pgmap v205: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-05-08T23:26:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:15 smithi016 ceph-mon[42541]: pgmap v205: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-05-08T23:26:16.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:15 smithi189 ceph-mon[46919]: pgmap v205: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 14 op/s 2024-05-08T23:26:17.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:26:17.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:17 smithi155 ceph-mon[47105]: pgmap v206: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-05-08T23:26:17.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:17 smithi016 ceph-mon[42541]: pgmap v206: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-05-08T23:26:18.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 -- 172.21.15.16:0/745360887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc100430 msgr2=0x7f66cc105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 --2- 172.21.15.16:0/745360887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc100430 0x7f66cc105bd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009990 tx=0x7f66c0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 -- 172.21.15.16:0/745360887 shutdown_connections 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 --2- 172.21.15.16:0/745360887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc1062b0 0x7f66cc10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 --2- 172.21.15.16:0/745360887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc0ffa80 0x7f66cc0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 --2- 172.21.15.16:0/745360887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc100430 0x7f66cc105bd0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 -- 172.21.15.16:0/745360887 >> 172.21.15.16:0/745360887 conn(0x7f66cc076110 msgr2=0x7f66cc076520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 -- 172.21.15.16:0/745360887 shutdown_connections 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.087+0000 7f66d2537700 1 -- 172.21.15.16:0/745360887 wait complete. 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 Processor -- start 2024-05-08T23:26:18.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 -- start start 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc0ffa80 0x7f66cc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc1062b0 0x7f66cc197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 0x7f66cc19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc19c640 con 0x7f66cc19bbc0 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc19c7b0 con 0x7f66cc0ffa80 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d2537700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc19c970 con 0x7f66cc1062b0 2024-05-08T23:26:18.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d0ad4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 0x7f66cc19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d0ad4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 0x7f66cc19c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57994/0 (socket says 172.21.15.16:57994) 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66d0ad4700 1 -- 172.21.15.16:0/234734565 learned_addr learned my addr 172.21.15.16:0/234734565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66cbfff700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc0ffa80 0x7f66cc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.088+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc1062b0 0x7f66cc197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d0ad4700 1 -- 172.21.15.16:0/234734565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc1062b0 msgr2=0x7f66cc197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc1062b0 0x7f66cc197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d0ad4700 1 -- 172.21.15.16:0/234734565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc0ffa80 msgr2=0x7f66cc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc0ffa80 0x7f66cc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d0ad4700 1 -- 172.21.15.16:0/234734565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00095f0 con 0x7f66cc19bbc0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc1062b0 0x7f66cc197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 0x7f66cc19c040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f66c4007ee0 tx=0x7f66c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66c97fa700 1 -- 172.21.15.16:0/234734565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4017070 con 0x7f66cc19bbc0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66c97fa700 1 -- 172.21.15.16:0/234734565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c400f040 con 0x7f66cc19bbc0 2024-05-08T23:26:18.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc1a3ed0 con 0x7f66cc19bbc0 2024-05-08T23:26:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66c97fa700 1 -- 172.21.15.16:0/234734565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4005710 con 0x7f66cc19bbc0 2024-05-08T23:26:18.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.089+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc1a4350 con 0x7f66cc19bbc0 2024-05-08T23:26:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.091+0000 7f66c97fa700 1 -- 172.21.15.16:0/234734565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f66c4005870 con 0x7f66cc19bbc0 2024-05-08T23:26:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.092+0000 7f66c97fa700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f66b40a1310 0x7f66b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.092+0000 7f66cbfff700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f66b40a1310 0x7f66b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.093+0000 7f66c97fa700 1 -- 172.21.15.16:0/234734565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f66c40cc720 con 0x7f66cc19bbc0 2024-05-08T23:26:18.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.093+0000 7f66cbfff700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f66b40a1310 0x7f66b40a37d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66bc007c60 tx=0x7f66bc0073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.093+0000 7f66de6f1b80 1 Processor -- start 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.093+0000 7f66de6f1b80 1 -- start start 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.093+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.093+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875edc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875edcca0 0x562875edd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875e2f150 con 0x562875ee2b30 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875e2cb10 con 0x562875ee2210 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875e28400 con 0x562875edcca0 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875edc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875edc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58002/0 (socket says 172.21.15.16:58002) 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 -- 172.21.15.16:0/2182588388 learned_addr learned my addr 172.21.15.16:0/2182588388 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875edcca0 0x562875edd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cbfff700 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 -- 172.21.15.16:0/2182588388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875edcca0 msgr2=0x562875edd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875edcca0 0x562875edd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 -- 172.21.15.16:0/2182588388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 msgr2=0x562875ee25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 -- 172.21.15.16:0/2182588388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562875ca93b0 con 0x562875ee2b30 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875edc760 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009990 tx=0x7f66c0005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a8045bd4cd0cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66b27fc700 1 -- 172.21.15.16:0/2182588388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0005230 con 0x562875ee2b30 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66b27fc700 1 -- 172.21.15.16:0/2182588388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c001e040 con 0x562875ee2b30 2024-05-08T23:26:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.094+0000 7f66b27fc700 1 -- 172.21.15.16:0/2182588388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0004700 con 0x562875ee2b30 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2182588388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 msgr2=0x562875edc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875edc760 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009990 tx=0x7f66c0005f00 comp rx=0 tx=0).stop 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2182588388 shutdown_connections 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875edcca0 0x562875edd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2182588388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875edc760 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2182588388 >> 172.21.15.16:0/2182588388 conn(0x562875e36210 msgr2=0x562875ed2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2182588388 shutdown_connections 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.095+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2182588388 wait complete. 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 -- start start 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 Processor -- start 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875ee5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875ee6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875dc4560 con 0x562875edcca0 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875dc4200 con 0x562875ee2210 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875cab6a0 con 0x562875ee2b30 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66de6f1b80 10 client.0 _ll_register_callbacks cb 0x562875dcdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.096+0000 7f66cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875ee5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66d0ad4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875ee6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875ee5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58012/0 (socket says 172.21.15.16:58012) 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 -- 172.21.15.16:0/1765703265 learned_addr learned my addr 172.21.15.16:0/1765703265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 -- 172.21.15.16:0/1765703265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 msgr2=0x562875ee6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875ee6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 -- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 msgr2=0x562875ee6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 -- 172.21.15.16:0/1765703265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc10e650 con 0x562875edcca0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875ee6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66cbfff700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875ee5f00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0116a0 tx=0x7f66bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb37a8429934c031 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66ab7fe700 1 -- 172.21.15.16:0/1765703265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc01f840 con 0x562875edcca0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66ab7fe700 1 -- 172.21.15.16:0/1765703265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f66bc017070 con 0x562875edcca0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562875d81270 con 0x562875edcca0 2024-05-08T23:26:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66ab7fe700 1 -- 172.21.15.16:0/1765703265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66bc010e80 con 0x562875edcca0 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66de6f1b80 10 client.16974 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.097+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562875c55b60 con 0x562875edcca0 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66ab7fe700 1 -- 172.21.15.16:0/1765703265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562875d81270 con 0x562875edcca0 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66ab7fe700 1 -- 172.21.15.16:0/1765703265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f66bc01e8c0 con 0x562875edcca0 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66de6f1b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66de6f1b80 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66de6f1b80 20 client.16974 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875ef6650 0x562875ef8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66de6f1b80 4 client.16974 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562875ea7810 con 0x562875ef6650 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.098+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875ef6650 0x562875ef8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875ef6650 0x562875ef8a70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c256a76887887f server_cookie=68a00c7024a29bf0 in_seq=0 out_seq=0 2024-05-08T23:26:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66ab7fe700 10 client.16974 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66ab7fe700 1 -- 172.21.15.16:0/1765703265 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562875ea7810 con 0x562875ef6650 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66ab7fe700 10 client.16974 handle_command_reply: tid=0 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66de6f1b80 1 client.16974 shutdown 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 msgr2=0x562875ee5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.099+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875ee5f00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0116a0 tx=0x7f66bc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 shutdown_connections 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875ee6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875ee5f00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875ef6650 0x562875ef8a70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/1765703265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875ee6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 >> 172.21.15.16:0/1765703265 conn(0x562875e36210 msgr2=0x562875ed25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66ab7fe700 0 client.16974 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 shutdown_connections 2024-05-08T23:26:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 1 -- 172.21.15.16:0/1765703265 wait complete. 2024-05-08T23:26:18.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.100+0000 7f66de6f1b80 20 client.16974 trim_cache size 0 max 16384 2024-05-08T23:26:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 Processor -- start 2024-05-08T23:26:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 -- start start 2024-05-08T23:26:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875edcca0 0x562875ed2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2210 0x562875ee67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875eed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875ecfdc0 con 0x562875ee2b30 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875efdc60 con 0x562875edcca0 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.120+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875ed3a90 con 0x562875ee2210 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875eed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875eed500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58028/0 (socket says 172.21.15.16:58028) 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 -- 172.21.15.16:0/2398883768 learned_addr learned my addr 172.21.15.16:0/2398883768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66cbfff700 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875edcca0 0x562875ed2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2210 0x562875ee67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 -- 172.21.15.16:0/2398883768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2210 msgr2=0x562875ee67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2210 0x562875ee67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 -- 172.21.15.16:0/2398883768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875edcca0 msgr2=0x562875ed2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875edcca0 0x562875ed2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 -- 172.21.15.16:0/2398883768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562875dee560 con 0x562875ee2b30 2024-05-08T23:26:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875eed500 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009510 tx=0x7f66c4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4961ce2e440672 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66b2ffd700 1 -- 172.21.15.16:0/2398883768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c400dc60 con 0x562875ee2b30 2024-05-08T23:26:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66b2ffd700 1 -- 172.21.15.16:0/2398883768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c4022250 con 0x562875ee2b30 2024-05-08T23:26:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.121+0000 7f66b2ffd700 1 -- 172.21.15.16:0/2398883768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40a1910 con 0x562875ee2b30 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2398883768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 msgr2=0x562875eed500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875eed500 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009510 tx=0x7f66c4084040 comp rx=0 tx=0).stop 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2398883768 shutdown_connections 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875ee2b30 0x562875eed500 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875edcca0 0x562875ed2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/2398883768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2210 0x562875ee67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2398883768 >> 172.21.15.16:0/2398883768 conn(0x562875e36210 msgr2=0x562875ee0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2398883768 shutdown_connections 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.122+0000 7f66de6f1b80 1 -- 172.21.15.16:0/2398883768 wait complete. 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 -- start start 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 Processor -- start 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875eed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875edf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875edf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875eddf10 con 0x562875edcca0 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875ede080 con 0x562875ee2210 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562875ede240 con 0x562875ee2b30 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66de6f1b80 10 client.0 _ll_register_callbacks cb 0x562875dcdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875eed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875eed9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58032/0 (socket says 172.21.15.16:58032) 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66cbfff700 1 -- 172.21.15.16:0/4116435589 learned_addr learned my addr 172.21.15.16:0/4116435589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.123+0000 7f66d0ad4700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875edf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875edf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cbfff700 1 -- 172.21.15.16:0/4116435589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 msgr2=0x562875edf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cbfff700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875edf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cbfff700 1 -- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 msgr2=0x562875edf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cbfff700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875edf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cbfff700 1 -- 172.21.15.16:0/4116435589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc1a4350 con 0x562875edcca0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66cbfff700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875eed9e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f66bc01f010 tx=0x7f66bc01e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ee87f54e86ac30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66ab7fe700 1 -- 172.21.15.16:0/4116435589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004230 con 0x562875edcca0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66ab7fe700 1 -- 172.21.15.16:0/4116435589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f66bc017070 con 0x562875edcca0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66c001e040 con 0x562875edcca0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66ab7fe700 1 -- 172.21.15.16:0/4116435589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66bc010e60 con 0x562875edcca0 2024-05-08T23:26:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66de6f1b80 10 client.16980 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.124+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562875cadcd0 con 0x562875edcca0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66ab7fe700 1 -- 172.21.15.16:0/4116435589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66c001e040 con 0x562875edcca0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66ab7fe700 1 -- 172.21.15.16:0/4116435589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f66bc003b10 con 0x562875edcca0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66de6f1b80 10 client.16980 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66de6f1b80 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66de6f1b80 20 client.16980 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875f4d6d0 0x562875f4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66de6f1b80 4 client.16980 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562875e20ec0 con 0x562875f4d6d0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875f4d6d0 0x562875f4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66ab7fe700 10 client.16980 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.125+0000 7f66cb7fe700 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875f4d6d0 0x562875f4faf0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4131c3314f45c8a2 server_cookie=9cef267a3dd43b6e in_seq=0 out_seq=0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66ab7fe700 1 -- 172.21.15.16:0/4116435589 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562875e20ec0 con 0x562875f4d6d0 2024-05-08T23:26:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66ab7fe700 10 client.16980 handle_command_reply: tid=0 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 client.16980 shutdown 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 msgr2=0x562875eed9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875eed9e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f66bc01f010 tx=0x7f66bc01e6e0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 shutdown_connections 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562875ee2b30 0x562875edf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562875edcca0 0x562875eed9e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x562875f4d6d0 0x562875f4faf0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 --2- 172.21.15.16:0/4116435589 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562875ee2210 0x562875edf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66ab7fe700 0 client.16980 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 >> 172.21.15.16:0/4116435589 conn(0x562875e36210 msgr2=0x562875ee4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 shutdown_connections 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 1 -- 172.21.15.16:0/4116435589 wait complete. 2024-05-08T23:26:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.126+0000 7f66de6f1b80 20 client.16980 trim_cache size 0 max 16384 2024-05-08T23:26:18.133 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:26:18.133 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:26:18.133 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:26:18.133 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f66b40a1310 msgr2=0x7f66b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f66b40a1310 0x7f66b40a37d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f66bc007c60 tx=0x7f66bc0073d0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 msgr2=0x7f66cc19c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 0x7f66cc19c040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f66c4007ee0 tx=0x7f66c400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 shutdown_connections 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f66b40a1310 0x7f66b40a37d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66cc19bbc0 0x7f66cc19c040 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66cc0ffa80 0x7f66cc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 --2- 172.21.15.16:0/234734565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66cc1062b0 0x7f66cc197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 >> 172.21.15.16:0/234734565 conn(0x7f66cc076110 msgr2=0x7f66cc0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 shutdown_connections 2024-05-08T23:26:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.127+0000 7f66d2537700 1 -- 172.21.15.16:0/234734565 wait complete. 2024-05-08T23:26:18.141 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:26:18.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:18.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:17 smithi189 ceph-mon[46919]: pgmap v206: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-05-08T23:26:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 -- 172.21.15.16:0/3705654209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102940 msgr2=0x7ffa0410ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 --2- 172.21.15.16:0/3705654209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102940 0x7ffa0410ae20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009990 tx=0x7ff9ec009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 -- 172.21.15.16:0/3705654209 shutdown_connections 2024-05-08T23:26:18.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 --2- 172.21.15.16:0/3705654209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa0410b3f0 0x7ffa0410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 --2- 172.21.15.16:0/3705654209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa04102020 0x7ffa04102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 --2- 172.21.15.16:0/3705654209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102940 0x7ffa0410ae20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 -- 172.21.15.16:0/3705654209 >> 172.21.15.16:0/3705654209 conn(0x7ffa040fb800 msgr2=0x7ffa040fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 -- 172.21.15.16:0/3705654209 shutdown_connections 2024-05-08T23:26:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.321+0000 7ffa0956c700 1 -- 172.21.15.16:0/3705654209 wait complete. 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.322+0000 7ffa0956c700 1 Processor -- start 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.322+0000 7ffa0956c700 1 -- start start 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa0956c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 0x7ffa041972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa0956c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa04102940 0x7ffa04197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa0956c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa0410b3f0 0x7ffa0419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa0956c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0419c190 con 0x7ffa04102020 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa0956c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0419c300 con 0x7ffa04102940 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa0956c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0419c4c0 con 0x7ffa0410b3f0 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 0x7ffa041972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 0x7ffa041972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58058/0 (socket says 172.21.15.16:58058) 2024-05-08T23:26:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 -- 172.21.15.16:0/1052231390 learned_addr learned my addr 172.21.15.16:0/1052231390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ff9fa7fc700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa04102940 0x7ffa04197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa037fe700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa0410b3f0 0x7ffa0419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 -- 172.21.15.16:0/1052231390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa0410b3f0 msgr2=0x7ffa0419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa0410b3f0 0x7ffa0419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 -- 172.21.15.16:0/1052231390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa04102940 msgr2=0x7ffa04197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa04102940 0x7ffa04197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 -- 172.21.15.16:0/1052231390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0095f0 con 0x7ffa04102020 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ff9fa7fc700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa04102940 0x7ffa04197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.323+0000 7ffa02ffd700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 0x7ffa041972c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400d950 tx=0x7ff9f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.324+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400f890 con 0x7ffa04102020 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.324+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9f400fe80 con 0x7ffa04102020 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.324+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400e3b0 con 0x7ffa04102020 2024-05-08T23:26:18.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.324+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0419c7a0 con 0x7ffa04102020 2024-05-08T23:26:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.324+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa041a3fe0 con 0x7ffa04102020 2024-05-08T23:26:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.325+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa04106510 con 0x7ffa04102020 2024-05-08T23:26:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.326+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff9f4010460 con 0x7ffa04102020 2024-05-08T23:26:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.326+0000 7ffa00ff9700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9f00a1310 0x7ff9f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.327+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff9f40cce20 con 0x7ffa04102020 2024-05-08T23:26:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.327+0000 7ff9fa7fc700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9f00a1310 0x7ff9f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.327+0000 7ff9fa7fc700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9f00a1310 0x7ff9f00a37d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec006010 tx=0x7ff9ec005960 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.330+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9f4095e70 con 0x7ffa04102020 2024-05-08T23:26:18.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.599+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa04069cb0 con 0x7ffa04102020 2024-05-08T23:26:18.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.600+0000 7ffa00ff9700 1 -- 172.21.15.16:0/1052231390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7ff9f4020090 con 0x7ffa04102020 2024-05-08T23:26:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:26:18.601 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9f00a1310 msgr2=0x7ff9f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9f00a1310 0x7ff9f00a37d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec006010 tx=0x7ff9ec005960 comp rx=0 tx=0).stop 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 msgr2=0x7ffa041972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 0x7ffa041972c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400d950 tx=0x7ff9f400dc60 comp rx=0 tx=0).stop 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 shutdown_connections 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9f00a1310 0x7ff9f00a37d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa0410b3f0 0x7ffa0419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa04102020 0x7ffa041972c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 --2- 172.21.15.16:0/1052231390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa04102940 0x7ffa04197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 >> 172.21.15.16:0/1052231390 conn(0x7ffa040fb800 msgr2=0x7ffa040fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 shutdown_connections 2024-05-08T23:26:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.602+0000 7ffa0956c700 1 -- 172.21.15.16:0/1052231390 wait complete. 2024-05-08T23:26:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:26:18.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 -- 172.21.15.16:0/1153699260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b80100060 msgr2=0x7f0b80100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 --2- 172.21.15.16:0/1153699260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b80100060 0x7f0b80100440 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009990 tx=0x7f0b74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 -- 172.21.15.16:0/1153699260 shutdown_connections 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 --2- 172.21.15.16:0/1153699260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b80105160 0x7f0b8010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 --2- 172.21.15.16:0/1153699260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b80100060 0x7f0b80100440 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 --2- 172.21.15.16:0/1153699260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b80100a10 0x7f0b80104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 -- 172.21.15.16:0/1153699260 >> 172.21.15.16:0/1153699260 conn(0x7f0b80076f40 msgr2=0x7f0b80077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 -- 172.21.15.16:0/1153699260 shutdown_connections 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.756+0000 7f0b885b1700 1 -- 172.21.15.16:0/1153699260 wait complete. 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 Processor -- start 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 -- start start 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b80100060 0x7f0b80197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b80105160 0x7f0b80197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 0x7f0b8019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019c660 con 0x7f0b8019bbe0 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019c7d0 con 0x7f0b80100060 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b885b1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019c990 con 0x7f0b80105160 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 0x7f0b8019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 0x7f0b8019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58076/0 (socket says 172.21.15.16:58076) 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b86b4e700 1 -- 172.21.15.16:0/3356850631 learned_addr learned my addr 172.21.15.16:0/3356850631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b8634d700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b80100060 0x7f0b80197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.757+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b80105160 0x7f0b80197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b86b4e700 1 -- 172.21.15.16:0/3356850631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b80105160 msgr2=0x7f0b80197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b80105160 0x7f0b80197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b86b4e700 1 -- 172.21.15.16:0/3356850631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b80100060 msgr2=0x7f0b80197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b80100060 0x7f0b80197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b86b4e700 1 -- 172.21.15.16:0/3356850631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b740095f0 con 0x7f0b8019bbe0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 0x7f0b8019c060 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0b78007ee0 tx=0x7f0b7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b737fe700 1 -- 172.21.15.16:0/3356850631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78017070 con 0x7f0b8019bbe0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b737fe700 1 -- 172.21.15.16:0/3356850631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b7800f040 con 0x7f0b8019bbe0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b737fe700 1 -- 172.21.15.16:0/3356850631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78005710 con 0x7f0b8019bbe0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b801a3ef0 con 0x7f0b8019bbe0 2024-05-08T23:26:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.758+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801a4320 con 0x7f0b8019bbe0 2024-05-08T23:26:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.761+0000 7f0b737fe700 1 -- 172.21.15.16:0/3356850631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0b78005870 con 0x7f0b8019bbe0 2024-05-08T23:26:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.761+0000 7f0b737fe700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0b6c0a1290 0x7f0b6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.761+0000 7f0b737fe700 1 -- 172.21.15.16:0/3356850631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0b780cc830 con 0x7f0b8019bbe0 2024-05-08T23:26:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.761+0000 7f0b8634d700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0b6c0a1290 0x7f0b6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b8634d700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0b6c0a1290 0x7f0b6c0a3750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400b2e0 tx=0x7f0b74005040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 Processor -- start 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 -- start start 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da144350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da09b150 con 0x5651da144890 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da098b10 con 0x5651da14a720 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.762+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da094400 con 0x5651da149e00 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58078/0 (socket says 172.21.15.16:58078) 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 -- 172.21.15.16:0/63363824 learned_addr learned my addr 172.21.15.16:0/63363824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b8634d700 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 -- 172.21.15.16:0/63363824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 msgr2=0x5651da14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 -- 172.21.15.16:0/63363824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 msgr2=0x5651da144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 -- 172.21.15.16:0/63363824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d9f153b0 con 0x5651da144890 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da144cd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0b780992a0 tx=0x7f0b78007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d70c6b5d8494485 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b70ff9700 1 -- 172.21.15.16:0/63363824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800dc60 con 0x5651da144890 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b70ff9700 1 -- 172.21.15.16:0/63363824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b78099ca0 con 0x5651da144890 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.763+0000 7f0b70ff9700 1 -- 172.21.15.16:0/63363824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78095cb0 con 0x5651da144890 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 -- 172.21.15.16:0/63363824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 msgr2=0x5651da144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da144cd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f0b780992a0 tx=0x7f0b78007f90 comp rx=0 tx=0).stop 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 -- 172.21.15.16:0/63363824 shutdown_connections 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da144cd0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/63363824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 -- 172.21.15.16:0/63363824 >> 172.21.15.16:0/63363824 conn(0x5651da0a2210 msgr2=0x5651da13edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 -- 172.21.15.16:0/63363824 shutdown_connections 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.764+0000 7f0b9476bb80 1 -- 172.21.15.16:0/63363824 wait complete. 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 -- start start 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 Processor -- start 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da14db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da14e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da030560 con 0x5651da144890 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da030200 con 0x5651da14a720 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d9f176a0 con 0x5651da149e00 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b9476bb80 10 client.0 _ll_register_callbacks cb 0x5651da039b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da14db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58086/0 (socket says 172.21.15.16:58086) 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 -- 172.21.15.16:0/856451774 learned_addr learned my addr 172.21.15.16:0/856451774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 -- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 msgr2=0x5651da14e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 -- 172.21.15.16:0/856451774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 msgr2=0x5651da14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b8634d700 1 -- 172.21.15.16:0/856451774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801a4320 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.765+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b8634d700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da14db90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009990 tx=0x7f0b74009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d55738d34ebc04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b69ffb700 1 -- 172.21.15.16:0/856451774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400be80 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b69ffb700 1 -- 172.21.15.16:0/856451774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0b7400f460 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651d9fed270 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b69ffb700 1 -- 172.21.15.16:0/856451774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b7400f660 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 10 client.16998 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5651d9ec1b60 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b69ffb700 1 -- 172.21.15.16:0/856451774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5651d9fed270 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b69ffb700 1 -- 172.21.15.16:0/856451774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0b7402f070 con 0x5651da144890 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 10 client.16998 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 10 client.16998 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 20 client.16998 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da15e2f0 0x5651da160710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 4 client.16998 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.766+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651da113810 con 0x5651da15e2f0 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.767+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da15e2f0 0x5651da160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.767+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da15e2f0 0x5651da160710 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0494f5eb8d1b2f4 server_cookie=923a0d21f982935d in_seq=0 out_seq=0 2024-05-08T23:26:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.767+0000 7f0b69ffb700 10 client.16998 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b69ffb700 1 -- 172.21.15.16:0/856451774 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651da113810 con 0x5651da15e2f0 2024-05-08T23:26:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b69ffb700 10 client.16998 handle_command_reply: tid=0 2024-05-08T23:26:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 client.16998 shutdown 2024-05-08T23:26:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 msgr2=0x5651da14db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da14db90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009990 tx=0x7f0b74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 shutdown_connections 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da15e2f0 0x5651da160710 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da14a720 0x5651da14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da144890 0x5651da14db90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/856451774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b69ffb700 0 client.16998 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 >> 172.21.15.16:0/856451774 conn(0x5651da0a2210 msgr2=0x5651da13e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 shutdown_connections 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 1 -- 172.21.15.16:0/856451774 wait complete. 2024-05-08T23:26:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.768+0000 7f0b9476bb80 20 client.16998 trim_cache size 0 max 16384 2024-05-08T23:26:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.782+0000 7f0b9476bb80 1 Processor -- start 2024-05-08T23:26:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.782+0000 7f0b9476bb80 1 -- start start 2024-05-08T23:26:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da144890 0x5651da144350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da149e00 0x5651da13ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da13bdf0 con 0x5651da14a720 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da15ae00 con 0x5651da149e00 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da140e80 con 0x5651da144890 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da14e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58090/0 (socket says 172.21.15.16:58090) 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2755348498 learned_addr learned my addr 172.21.15.16:0/2755348498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b8634d700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da144890 0x5651da144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da149e00 0x5651da13ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2755348498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da144890 msgr2=0x5651da144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da144890 0x5651da144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2755348498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da149e00 msgr2=0x5651da13ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da149e00 0x5651da13ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2755348498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651da05a560 con 0x5651da14a720 2024-05-08T23:26:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b8634d700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da144890 0x5651da144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da149e00 0x5651da13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.783+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da14e0e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009510 tx=0x7f0b7809c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff902f5ca7ee202a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b717fa700 1 -- 172.21.15.16:0/2755348498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800dc60 con 0x5651da14a720 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b717fa700 1 -- 172.21.15.16:0/2755348498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b7809d3d0 con 0x5651da14a720 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b717fa700 1 -- 172.21.15.16:0/2755348498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78097a60 con 0x5651da14a720 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2755348498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 msgr2=0x5651da14e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da14e0e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009510 tx=0x7f0b7809c040 comp rx=0 tx=0).stop 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2755348498 shutdown_connections 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da14e0e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da144890 0x5651da144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2755348498 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da149e00 0x5651da13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2755348498 >> 172.21.15.16:0/2755348498 conn(0x5651da0a2210 msgr2=0x5651da14c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2755348498 shutdown_connections 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.784+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2755348498 wait complete. 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 -- start start 2024-05-08T23:26:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 Processor -- start 2024-05-08T23:26:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da144890 0x5651da1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da153150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da153690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da153da0 con 0x5651da14a720 2024-05-08T23:26:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da153f10 con 0x5651da144890 2024-05-08T23:26:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651da1b37b0 con 0x5651da149e00 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b9476bb80 10 client.0 _ll_register_callbacks cb 0x5651da039b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58098/0 (socket says 172.21.15.16:58098) 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2747130502 learned_addr learned my addr 172.21.15.16:0/2747130502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b85b4c700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b8634d700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da144890 0x5651da1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 msgr2=0x5651da153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2747130502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da144890 msgr2=0x5651da1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da144890 0x5651da1554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.785+0000 7f0b86b4e700 1 -- 172.21.15.16:0/2747130502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651d9f153b0 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b86b4e700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da153690 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d7b0 tx=0x7f0b78007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b431fe09ab972c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b69ffb700 1 -- 172.21.15.16:0/2747130502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780a1a20 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b69ffb700 1 -- 172.21.15.16:0/2747130502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0b780995a0 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b78099ca0 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b69ffb700 1 -- 172.21.15.16:0/2747130502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b780cc440 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 10 client.17004 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5651d9f19cd0 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b69ffb700 1 -- 172.21.15.16:0/2747130502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b78099ca0 con 0x5651da14a720 2024-05-08T23:26:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b69ffb700 1 -- 172.21.15.16:0/2747130502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0b78098a90 con 0x5651da14a720 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 10 client.17004 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 10 client.17004 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 20 client.17004 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da1b48b0 0x5651da1b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 4 client.17004 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.786+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651da08cec0 con 0x5651da1b48b0 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b8634d700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da1b48b0 0x5651da1b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b8634d700 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da1b48b0 0x5651da1b6ca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d7503f284d8ed86 server_cookie=8f59e63c70cfe1a in_seq=0 out_seq=0 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b69ffb700 10 client.17004 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b69ffb700 1 -- 172.21.15.16:0/2747130502 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651da08cec0 con 0x5651da1b48b0 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b69ffb700 10 client.17004 handle_command_reply: tid=0 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 client.17004 shutdown 2024-05-08T23:26:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 msgr2=0x5651da153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da153690 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d7b0 tx=0x7f0b78007490 comp rx=0 tx=0).stop 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 shutdown_connections 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651da14a720 0x5651da153690 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5651da1b48b0 0x5651da1b6ca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5651da144890 0x5651da1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 --2- 172.21.15.16:0/2747130502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651da149e00 0x5651da153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b69ffb700 0 client.17004 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 >> 172.21.15.16:0/2747130502 conn(0x5651da0a2210 msgr2=0x5651da148c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 shutdown_connections 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.787+0000 7f0b9476bb80 1 -- 172.21.15.16:0/2747130502 wait complete. 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b9476bb80 20 client.17004 trim_cache size 0 max 16384 2024-05-08T23:26:18.795 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:26:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0b6c0a1290 msgr2=0x7f0b6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0b6c0a1290 0x7f0b6c0a3750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400b2e0 tx=0x7f0b74005040 comp rx=0 tx=0).stop 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 msgr2=0x7f0b8019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 0x7f0b8019c060 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0b78007ee0 tx=0x7f0b7800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 shutdown_connections 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0b6c0a1290 0x7f0b6c0a3750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8019bbe0 0x7f0b8019c060 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b80100060 0x7f0b80197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 --2- 172.21.15.16:0/3356850631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0b80105160 0x7f0b80197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 >> 172.21.15.16:0/3356850631 conn(0x7f0b80076f40 msgr2=0x7f0b80068430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 shutdown_connections 2024-05-08T23:26:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.788+0000 7f0b885b1700 1 -- 172.21.15.16:0/3356850631 wait complete. 2024-05-08T23:26:18.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.940+0000 7ff82ec2b700 1 -- 172.21.15.16:0/3481477003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828105f90 msgr2=0x7ff828106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.940+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/3481477003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828105f90 0x7ff828106370 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0099c0 tx=0x7ff81c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 -- 172.21.15.16:0/3481477003 shutdown_connections 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/3481477003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff828100ab0 0x7ff82810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/3481477003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828105f90 0x7ff828106370 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/3481477003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8281000f0 0x7ff828100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 -- 172.21.15.16:0/3481477003 >> 172.21.15.16:0/3481477003 conn(0x7ff828076ba0 msgr2=0x7ff828076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 -- 172.21.15.16:0/3481477003 shutdown_connections 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 -- 172.21.15.16:0/3481477003 wait complete. 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 Processor -- start 2024-05-08T23:26:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.941+0000 7ff82ec2b700 1 -- start start 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8281000f0 0x7ff828195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 0x7ff828196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff828105f90 0x7ff8281969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a56e0 con 0x7ff828100ab0 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a5850 con 0x7ff8281000f0 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8281a5a10 con 0x7ff828105f90 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 0x7ff828196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 0x7ff828196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58104/0 (socket says 172.21.15.16:58104) 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 -- 172.21.15.16:0/2742715452 learned_addr learned my addr 172.21.15.16:0/2742715452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8281000f0 0x7ff828195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff828105f90 0x7ff8281969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 -- 172.21.15.16:0/2742715452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff828105f90 msgr2=0x7ff8281969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff828105f90 0x7ff8281969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 -- 172.21.15.16:0/2742715452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8281000f0 msgr2=0x7ff828195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8281000f0 0x7ff828195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 -- 172.21.15.16:0/2742715452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0095f0 con 0x7ff828100ab0 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff827fff700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 0x7ff828196480 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9a0 tx=0x7ff81400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff825ffb700 1 -- 172.21.15.16:0/2742715452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814009790 con 0x7ff828100ab0 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8281a5cf0 con 0x7ff828100ab0 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff825ffb700 1 -- 172.21.15.16:0/2742715452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff814010430 con 0x7ff828100ab0 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.942+0000 7ff825ffb700 1 -- 172.21.15.16:0/2742715452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400cea0 con 0x7ff828100ab0 2024-05-08T23:26:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.943+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8281a61b0 con 0x7ff828100ab0 2024-05-08T23:26:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.945+0000 7ff825ffb700 1 -- 172.21.15.16:0/2742715452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff8140105a0 con 0x7ff828100ab0 2024-05-08T23:26:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.946+0000 7ff825ffb700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff8180a1290 0x7ff8180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.946+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff8180a1290 0x7ff8180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.946+0000 7ff825ffb700 1 -- 172.21.15.16:0/2742715452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff814014070 con 0x7ff828100ab0 2024-05-08T23:26:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff8180a1290 0x7ff8180a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009990 tx=0x7ff81c019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 Processor -- start 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 -- start start 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe625661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe62560350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62560cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe624b7150 con 0x55fe62565e00 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe624b4b10 con 0x55fe62566720 2024-05-08T23:26:18.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.947+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe624b0400 con 0x55fe62560890 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe625661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe625661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58106/0 (socket says 172.21.15.16:58106) 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 -- 172.21.15.16:0/3290416422 learned_addr learned my addr 172.21.15.16:0/3290416422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62560cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff827fff700 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe62560350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 -- 172.21.15.16:0/3290416422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 msgr2=0x55fe62560cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62560cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 -- 172.21.15.16:0/3290416422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 msgr2=0x55fe62560350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe62560350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 -- 172.21.15.16:0/3290416422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe623313b0 con 0x55fe62565e00 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62560cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe625661e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff828067bb0 tx=0x7ff81c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae271e505bd4b499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff812ffd700 1 -- 172.21.15.16:0/3290416422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c005620 con 0x55fe62565e00 2024-05-08T23:26:18.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff812ffd700 1 -- 172.21.15.16:0/3290416422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff81c01d040 con 0x55fe62565e00 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.948+0000 7ff812ffd700 1 -- 172.21.15.16:0/3290416422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00f8d0 con 0x55fe62565e00 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3290416422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 msgr2=0x55fe625661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe625661e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff828067bb0 tx=0x7ff81c01f000 comp rx=0 tx=0).stop 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3290416422 shutdown_connections 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62560cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe625661e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3290416422 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe62560350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3290416422 >> 172.21.15.16:0/3290416422 conn(0x55fe624be210 msgr2=0x55fe6255adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3290416422 shutdown_connections 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.949+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3290416422 wait complete. 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 -- start start 2024-05-08T23:26:18.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 Processor -- start 2024-05-08T23:26:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62569a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62565e00 0x55fe62569fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 0x55fe6256a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6244c560 con 0x55fe62566720 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6244c200 con 0x55fe62565e00 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe623336a0 con 0x55fe62560890 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff83ade5b80 10 client.0 _ll_register_callbacks cb 0x55fe62455b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff82d1c8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 0x55fe6256a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff82d1c8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 0x55fe6256a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58114/0 (socket says 172.21.15.16:58114) 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff82d1c8700 1 -- 172.21.15.16:0/836401672 learned_addr learned my addr 172.21.15.16:0/836401672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62569a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.950+0000 7ff827fff700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62565e00 0x55fe62569fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff82d1c8700 1 -- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 msgr2=0x55fe62569a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62569a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff82d1c8700 1 -- 172.21.15.16:0/836401672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62565e00 msgr2=0x55fe62569fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62565e00 0x55fe62569fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff82d1c8700 1 -- 172.21.15.16:0/836401672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82810d550 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff827fff700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62565e00 0x55fe62569fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 0x55fe6256a4e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff820007c60 tx=0x7ff820007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101ab147506f2757 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff807fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff807fff700 1 -- 172.21.15.16:0/836401672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82000e040 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff807fff700 1 -- 172.21.15.16:0/836401672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff820013b30 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe62409270 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff807fff700 1 -- 172.21.15.16:0/836401672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff820013d30 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff83ade5b80 10 client.17016 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fe622ddb60 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.951+0000 7ff807fff700 1 -- 172.21.15.16:0/836401672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe62409270 con 0x55fe62566720 2024-05-08T23:26:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff807fff700 1 -- 172.21.15.16:0/836401672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff82001e3b0 con 0x55fe62566720 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff83ade5b80 10 client.17016 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff83ade5b80 10 client.17016 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff83ade5b80 20 client.17016 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe6257a1f0 0x55fe6257c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff83ade5b80 4 client.17016 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe6252f810 con 0x55fe6257a1f0 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.952+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe6257a1f0 0x55fe6257c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe6257a1f0 0x55fe6257c610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=388089259b569298 server_cookie=dd71b440321e8fc8 in_seq=0 out_seq=0 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff807fff700 10 client.17016 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff807fff700 1 -- 172.21.15.16:0/836401672 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe6252f810 con 0x55fe6257a1f0 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff807fff700 10 client.17016 handle_command_reply: tid=0 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 client.17016 shutdown 2024-05-08T23:26:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 msgr2=0x55fe6256a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 0x55fe6256a4e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff820007c60 tx=0x7ff820007c90 comp rx=0 tx=0).stop 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 shutdown_connections 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62566720 0x55fe6256a4e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62560890 0x55fe62569a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe6257a1f0 0x55fe6257c610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/836401672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62565e00 0x55fe62569fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 >> 172.21.15.16:0/836401672 conn(0x55fe624be210 msgr2=0x55fe6255a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.953+0000 7ff807fff700 0 client.17016 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.954+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 shutdown_connections 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.954+0000 7ff83ade5b80 1 -- 172.21.15.16:0/836401672 wait complete. 2024-05-08T23:26:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.954+0000 7ff83ade5b80 20 client.17016 trim_cache size 0 max 16384 2024-05-08T23:26:18.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:18 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1052231390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 Processor -- start 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 -- start start 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 0x55fe62569340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62565e00 0x55fe625652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe6255ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe6256b0d0 con 0x55fe62560890 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe625612a0 con 0x55fe62566720 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe62557df0 con 0x55fe62565e00 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff82c9c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 0x55fe62569340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.968+0000 7ff82c9c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 0x55fe62569340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58120/0 (socket says 172.21.15.16:58120) 2024-05-08T23:26:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 -- 172.21.15.16:0/71200506 learned_addr learned my addr 172.21.15.16:0/71200506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff827fff700 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62565e00 0x55fe625652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe6255ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 -- 172.21.15.16:0/71200506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62565e00 msgr2=0x55fe625652b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62565e00 0x55fe625652b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 -- 172.21.15.16:0/71200506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 msgr2=0x55fe6255ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe6255ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 -- 172.21.15.16:0/71200506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe62476560 con 0x55fe62560890 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 0x55fe62569340 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff81c031040 tx=0x7ff81c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27c13a755efcd50 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff8137fe700 1 -- 172.21.15.16:0/71200506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c028070 con 0x55fe62560890 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff8137fe700 1 -- 172.21.15.16:0/71200506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff81c03d070 con 0x55fe62560890 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.969+0000 7ff8137fe700 1 -- 172.21.15.16:0/71200506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004d40 con 0x55fe62560890 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 -- 172.21.15.16:0/71200506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 msgr2=0x55fe62569340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 0x55fe62569340 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff81c031040 tx=0x7ff81c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 -- 172.21.15.16:0/71200506 shutdown_connections 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62566720 0x55fe6255ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62560890 0x55fe62569340 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/71200506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62565e00 0x55fe625652b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 -- 172.21.15.16:0/71200506 >> 172.21.15.16:0/71200506 conn(0x55fe624be210 msgr2=0x55fe62564ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 -- 172.21.15.16:0/71200506 shutdown_connections 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 -- 172.21.15.16:0/71200506 wait complete. 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 -- start start 2024-05-08T23:26:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.970+0000 7ff83ade5b80 1 Processor -- start 2024-05-08T23:26:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62560890 0x55fe625713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe62561930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62566720 0x55fe62561e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe62562580 con 0x55fe62565e00 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe625626f0 con 0x55fe62560890 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe625628b0 con 0x55fe62566720 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff83ade5b80 10 client.0 _ll_register_callbacks cb 0x55fe62455b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe62561930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe62561930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58126/0 (socket says 172.21.15.16:58126) 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 -- 172.21.15.16:0/3646652476 learned_addr learned my addr 172.21.15.16:0/3646652476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62560890 0x55fe625713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62566720 0x55fe62561e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 -- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62566720 msgr2=0x55fe62561e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62566720 0x55fe62561e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 -- 172.21.15.16:0/3646652476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62560890 msgr2=0x55fe625713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62560890 0x55fe625713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff827fff700 1 -- 172.21.15.16:0/3646652476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe623313b0 con 0x55fe62565e00 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff82d1c8700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62566720 0x55fe62561e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.971+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62560890 0x55fe625713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff827fff700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe62561930 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff8140ca680 tx=0x7ff8140cab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23321dbed1ec5b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff807fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff807fff700 1 -- 172.21.15.16:0/3646652476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814099fe0 con 0x55fe62565e00 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff81c01d040 con 0x55fe62565e00 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff807fff700 1 -- 172.21.15.16:0/3646652476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff81409de10 con 0x55fe62565e00 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff807fff700 1 -- 172.21.15.16:0/3646652476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8140a2460 con 0x55fe62565e00 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 10 client.17022 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fe62335cd0 con 0x55fe62565e00 2024-05-08T23:26:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff807fff700 1 -- 172.21.15.16:0/3646652476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff814063070 con 0x55fe62565e00 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff807fff700 1 -- 172.21.15.16:0/3646652476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff8140b8080 con 0x55fe62565e00 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 10 client.17022 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 20 client.17022 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe625d0d80 0x55fe625d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 4 client.17022 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.972+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe624a8ec0 con 0x55fe625d0d80 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.973+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe625d0d80 0x55fe625d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.973+0000 7ff82c9c7700 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe625d0d80 0x55fe625d31a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67bf2b97d092d2f4 server_cookie=af820bb14795f707 in_seq=0 out_seq=0 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.973+0000 7ff807fff700 10 client.17022 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.973+0000 7ff807fff700 1 -- 172.21.15.16:0/3646652476 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe624a8ec0 con 0x55fe625d0d80 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.973+0000 7ff807fff700 10 client.17022 handle_command_reply: tid=0 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.973+0000 7ff83ade5b80 1 client.17022 shutdown 2024-05-08T23:26:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 msgr2=0x55fe62561930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe62561930 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff8140ca680 tx=0x7ff8140cab80 comp rx=0 tx=0).stop 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 shutdown_connections 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe62566720 0x55fe62561e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55fe625d0d80 0x55fe625d31a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55fe62560890 0x55fe625713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 --2- 172.21.15.16:0/3646652476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe62565e00 0x55fe62561930 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff807fff700 0 client.17022 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 >> 172.21.15.16:0/3646652476 conn(0x55fe624be210 msgr2=0x55fe6256a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 shutdown_connections 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 1 -- 172.21.15.16:0/3646652476 wait complete. 2024-05-08T23:26:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.974+0000 7ff83ade5b80 20 client.17022 trim_cache size 0 max 16384 2024-05-08T23:26:18.981 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:26:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff8180a1290 msgr2=0x7ff8180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff8180a1290 0x7ff8180a3750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009990 tx=0x7ff81c019040 comp rx=0 tx=0).stop 2024-05-08T23:26:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 msgr2=0x7ff828196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 0x7ff828196480 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9a0 tx=0x7ff81400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 shutdown_connections 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff8180a1290 0x7ff8180a3750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff828105f90 0x7ff8281969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8281000f0 0x7ff828195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 --2- 172.21.15.16:0/2742715452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828100ab0 0x7ff828196480 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 >> 172.21.15.16:0/2742715452 conn(0x7ff828076ba0 msgr2=0x7ff8280ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 shutdown_connections 2024-05-08T23:26:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:18.975+0000 7ff82ec2b700 1 -- 172.21.15.16:0/2742715452 wait complete. 2024-05-08T23:26:18.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:18 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1052231390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:26:18.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:26:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 -- 172.21.15.16:0/3066211331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c069dd0 msgr2=0x7f549c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 --2- 172.21.15.16:0/3066211331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c069dd0 0x7f549c105b70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f548c009990 tx=0x7f548c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 -- 172.21.15.16:0/3066211331 shutdown_connections 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 --2- 172.21.15.16:0/3066211331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f549c1062a0 0x7f549c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 --2- 172.21.15.16:0/3066211331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c069420 0x7f549c069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 --2- 172.21.15.16:0/3066211331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c069dd0 0x7f549c105b70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.127+0000 7f54a393b700 1 -- 172.21.15.16:0/3066211331 >> 172.21.15.16:0/3066211331 conn(0x7f549c076ba0 msgr2=0x7f549c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.128+0000 7f54a393b700 1 -- 172.21.15.16:0/3066211331 shutdown_connections 2024-05-08T23:26:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.128+0000 7f54a393b700 1 -- 172.21.15.16:0/3066211331 wait complete. 2024-05-08T23:26:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.128+0000 7f54a393b700 1 Processor -- start 2024-05-08T23:26:19.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.128+0000 7f54a393b700 1 -- start start 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a393b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c069420 0x7f549c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a393b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f549c1062a0 0x7f549c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a393b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 0x7f549c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a393b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c19c590 con 0x7f549c19bb10 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a393b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c19c700 con 0x7f549c069420 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a393b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c19c8c0 con 0x7f549c1062a0 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 0x7f549c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 0x7f549c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58136/0 (socket says 172.21.15.16:58136) 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 -- 172.21.15.16:0/1882532451 learned_addr learned my addr 172.21.15.16:0/1882532451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a16d7700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c069420 0x7f549c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f549c1062a0 0x7f549c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 -- 172.21.15.16:0/1882532451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f549c1062a0 msgr2=0x7f549c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f549c1062a0 0x7f549c1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 -- 172.21.15.16:0/1882532451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c069420 msgr2=0x7f549c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c069420 0x7f549c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 -- 172.21.15.16:0/1882532451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0095f0 con 0x7f549c19bb10 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.129+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 0x7f549c19bf90 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f549400ef30 tx=0x7f549400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.130+0000 7f54927fc700 1 -- 172.21.15.16:0/1882532451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494018070 con 0x7f549c19bb10 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.130+0000 7f54927fc700 1 -- 172.21.15.16:0/1882532451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f549400f040 con 0x7f549c19bb10 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.130+0000 7f54927fc700 1 -- 172.21.15.16:0/1882532451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549400cc70 con 0x7f549c19bb10 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.130+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c1a3dc0 con 0x7f549c19bb10 2024-05-08T23:26:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.130+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c1a4250 con 0x7f549c19bb10 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.132+0000 7f54927fc700 1 -- 172.21.15.16:0/1882532451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5494007500 con 0x7f549c19bb10 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.132+0000 7f54927fc700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f54880a1290 0x7f54880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.133+0000 7f54a16d7700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f54880a1290 0x7f54880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.133+0000 7f54927fc700 1 -- 172.21.15.16:0/1882532451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f54940ccba0 con 0x7f549c19bb10 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.133+0000 7f54a16d7700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f54880a1290 0x7f54880a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f549800ac50 tx=0x7f549800a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 Processor -- start 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 -- start start 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7cea720 0x557ad7ce4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce4890 0x557ad7ce4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7c3b150 con 0x557ad7ce9e00 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7c38b10 con 0x557ad7ce4890 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7c34400 con 0x557ad7cea720 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54a16d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54a16d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58140/0 (socket says 172.21.15.16:58140) 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54a16d7700 1 -- 172.21.15.16:0/805398597 learned_addr learned my addr 172.21.15.16:0/805398597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.134+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7cea720 0x557ad7ce4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce4890 0x557ad7ce4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a16d7700 1 -- 172.21.15.16:0/805398597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7cea720 msgr2=0x557ad7ce4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a16d7700 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7cea720 0x557ad7ce4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a16d7700 1 -- 172.21.15.16:0/805398597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce4890 msgr2=0x557ad7ce4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a16d7700 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce4890 0x557ad7ce4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a16d7700 1 -- 172.21.15.16:0/805398597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ad7ab53b0 con 0x557ad7ce9e00 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54a16d7700 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cea1e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f549800ff90 tx=0x7f5498012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0e1e2c018808c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54877fe700 1 -- 172.21.15.16:0/805398597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498003a90 con 0x557ad7ce9e00 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54877fe700 1 -- 172.21.15.16:0/805398597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5498003bf0 con 0x557ad7ce9e00 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.135+0000 7f54877fe700 1 -- 172.21.15.16:0/805398597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549801d660 con 0x557ad7ce9e00 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 -- 172.21.15.16:0/805398597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 msgr2=0x557ad7cea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cea1e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f549800ff90 tx=0x7f5498012000 comp rx=0 tx=0).stop 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 -- 172.21.15.16:0/805398597 shutdown_connections 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce4890 0x557ad7ce4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cea1e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/805398597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7cea720 0x557ad7ce4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 -- 172.21.15.16:0/805398597 >> 172.21.15.16:0/805398597 conn(0x557ad7c42210 msgr2=0x557ad7cdedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 -- 172.21.15.16:0/805398597 shutdown_connections 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 -- 172.21.15.16:0/805398597 wait complete. 2024-05-08T23:26:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 -- start start 2024-05-08T23:26:19.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.136+0000 7f54afaf5b80 1 Processor -- start 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 0x557ad7cedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce9e00 0x557ad7cee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7cee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7bd0560 con 0x557ad7ce4890 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7bd0200 con 0x557ad7cea720 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7ab76a0 con 0x557ad7ce9e00 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54afaf5b80 10 client.0 _ll_register_callbacks cb 0x557ad7bd9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 0x557ad7cedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 0x557ad7cedbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58148/0 (socket says 172.21.15.16:58148) 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 -- 172.21.15.16:0/4016657878 learned_addr learned my addr 172.21.15.16:0/4016657878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7cee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce9e00 0x557ad7cee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 -- 172.21.15.16:0/4016657878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce9e00 msgr2=0x557ad7cee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce9e00 0x557ad7cee100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 -- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 msgr2=0x557ad7cee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7cee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a16d7700 1 -- 172.21.15.16:0/4016657878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549800fc40 con 0x557ad7ce4890 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.137+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce9e00 0x557ad7cee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f54a16d7700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 0x557ad7cedbc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f549800fa30 tx=0x7f5498003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78500badbd1f2df4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f5484ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f5484ff9700 1 -- 172.21.15.16:0/4016657878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549802b070 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f5484ff9700 1 -- 172.21.15.16:0/4016657878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5498010430 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ad7b8d270 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f5484ff9700 1 -- 172.21.15.16:0/4016657878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54980105f0 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f54afaf5b80 10 client.17034 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557ad7a61b60 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f5484ff9700 1 -- 172.21.15.16:0/4016657878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557ad7b8d270 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f5484ff9700 1 -- 172.21.15.16:0/4016657878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f549801db50 con 0x557ad7ce4890 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.138+0000 7f54afaf5b80 10 client.17034 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54afaf5b80 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54afaf5b80 20 client.17034 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7cfe1c0 0x557ad7d005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54afaf5b80 4 client.17034 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ad7cb3810 con 0x557ad7cfe1c0 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7cfe1c0 0x557ad7d005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7cfe1c0 0x557ad7d005b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faadd19b6884fcb8 server_cookie=6f3ad77e4349c068 in_seq=0 out_seq=0 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.139+0000 7f5484ff9700 10 client.17034 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f5484ff9700 1 -- 172.21.15.16:0/4016657878 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ad7cb3810 con 0x557ad7cfe1c0 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f5484ff9700 10 client.17034 handle_command_reply: tid=0 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 client.17034 shutdown 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 msgr2=0x557ad7cedbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 0x557ad7cedbc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f549800fa30 tx=0x7f5498003f90 comp rx=0 tx=0).stop 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 shutdown_connections 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7cfe1c0 0x557ad7d005b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7cee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f5484ff9700 0 client.17034 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce4890 0x557ad7cedbc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/4016657878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce9e00 0x557ad7cee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 >> 172.21.15.16:0/4016657878 conn(0x557ad7c42210 msgr2=0x557ad7cde5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 shutdown_connections 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 1 -- 172.21.15.16:0/4016657878 wait complete. 2024-05-08T23:26:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.140+0000 7f54afaf5b80 20 client.17034 trim_cache size 0 max 16384 2024-05-08T23:26:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 Processor -- start 2024-05-08T23:26:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 -- start start 2024-05-08T23:26:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cfa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cf9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7ce93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7cef190 con 0x557ad7ce9e00 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7d057c0 con 0x557ad7cea720 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.154+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7cdfc30 con 0x557ad7ce4890 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cf9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cf9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58160/0 (socket says 172.21.15.16:58160) 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 -- 172.21.15.16:0/316160650 learned_addr learned my addr 172.21.15.16:0/316160650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a16d7700 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cfa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7ce93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 -- 172.21.15.16:0/316160650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 msgr2=0x557ad7cfa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cfa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 -- 172.21.15.16:0/316160650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 msgr2=0x557ad7ce93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7ce93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 -- 172.21.15.16:0/316160650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ad7bfa560 con 0x557ad7ce9e00 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a16d7700 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cfa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cf9c20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098e0 tx=0x7f548c0052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a06fe1f1a10f68d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f5487fff700 1 -- 172.21.15.16:0/316160650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0292b0 con 0x557ad7ce9e00 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f5487fff700 1 -- 172.21.15.16:0/316160650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f548c01e040 con 0x557ad7ce9e00 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.155+0000 7f5487fff700 1 -- 172.21.15.16:0/316160650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00bcc0 con 0x557ad7ce9e00 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 -- 172.21.15.16:0/316160650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 msgr2=0x557ad7cf9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cf9c20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098e0 tx=0x7f548c0052c0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 -- 172.21.15.16:0/316160650 shutdown_connections 2024-05-08T23:26:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7cea720 0x557ad7ce93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cfa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/316160650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7ce9e00 0x557ad7cf9c20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 -- 172.21.15.16:0/316160650 >> 172.21.15.16:0/316160650 conn(0x557ad7c42210 msgr2=0x557ad7ce8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 -- 172.21.15.16:0/316160650 shutdown_connections 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 -- 172.21.15.16:0/316160650 wait complete. 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.156+0000 7f54afaf5b80 1 -- start start 2024-05-08T23:26:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 Processor -- start 2024-05-08T23:26:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cf2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce9e00 0x557ad7cf34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 0x557ad7d57aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7cf3c90 con 0x557ad7cea720 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7cf3e00 con 0x557ad7ce9e00 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ad7d58030 con 0x557ad7ce4890 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54afaf5b80 10 client.0 _ll_register_callbacks cb 0x557ad7bd9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 0x557ad7d57aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 0x557ad7d57aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58176/0 (socket says 172.21.15.16:58176) 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 -- 172.21.15.16:0/2689912060 learned_addr learned my addr 172.21.15.16:0/2689912060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a16d7700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cf2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a0ed6700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce9e00 0x557ad7cf34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 -- 172.21.15.16:0/2689912060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 msgr2=0x557ad7cf2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cf2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 -- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce9e00 msgr2=0x557ad7cf34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce9e00 0x557ad7cf34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.157+0000 7f54a1ed8700 1 -- 172.21.15.16:0/2689912060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ad7a61b60 con 0x557ad7cea720 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54a1ed8700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 0x557ad7d57aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f54940c9fb0 tx=0x7f5494099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14dc6e2ed44179e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f5484ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f5484ff9700 1 -- 172.21.15.16:0/2689912060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549409d7d0 con 0x557ad7cea720 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f5484ff9700 1 -- 172.21.15.16:0/2689912060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f54940a1d90 con 0x557ad7cea720 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5498003bf0 con 0x557ad7cea720 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f5484ff9700 1 -- 172.21.15.16:0/2689912060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5494097d40 con 0x557ad7cea720 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 10 client.17040 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f549c10e690 con 0x557ad7cea720 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f5484ff9700 1 -- 172.21.15.16:0/2689912060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5498003bf0 con 0x557ad7cea720 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f5484ff9700 1 -- 172.21.15.16:0/2689912060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54940a1900 con 0x557ad7cea720 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 20 client.17040 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7d59820 0x557ad7d5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 4 client.17040 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.158+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ad7c2cec0 con 0x557ad7d59820 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54a16d7700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7d59820 0x557ad7d5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f5484ff9700 10 client.17040 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54a16d7700 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7d59820 0x557ad7d5bc40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f862d0be60f069 server_cookie=52ee164b4e3c6ffd in_seq=0 out_seq=0 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f5484ff9700 1 -- 172.21.15.16:0/2689912060 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ad7c2cec0 con 0x557ad7d59820 2024-05-08T23:26:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f5484ff9700 10 client.17040 handle_command_reply: tid=0 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 client.17040 shutdown 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 msgr2=0x557ad7d57aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 0x557ad7d57aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f54940c9fb0 tx=0x7f5494099400 comp rx=0 tx=0).stop 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 shutdown_connections 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ad7cea720 0x557ad7d57aa0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ad7ce4890 0x557ad7cf2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557ad7d59820 0x557ad7d5bc40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 --2- 172.21.15.16:0/2689912060 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557ad7ce9e00 0x557ad7cf34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f5484ff9700 0 client.17040 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.159+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 >> 172.21.15.16:0/2689912060 conn(0x557ad7c42210 msgr2=0x557ad7cee4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 shutdown_connections 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54afaf5b80 1 -- 172.21.15.16:0/2689912060 wait complete. 2024-05-08T23:26:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54afaf5b80 20 client.17040 trim_cache size 0 max 16384 2024-05-08T23:26:19.169 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:26:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f54880a1290 msgr2=0x7f54880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f54880a1290 0x7f54880a3750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f549800ac50 tx=0x7f549800a380 comp rx=0 tx=0).stop 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 msgr2=0x7f549c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 0x7f549c19bf90 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f549400ef30 tx=0x7f549400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 shutdown_connections 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f54880a1290 0x7f54880a3750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c19bb10 0x7f549c19bf90 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f549c069420 0x7f549c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 --2- 172.21.15.16:0/1882532451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f549c1062a0 0x7f549c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 >> 172.21.15.16:0/1882532451 conn(0x7f549c076ba0 msgr2=0x7f549c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 shutdown_connections 2024-05-08T23:26:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:19.160+0000 7f54a393b700 1 -- 172.21.15.16:0/1882532451 wait complete. 2024-05-08T23:26:19.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:18 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1052231390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:26:19.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:19 smithi155 ceph-mon[47105]: pgmap v207: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-05-08T23:26:20.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:19 smithi016 ceph-mon[42541]: pgmap v207: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-05-08T23:26:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:19 smithi189 ceph-mon[46919]: pgmap v207: 129 pgs: 129 active+clean; 8.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 12 op/s 2024-05-08T23:26:20.175 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:26:20.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 -- 172.21.15.16:0/1008545068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105160 msgr2=0x7f250c10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 --2- 172.21.15.16:0/1008545068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105160 0x7f250c10cce0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f250400b0a0 tx=0x7f250400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 -- 172.21.15.16:0/1008545068 shutdown_connections 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 --2- 172.21.15.16:0/1008545068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c105160 0x7f250c10cce0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 --2- 172.21.15.16:0/1008545068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c100060 0x7f250c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 --2- 172.21.15.16:0/1008545068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c100a10 0x7f250c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 -- 172.21.15.16:0/1008545068 >> 172.21.15.16:0/1008545068 conn(0x7f250c076f40 msgr2=0x7f250c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 -- 172.21.15.16:0/1008545068 shutdown_connections 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.326+0000 7f2513d51700 1 -- 172.21.15.16:0/1008545068 wait complete. 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 Processor -- start 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 -- start start 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 0x7f250c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c105160 0x7f250c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c19bce0 0x7f250c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c19c760 con 0x7f250c100060 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c19c8d0 con 0x7f250c105160 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2513d51700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c19ca90 con 0x7f250c19bce0 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2511aed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 0x7f250c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2511aed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 0x7f250c197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58188/0 (socket says 172.21.15.16:58188) 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f2511aed700 1 -- 172.21.15.16:0/2644588552 learned_addr learned my addr 172.21.15.16:0/2644588552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.327+0000 7f25122ee700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c19bce0 0x7f250c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f25112ec700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c105160 0x7f250c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2511aed700 1 -- 172.21.15.16:0/2644588552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c19bce0 msgr2=0x7f250c19c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2511aed700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c19bce0 0x7f250c19c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2511aed700 1 -- 172.21.15.16:0/2644588552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c105160 msgr2=0x7f250c197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2511aed700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c105160 0x7f250c197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2511aed700 1 -- 172.21.15.16:0/2644588552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504009d00 con 0x7f250c100060 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2511aed700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 0x7f250c197450 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f250800d950 tx=0x7f250800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f24feffd700 1 -- 172.21.15.16:0/2644588552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508009940 con 0x7f250c100060 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250c1a3ff0 con 0x7f250c100060 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f24feffd700 1 -- 172.21.15.16:0/2644588552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2508010460 con 0x7f250c100060 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f24feffd700 1 -- 172.21.15.16:0/2644588552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250800f620 con 0x7f250c100060 2024-05-08T23:26:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.328+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c1a44c0 con 0x7f250c100060 2024-05-08T23:26:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.330+0000 7f24feffd700 1 -- 172.21.15.16:0/2644588552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f25080105d0 con 0x7f250c100060 2024-05-08T23:26:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.331+0000 7f24feffd700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.331+0000 7f24feffd700 1 -- 172.21.15.16:0/2644588552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f25080ccf10 con 0x7f250c100060 2024-05-08T23:26:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.331+0000 7f25112ec700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.332+0000 7f25112ec700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f250c198950 tx=0x7f250000d4b0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.332+0000 7f251ff0bb80 1 Processor -- start 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.332+0000 7f251ff0bb80 1 -- start start 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcc9c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc9d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcbef1d0 con 0x55b3bcc9cd20 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcbecb90 con 0x55b3bcca2bb0 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcbe8480 con 0x55b3bcca2290 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc9d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc9d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58204/0 (socket says 172.21.15.16:58204) 2024-05-08T23:26:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 -- 172.21.15.16:0/2055824920 learned_addr learned my addr 172.21.15.16:0/2055824920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f2511aed700 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25112ec700 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcc9c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 -- 172.21.15.16:0/2055824920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 msgr2=0x55b3bcca2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 -- 172.21.15.16:0/2055824920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 msgr2=0x55b3bcc9c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcc9c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 -- 172.21.15.16:0/2055824920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3bca69430 con 0x55b3bcc9cd20 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25112ec700 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcc9c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.333+0000 7f25122ee700 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc9d160 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2504007b80 tx=0x7f2504007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5acbb53e7edbce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f24f7fff700 1 -- 172.21.15.16:0/2055824920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400e040 con 0x55b3bcc9cd20 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f24f7fff700 1 -- 172.21.15.16:0/2055824920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2504006020 con 0x55b3bcc9cd20 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f24f7fff700 1 -- 172.21.15.16:0/2055824920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504003e20 con 0x55b3bcc9cd20 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2055824920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 msgr2=0x55b3bcc9d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc9d160 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2504007b80 tx=0x7f2504007c60 comp rx=0 tx=0).stop 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2055824920 shutdown_connections 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc9d160 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2055824920 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcc9c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.334+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2055824920 >> 172.21.15.16:0/2055824920 conn(0x55b3bcbf6290 msgr2=0x55b3bcc92e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.335+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2055824920 shutdown_connections 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.335+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2055824920 wait complete. 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.335+0000 7f251ff0bb80 1 -- start start 2024-05-08T23:26:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.335+0000 7f251ff0bb80 1 Processor -- start 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.335+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcc9cd20 0x55b3bcca8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 0x55b3bcca5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcb845e0 con 0x55b3bcca2290 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcb84280 con 0x55b3bcca2bb0 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bca6b720 con 0x55b3bcc9cd20 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 10 client.0 _ll_register_callbacks cb 0x55b3bcb8dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 0x55b3bcca5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 0x55b3bcca5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58216/0 (socket says 172.21.15.16:58216) 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 -- 172.21.15.16:0/3215051859 learned_addr learned my addr 172.21.15.16:0/3215051859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f2511aed700 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcc9cd20 0x55b3bcca8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 -- 172.21.15.16:0/3215051859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcc9cd20 msgr2=0x55b3bcca8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcc9cd20 0x55b3bcca8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 -- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 msgr2=0x55b3bcca6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 -- 172.21.15.16:0/3215051859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c10d550 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f25112ec700 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 0x55b3bcca5ea0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f250000fee0 tx=0x7f2500004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7445df845d952950 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f24f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f24f57fa700 1 -- 172.21.15.16:0/3215051859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000f820 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f24f57fa700 1 -- 172.21.15.16:0/3215051859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f250001f630 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.336+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3bcb412f0 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f24f57fa700 1 -- 172.21.15.16:0/3215051859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2500011e70 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 10 client.17052 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3bca15be0 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f24f57fa700 1 -- 172.21.15.16:0/3215051859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b3bcb412f0 con 0x55b3bcca2290 2024-05-08T23:26:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f24f57fa700 1 -- 172.21.15.16:0/3215051859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2500025020 con 0x55b3bcca2290 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 10 client.17052 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 20 client.17052 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bccb65c0 0x55b3bccb89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 4 client.17052 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3bcc67890 con 0x55b3bccb65c0 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.337+0000 7f2511aed700 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bccb65c0 0x55b3bccb89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.338+0000 7f24f57fa700 10 client.17052 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.338+0000 7f2511aed700 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bccb65c0 0x55b3bccb89e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14aa5014a035fa46 server_cookie=2dd0acc706bc78e in_seq=0 out_seq=0 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.338+0000 7f24f57fa700 1 -- 172.21.15.16:0/3215051859 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3bcc67890 con 0x55b3bccb65c0 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.338+0000 7f24f57fa700 10 client.17052 handle_command_reply: tid=0 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 client.17052 shutdown 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 msgr2=0x55b3bcca5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 0x55b3bcca5ea0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f250000fee0 tx=0x7f2500004750 comp rx=0 tx=0).stop 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 shutdown_connections 2024-05-08T23:26:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bccb65c0 0x55b3bccb89e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcc9cd20 0x55b3bcca8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f24f57fa700 0 client.17052 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/3215051859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2290 0x55b3bcca5ea0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 >> 172.21.15.16:0/3215051859 conn(0x55b3bcbf6290 msgr2=0x55b3bcc92620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 shutdown_connections 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 1 -- 172.21.15.16:0/3215051859 wait complete. 2024-05-08T23:26:20.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.339+0000 7f251ff0bb80 20 client.17052 trim_cache size 0 max 16384 2024-05-08T23:26:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.357+0000 7f251ff0bb80 1 Processor -- start 2024-05-08T23:26:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.357+0000 7f251ff0bb80 1 -- start start 2024-05-08T23:26:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcca7470 con 0x55b3bcc9cd20 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcc8b0c0 con 0x55b3bcca2bb0 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcc932e0 con 0x55b3bcca2290 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc92da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58230/0 (socket says 172.21.15.16:58230) 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 -- 172.21.15.16:0/1130906923 learned_addr learned my addr 172.21.15.16:0/1130906923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f25122ee700 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f25112ec700 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 -- 172.21.15.16:0/1130906923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 msgr2=0x55b3bcca0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 -- 172.21.15.16:0/1130906923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 msgr2=0x55b3bcca0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 -- 172.21.15.16:0/1130906923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3bcc9d3d0 con 0x55b3bcc9cd20 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f25112ec700 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f2511aed700 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc92da0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2508010880 tx=0x7f2508096090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de44cf33f5eef583 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f24fcff9700 1 -- 172.21.15.16:0/1130906923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080b1070 con 0x55b3bcc9cd20 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f24fcff9700 1 -- 172.21.15.16:0/1130906923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2508099c20 con 0x55b3bcc9cd20 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.358+0000 7f24fcff9700 1 -- 172.21.15.16:0/1130906923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508099e80 con 0x55b3bcc9cd20 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 -- 172.21.15.16:0/1130906923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 msgr2=0x55b3bcc92da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc92da0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2508010880 tx=0x7f2508096090 comp rx=0 tx=0).stop 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 -- 172.21.15.16:0/1130906923 shutdown_connections 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcca2bb0 0x55b3bcca0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcc9cd20 0x55b3bcc92da0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/1130906923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bcca0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 -- 172.21.15.16:0/1130906923 >> 172.21.15.16:0/1130906923 conn(0x55b3bcbf6290 msgr2=0x55b3bccb20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 -- 172.21.15.16:0/1130906923 shutdown_connections 2024-05-08T23:26:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 -- 172.21.15.16:0/1130906923 wait complete. 2024-05-08T23:26:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.359+0000 7f251ff0bb80 1 -- start start 2024-05-08T23:26:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 Processor -- start 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcc9cd20 0x55b3bccad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bccab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 0x55b3bccabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bccac220 con 0x55b3bcca2bb0 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bccac390 con 0x55b3bcc9cd20 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3bcd0da90 con 0x55b3bcca2290 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f251ff0bb80 10 client.0 _ll_register_callbacks cb 0x55b3bcb8dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f25122ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 0x55b3bccabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f25122ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 0x55b3bccabb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58240/0 (socket says 172.21.15.16:58240) 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f25122ee700 1 -- 172.21.15.16:0/2990106209 learned_addr learned my addr 172.21.15.16:0/2990106209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.360+0000 7f25112ec700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bccab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f2511aed700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcc9cd20 0x55b3bccad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f25122ee700 1 -- 172.21.15.16:0/2990106209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 msgr2=0x55b3bccab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f25122ee700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bccab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f25122ee700 1 -- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcc9cd20 msgr2=0x55b3bccad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f25122ee700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcc9cd20 0x55b3bccad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f25122ee700 1 -- 172.21.15.16:0/2990106209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c1a44c0 con 0x55b3bcca2bb0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f25122ee700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 0x55b3bccabb10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f250400b830 tx=0x7f25040076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae479c09058fff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f24f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f24f57fa700 1 -- 172.21.15.16:0/2990106209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400e040 con 0x55b3bcca2bb0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f24f57fa700 1 -- 172.21.15.16:0/2990106209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f2504003a00 con 0x55b3bcca2bb0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2504006020 con 0x55b3bcca2bb0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f24f57fa700 1 -- 172.21.15.16:0/2990106209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2504003bc0 con 0x55b3bcca2bb0 2024-05-08T23:26:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f251ff0bb80 10 client.17058 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.361+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3bcbae5e0 con 0x55b3bcca2bb0 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f24f57fa700 1 -- 172.21.15.16:0/2990106209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2504006020 con 0x55b3bcca2bb0 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f24f57fa700 1 -- 172.21.15.16:0/2990106209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2504019070 con 0x55b3bcca2bb0 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f251ff0bb80 10 client.17058 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f251ff0bb80 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f251ff0bb80 20 client.17058 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bcd0ed20 0x55b3bcd11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f251ff0bb80 4 client.17058 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b3bcbe0f40 con 0x55b3bcd0ed20 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f2511aed700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bcd0ed20 0x55b3bcd11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f2511aed700 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bcd0ed20 0x55b3bcd11110 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e6f8f39c28c30c4 server_cookie=5af8f1522f745d43 in_seq=0 out_seq=0 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.362+0000 7f24f57fa700 10 client.17058 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.363+0000 7f24f57fa700 1 -- 172.21.15.16:0/2990106209 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b3bcbe0f40 con 0x55b3bcd0ed20 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.363+0000 7f24f57fa700 10 client.17058 handle_command_reply: tid=0 2024-05-08T23:26:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.363+0000 7f251ff0bb80 1 client.17058 shutdown 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 msgr2=0x55b3bccabb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 0x55b3bccabb10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f250400b830 tx=0x7f25040076d0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 shutdown_connections 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3bcca2bb0 0x55b3bccabb10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b3bcd0ed20 0x55b3bcd11110 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b3bcc9cd20 0x55b3bccad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 --2- 172.21.15.16:0/2990106209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b3bcca2290 0x55b3bccab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 >> 172.21.15.16:0/2990106209 conn(0x55b3bcbf6290 msgr2=0x55b3bccb20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f24f57fa700 0 client.17058 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 shutdown_connections 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 1 -- 172.21.15.16:0/2990106209 wait complete. 2024-05-08T23:26:20.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.364+0000 7f251ff0bb80 20 client.17058 trim_cache size 0 max 16384 2024-05-08T23:26:20.371 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:26:20.371 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:26:20.371 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5db1443f-0ff5-4fc0-86b1-e71970ddc2d4", 2024-05-08T23:26:20.371 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:26:20.371 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 msgr2=0x7f24f80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f250c198950 tx=0x7f250000d4b0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 msgr2=0x7f250c197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 0x7f250c197450 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f250800d950 tx=0x7f250800dc60 comp rx=0 tx=0).stop 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 shutdown_connections 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f24f80a1290 0x7f24f80a3750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f250c19bce0 0x7f250c19c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c100060 0x7f250c197450 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 --2- 172.21.15.16:0/2644588552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f250c105160 0x7f250c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 >> 172.21.15.16:0/2644588552 conn(0x7f250c076f40 msgr2=0x7f250c068370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 shutdown_connections 2024-05-08T23:26:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.365+0000 7f2513d51700 1 -- 172.21.15.16:0/2644588552 wait complete. 2024-05-08T23:26:20.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 -- 172.21.15.16:0/487264532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c105160 msgr2=0x7fc27c10cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 --2- 172.21.15.16:0/487264532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c105160 0x7fc27c10cce0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc27400b0a0 tx=0x7fc27400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 -- 172.21.15.16:0/487264532 shutdown_connections 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 --2- 172.21.15.16:0/487264532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c105160 0x7fc27c10cce0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 --2- 172.21.15.16:0/487264532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc27c100060 0x7fc27c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 --2- 172.21.15.16:0/487264532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c100a10 0x7fc27c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 -- 172.21.15.16:0/487264532 >> 172.21.15.16:0/487264532 conn(0x7fc27c076f40 msgr2=0x7fc27c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 -- 172.21.15.16:0/487264532 shutdown_connections 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.517+0000 7fc28286c700 1 -- 172.21.15.16:0/487264532 wait complete. 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 Processor -- start 2024-05-08T23:26:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 -- start start 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 0x7fc27c197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc27c105160 0x7fc27c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c19bd70 0x7fc27c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c19c7f0 con 0x7fc27c100060 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c19c960 con 0x7fc27c105160 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc28286c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c19cb20 con 0x7fc27c19bd70 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 0x7fc27c197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 0x7fc27c197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58260/0 (socket says 172.21.15.16:58260) 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc27bfff700 1 -- 172.21.15.16:0/308877316 learned_addr learned my addr 172.21.15.16:0/308877316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.518+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc27c105160 0x7fc27c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc280e09700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c19bd70 0x7fc27c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc27bfff700 1 -- 172.21.15.16:0/308877316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c19bd70 msgr2=0x7fc27c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc27bfff700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c19bd70 0x7fc27c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc27bfff700 1 -- 172.21.15.16:0/308877316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc27c105160 msgr2=0x7fc27c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc27bfff700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc27c105160 0x7fc27c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc27bfff700 1 -- 172.21.15.16:0/308877316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009d00 con 0x7fc27c100060 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc27bfff700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 0x7fc27c197490 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00d950 tx=0x7fc26c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc2797fa700 1 -- 172.21.15.16:0/308877316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c009940 con 0x7fc27c100060 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc2797fa700 1 -- 172.21.15.16:0/308877316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc26c010460 con 0x7fc27c100060 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c1a4080 con 0x7fc27c100060 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc2797fa700 1 -- 172.21.15.16:0/308877316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00f620 con 0x7fc27c100060 2024-05-08T23:26:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.519+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c1a4560 con 0x7fc27c100060 2024-05-08T23:26:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.521+0000 7fc2797fa700 1 -- 172.21.15.16:0/308877316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc26c0105d0 con 0x7fc27c100060 2024-05-08T23:26:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.522+0000 7fc2797fa700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc2640a1290 0x7fc2640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.522+0000 7fc2797fa700 1 -- 172.21.15.16:0/308877316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc26c0cced0 con 0x7fc27c100060 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.522+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc2640a1290 0x7fc2640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc2640a1290 0x7fc2640a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc27c1989e0 tx=0x7fc270009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 Processor -- start 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 -- start start 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d4720 0x55ba230ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba23025150 con 0x55ba230d3e00 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba23022b10 con 0x55ba230d4720 2024-05-08T23:26:20.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2301e400 con 0x55ba230ce890 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58268/0 (socket says 172.21.15.16:58268) 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc27bfff700 1 -- 172.21.15.16:0/2821452508 learned_addr learned my addr 172.21.15.16:0/2821452508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d4720 0x55ba230ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.523+0000 7fc280e09700 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc27bfff700 1 -- 172.21.15.16:0/2821452508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 msgr2=0x55ba230cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc27bfff700 1 -- 172.21.15.16:0/2821452508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d4720 msgr2=0x55ba230ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d4720 0x55ba230ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc27bfff700 1 -- 172.21.15.16:0/2821452508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba22e9f3b0 con 0x55ba230d3e00 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d41e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc26c099bd0 tx=0x7fc26c099620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2adbf02eefb38df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc2627fc700 1 -- 172.21.15.16:0/2821452508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c09a340 con 0x55ba230d3e00 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc2627fc700 1 -- 172.21.15.16:0/2821452508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc26c099c40 con 0x55ba230d3e00 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.524+0000 7fc2627fc700 1 -- 172.21.15.16:0/2821452508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c099330 con 0x55ba230d3e00 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2821452508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 msgr2=0x55ba230d41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d41e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc26c099bd0 tx=0x7fc26c099620 comp rx=0 tx=0).stop 2024-05-08T23:26:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2821452508 shutdown_connections 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d41e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2821452508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d4720 0x55ba230ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2821452508 >> 172.21.15.16:0/2821452508 conn(0x55ba2302c210 msgr2=0x55ba230c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2821452508 shutdown_connections 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2821452508 wait complete. 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.525+0000 7fc28ea26b80 1 -- start start 2024-05-08T23:26:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 Processor -- start 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 0x55ba230d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba22fba560 con 0x55ba230d4720 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba22fba200 con 0x55ba230d3e00 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba22ea16a0 con 0x55ba230ce890 2024-05-08T23:26:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc28ea26b80 10 client.0 _ll_register_callbacks cb 0x55ba22fc3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc27b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc280e09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 0x55ba230d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:52500/0 (socket says 172.21.15.16:52500) 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc27bfff700 1 -- 172.21.15.16:0/2310879900 learned_addr learned my addr 172.21.15.16:0/2310879900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc27b7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230d80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:41238/0 (socket says 172.21.15.16:41238) 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.526+0000 7fc280e09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 0x55ba230d85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58270/0 (socket says 172.21.15.16:58270) 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc280e09700 1 -- 172.21.15.16:0/2310879900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 msgr2=0x55ba230d7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc280e09700 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230d7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc280e09700 1 -- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 msgr2=0x55ba230d80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc280e09700 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230d80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc280e09700 1 -- 172.21.15.16:0/2310879900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c1a4560 con 0x55ba230d4720 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc280e09700 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 0x55ba230d85e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc274009fd0 tx=0x7fc27400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d2447ac9d5812c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc25b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2310879900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e030 con 0x55ba230d4720 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2310879900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc274007c20 con 0x55ba230d4720 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba22f77270 con 0x55ba230d4720 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2310879900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27400be50 con 0x55ba230d4720 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc28ea26b80 10 client.17070 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba22e4bb60 con 0x55ba230d4720 2024-05-08T23:26:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2310879900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ba22f77270 con 0x55ba230d4720 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.527+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2310879900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc27400b0d0 con 0x55ba230d4720 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc28ea26b80 10 client.17070 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc28ea26b80 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc28ea26b80 20 client.17070 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba230e8280 0x55ba230ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc28ea26b80 4 client.17070 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba2309d810 con 0x55ba230e8280 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba230e8280 0x55ba230ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc25b7fe700 10 client.17070 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.528+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba230e8280 0x55ba230ea6a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70252ad0efa90eb1 server_cookie=974e165d3206b646 in_seq=0 out_seq=0 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2310879900 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba2309d810 con 0x55ba230e8280 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc25b7fe700 10 client.17070 handle_command_reply: tid=0 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 client.17070 shutdown 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 msgr2=0x55ba230d85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 0x55ba230d85e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc274009fd0 tx=0x7fc27400ba50 comp rx=0 tx=0).stop 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 shutdown_connections 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d4720 0x55ba230d85e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230ce890 0x55ba230d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba230e8280 0x55ba230ea6a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2310879900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230d80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc25b7fe700 0 client.17070 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 >> 172.21.15.16:0/2310879900 conn(0x55ba2302c210 msgr2=0x55ba230c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 shutdown_connections 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.529+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2310879900 wait complete. 2024-05-08T23:26:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.530+0000 7fc28ea26b80 20 client.17070 trim_cache size 0 max 16384 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 Processor -- start 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 -- start start 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230ce890 0x55ba230ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba230c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba230e5970 con 0x55ba230d3e00 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba230cae90 con 0x55ba230ce890 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba230c9b30 con 0x55ba230d4720 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc27b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc27b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58272/0 (socket says 172.21.15.16:58272) 2024-05-08T23:26:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc27b7fe700 1 -- 172.21.15.16:0/4042241805 learned_addr learned my addr 172.21.15.16:0/4042241805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc27bfff700 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230ce890 0x55ba230ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.548+0000 7fc280e09700 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba230c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc27b7fe700 1 -- 172.21.15.16:0/4042241805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 msgr2=0x55ba230c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba230c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc27b7fe700 1 -- 172.21.15.16:0/4042241805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230ce890 msgr2=0x55ba230ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230ce890 0x55ba230ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc27b7fe700 1 -- 172.21.15.16:0/4042241805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba230cef40 con 0x55ba230d3e00 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc280e09700 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba230c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d34d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc270009fd0 tx=0x7fc270014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70649902f7627f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc262ffd700 1 -- 172.21.15.16:0/4042241805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270024070 con 0x55ba230d3e00 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc262ffd700 1 -- 172.21.15.16:0/4042241805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc270004050 con 0x55ba230d3e00 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc262ffd700 1 -- 172.21.15.16:0/4042241805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700037f0 con 0x55ba230d3e00 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc28ea26b80 1 -- 172.21.15.16:0/4042241805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 msgr2=0x55ba230d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.549+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d34d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc270009fd0 tx=0x7fc270014000 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 -- 172.21.15.16:0/4042241805 shutdown_connections 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba230c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230ce890 0x55ba230ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/4042241805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230d3e00 0x55ba230d34d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 -- 172.21.15.16:0/4042241805 >> 172.21.15.16:0/4042241805 conn(0x55ba2302c210 msgr2=0x55ba230d6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 -- 172.21.15.16:0/4042241805 shutdown_connections 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 -- 172.21.15.16:0/4042241805 wait complete. 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 -- start start 2024-05-08T23:26:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.550+0000 7fc28ea26b80 1 Processor -- start 2024-05-08T23:26:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 0x55ba230de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230de560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba2313e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba230dee40 con 0x55ba230ce890 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2313ecd0 con 0x55ba230d3e00 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2313ee10 con 0x55ba230d4720 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc28ea26b80 10 client.0 _ll_register_callbacks cb 0x55ba22fc3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 0x55ba230de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 0x55ba230de020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58274/0 (socket says 172.21.15.16:58274) 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 -- 172.21.15.16:0/2508304759 learned_addr learned my addr 172.21.15.16:0/2508304759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230de560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc280e09700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba2313e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 -- 172.21.15.16:0/2508304759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 msgr2=0x55ba2313e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba2313e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 -- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 msgr2=0x55ba230de560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230de560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 -- 172.21.15.16:0/2508304759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc270017070 con 0x55ba230ce890 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230de560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc280e09700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba2313e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc27bfff700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 0x55ba230de020 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0966b0 tx=0x7fc26c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05c1d5b9daa17a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.551+0000 7fc25b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2508304759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c099330 con 0x55ba230ce890 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2508304759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc26c099ea0 con 0x55ba230ce890 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc26c099c40 con 0x55ba230ce890 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2508304759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26c0a2140 con 0x55ba230ce890 2024-05-08T23:26:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 10 client.17076 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ba22fe4560 con 0x55ba230ce890 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2508304759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc26c099c40 con 0x55ba230ce890 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2508304759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc26c078d20 con 0x55ba230ce890 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 10 client.17076 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 20 client.17076 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba231401b0 0x55ba231425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 4 client.17076 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba23016ec0 con 0x55ba231401b0 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.552+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba231401b0 0x55ba231425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc27b7fe700 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba231401b0 0x55ba231425d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f0a1ff86784e4d server_cookie=44d3f4f3db9eb615 in_seq=0 out_seq=0 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc25b7fe700 10 client.17076 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc25b7fe700 1 -- 172.21.15.16:0/2508304759 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba23016ec0 con 0x55ba231401b0 2024-05-08T23:26:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc25b7fe700 10 client.17076 handle_command_reply: tid=0 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 client.17076 shutdown 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 msgr2=0x55ba230de020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 0x55ba230de020 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0966b0 tx=0x7fc26c000bc0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 shutdown_connections 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba230d4720 0x55ba2313e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba230ce890 0x55ba230de020 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ba231401b0 0x55ba231425d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc28ea26b80 1 --2- 172.21.15.16:0/2508304759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ba230d3e00 0x55ba230de560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.553+0000 7fc25b7fe700 0 client.17076 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 >> 172.21.15.16:0/2508304759 conn(0x55ba2302c210 msgr2=0x55ba230d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 shutdown_connections 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28ea26b80 1 -- 172.21.15.16:0/2508304759 wait complete. 2024-05-08T23:26:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28ea26b80 20 client.17076 trim_cache size 0 max 16384 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: "5db1443f-0ff5-4fc0-86b1-e71970ddc2d4": { 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "5db1443f-0ff5-4fc0-86b1-e71970ddc2d4", 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:26:20.562 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:26:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc2640a1290 msgr2=0x7fc2640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc2640a1290 0x7fc2640a3750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc27c1989e0 tx=0x7fc270009260 comp rx=0 tx=0).stop 2024-05-08T23:26:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 msgr2=0x7fc27c197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 0x7fc27c197490 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00d950 tx=0x7fc26c00dc60 comp rx=0 tx=0).stop 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 shutdown_connections 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc2640a1290 0x7fc2640a3750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc27c19bd70 0x7fc27c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc27c100060 0x7fc27c197490 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 --2- 172.21.15.16:0/308877316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc27c105160 0x7fc27c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 >> 172.21.15.16:0/308877316 conn(0x7fc27c076f40 msgr2=0x7fc27c068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 shutdown_connections 2024-05-08T23:26:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:20.554+0000 7fc28286c700 1 -- 172.21.15.16:0/308877316 wait complete. 2024-05-08T23:26:20.568 INFO:tasks.cephfs.filesystem:scrub status for tag:5db1443f-0ff5-4fc0-86b1-e71970ddc2d4 - {'path': '/', 'tag': '5db1443f-0ff5-4fc0-86b1-e71970ddc2d4', 'options': 'recursive,force'} 2024-05-08T23:26:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:21 smithi155 ceph-mon[47105]: pgmap v208: 129 pgs: 129 active+clean; 8.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-05-08T23:26:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:21 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:26:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:21 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:21 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:26:21.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:21 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:21 smithi016 ceph-mon[42541]: pgmap v208: 129 pgs: 129 active+clean; 8.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-05-08T23:26:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:21 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:26:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:21 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:21 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:26:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:21 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:21 smithi189 ceph-mon[46919]: pgmap v208: 129 pgs: 129 active+clean; 8.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-05-08T23:26:22.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:21 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:26:22.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:21 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:22.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:21 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:26:22.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:21 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:23.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:23 smithi155 ceph-mon[47105]: pgmap v209: 129 pgs: 129 active+clean; 8.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-05-08T23:26:24.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:23 smithi016 ceph-mon[42541]: pgmap v209: 129 pgs: 129 active+clean; 8.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-05-08T23:26:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:23 smithi189 ceph-mon[46919]: pgmap v209: 129 pgs: 129 active+clean; 8.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 12 op/s 2024-05-08T23:26:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:25 smithi155 ceph-mon[47105]: pgmap v210: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-05-08T23:26:25.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:25 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:26:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:25 smithi016 ceph-mon[42541]: pgmap v210: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-05-08T23:26:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:25 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:26:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:25 smithi189 ceph-mon[46919]: pgmap v210: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-05-08T23:26:26.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:25 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:26:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:27 smithi155 ceph-mon[47105]: pgmap v211: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:27 smithi016 ceph-mon[42541]: pgmap v211: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:27 smithi189 ceph-mon[46919]: pgmap v211: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:29.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:29 smithi155 ceph-mon[47105]: pgmap v212: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:30.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:29 smithi016 ceph-mon[42541]: pgmap v212: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:30.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:29 smithi189 ceph-mon[46919]: pgmap v212: 129 pgs: 129 active+clean; 8.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:31.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:31 smithi155 ceph-mon[47105]: pgmap v213: 129 pgs: 129 active+clean; 8.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:32.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:31 smithi016 ceph-mon[42541]: pgmap v213: 129 pgs: 129 active+clean; 8.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:32.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:31 smithi189 ceph-mon[46919]: pgmap v213: 129 pgs: 129 active+clean; 8.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:33.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:33 smithi155 ceph-mon[47105]: pgmap v214: 129 pgs: 129 active+clean; 8.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:34.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:33 smithi016 ceph-mon[42541]: pgmap v214: 129 pgs: 129 active+clean; 8.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:34.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:33 smithi189 ceph-mon[46919]: pgmap v214: 129 pgs: 129 active+clean; 8.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:35.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:35 smithi155 ceph-mon[47105]: pgmap v215: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-05-08T23:26:36.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:35 smithi016 ceph-mon[42541]: pgmap v215: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-05-08T23:26:36.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:35 smithi189 ceph-mon[46919]: pgmap v215: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-05-08T23:26:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:37 smithi155 ceph-mon[47105]: pgmap v216: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:38.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:37 smithi016 ceph-mon[42541]: pgmap v216: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:38.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:37 smithi189 ceph-mon[46919]: pgmap v216: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:39.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:39 smithi155 ceph-mon[47105]: pgmap v217: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:40.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:39 smithi016 ceph-mon[42541]: pgmap v217: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:40.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:39 smithi189 ceph-mon[46919]: pgmap v217: 129 pgs: 129 active+clean; 8.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:41.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:41 smithi155 ceph-mon[47105]: pgmap v218: 129 pgs: 129 active+clean; 8.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:42.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:41 smithi016 ceph-mon[42541]: pgmap v218: 129 pgs: 129 active+clean; 8.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:42.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:41 smithi189 ceph-mon[46919]: pgmap v218: 129 pgs: 129 active+clean; 8.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:44.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:43 smithi016 ceph-mon[42541]: pgmap v219: 129 pgs: 129 active+clean; 8.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:44.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:43 smithi189 ceph-mon[46919]: pgmap v219: 129 pgs: 129 active+clean; 8.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:43 smithi155 ceph-mon[47105]: pgmap v219: 129 pgs: 129 active+clean; 8.9 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:46.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:45 smithi016 ceph-mon[42541]: pgmap v220: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-05-08T23:26:46.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:45 smithi189 ceph-mon[46919]: pgmap v220: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-05-08T23:26:46.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:45 smithi155 ceph-mon[47105]: pgmap v220: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-05-08T23:26:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:47 smithi016 ceph-mon[42541]: pgmap v221: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:26:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:26:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:26:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:26:48.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:47 smithi189 ceph-mon[46919]: pgmap v221: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:26:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:26:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:26:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:26:48.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:47 smithi155 ceph-mon[47105]: pgmap v221: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:48.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:26:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:26:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:26:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:26:50.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:49 smithi016 ceph-mon[42541]: pgmap v222: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:50.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:49 smithi189 ceph-mon[46919]: pgmap v222: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:50.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:49 smithi155 ceph-mon[47105]: pgmap v222: 129 pgs: 129 active+clean; 9.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2024-05-08T23:26:50.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:26:50.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 -- 172.21.15.16:0/2523442860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 msgr2=0x7ff2e8101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 --2- 172.21.15.16:0/2523442860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8101540 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009990 tx=0x7ff2e4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 -- 172.21.15.16:0/2523442860 shutdown_connections 2024-05-08T23:26:50.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 --2- 172.21.15.16:0/2523442860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e8106280 0x7ff2e810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 --2- 172.21.15.16:0/2523442860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8101540 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 --2- 172.21.15.16:0/2523442860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e8101b10 0x7ff2e8105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 -- 172.21.15.16:0/2523442860 >> 172.21.15.16:0/2523442860 conn(0x7ff2e80fc9c0 msgr2=0x7ff2e80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 -- 172.21.15.16:0/2523442860 shutdown_connections 2024-05-08T23:26:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.733+0000 7ff2f0388700 1 -- 172.21.15.16:0/2523442860 wait complete. 2024-05-08T23:26:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 Processor -- start 2024-05-08T23:26:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 -- start start 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e8106280 0x7ff2e81978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e819bc40 0x7ff2e819c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e819c6c0 con 0x7ff2e8101160 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e819c830 con 0x7ff2e8106280 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2f0388700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e819c9f0 con 0x7ff2e819bc40 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.734+0000 7ff2ee124700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48024/0 (socket says 172.21.15.16:48024) 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 -- 172.21.15.16:0/1583606398 learned_addr learned my addr 172.21.15.16:0/1583606398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee925700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e819bc40 0x7ff2e819c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ed923700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e8106280 0x7ff2e81978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 -- 172.21.15.16:0/1583606398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e819bc40 msgr2=0x7ff2e819c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e819bc40 0x7ff2e819c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 -- 172.21.15.16:0/1583606398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e8106280 msgr2=0x7ff2e81978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e8106280 0x7ff2e81978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 -- 172.21.15.16:0/1583606398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e40095f0 con 0x7ff2e8101160 2024-05-08T23:26:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2ee124700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8197360 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4004730 tx=0x7ff2e4004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2df7fe700 1 -- 172.21.15.16:0/1583606398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e401d070 con 0x7ff2e8101160 2024-05-08T23:26:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2df7fe700 1 -- 172.21.15.16:0/1583606398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2e400bea0 con 0x7ff2e8101160 2024-05-08T23:26:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e81a3ef0 con 0x7ff2e8101160 2024-05-08T23:26:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2df7fe700 1 -- 172.21.15.16:0/1583606398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4022e40 con 0x7ff2e8101160 2024-05-08T23:26:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.735+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e81a4310 con 0x7ff2e8101160 2024-05-08T23:26:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.738+0000 7ff2df7fe700 1 -- 172.21.15.16:0/1583606398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff2e40225e0 con 0x7ff2e8101160 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.738+0000 7ff2df7fe700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff2d40a1310 0x7ff2d40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.739+0000 7ff2ed923700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff2d40a1310 0x7ff2d40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.739+0000 7ff2df7fe700 1 -- 172.21.15.16:0/1583606398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff2e40ce220 con 0x7ff2e8101160 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.739+0000 7ff2ed923700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff2d40a1310 0x7ff2d40a37d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff2e81988b0 tx=0x7ff2d800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.739+0000 7ff2fc542b80 1 Processor -- start 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.739+0000 7ff2fc542b80 1 -- start start 2024-05-08T23:26:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb12e1f0 0x555fbb12e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb128760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb128ca0 0x555fbb129100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb07b150 con 0x555fbb12eb10 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb078b10 con 0x555fbb128ca0 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb074400 con 0x555fbb12e1f0 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb128760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb128760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48040/0 (socket says 172.21.15.16:48040) 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 -- 172.21.15.16:0/3279419822 learned_addr learned my addr 172.21.15.16:0/3279419822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ee124700 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb12e1f0 0x555fbb12e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ee925700 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb128ca0 0x555fbb129100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 -- 172.21.15.16:0/3279419822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb12e1f0 msgr2=0x555fbb12e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb12e1f0 0x555fbb12e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 -- 172.21.15.16:0/3279419822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb128ca0 msgr2=0x555fbb129100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb128ca0 0x555fbb129100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 -- 172.21.15.16:0/3279419822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fbaef53b0 con 0x555fbb12eb10 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.740+0000 7ff2ed923700 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb128760 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8010f40 tx=0x7ff2d8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c0252d449370e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2dcda4700 1 -- 172.21.15.16:0/3279419822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800b950 con 0x555fbb12eb10 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2dcda4700 1 -- 172.21.15.16:0/3279419822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d8015070 con 0x555fbb12eb10 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2dcda4700 1 -- 172.21.15.16:0/3279419822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d801c610 con 0x555fbb12eb10 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 -- 172.21.15.16:0/3279419822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 msgr2=0x555fbb128760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb128760 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8010f40 tx=0x7ff2d8013000 comp rx=0 tx=0).stop 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 -- 172.21.15.16:0/3279419822 shutdown_connections 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb128ca0 0x555fbb129100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb12e1f0 0x555fbb12e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/3279419822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb128760 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.741+0000 7ff2fc542b80 1 -- 172.21.15.16:0/3279419822 >> 172.21.15.16:0/3279419822 conn(0x555fbb082210 msgr2=0x555fbb11edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 -- 172.21.15.16:0/3279419822 shutdown_connections 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 -- 172.21.15.16:0/3279419822 wait complete. 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 -- start start 2024-05-08T23:26:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 Processor -- start 2024-05-08T23:26:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb131ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb132400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb132940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb010560 con 0x555fbb12e1f0 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb010200 con 0x555fbb12eb10 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbaef76a0 con 0x555fbb128ca0 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.742+0000 7ff2fc542b80 10 client.0 _ll_register_callbacks cb 0x555fbb019b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb132400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ee124700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb131ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb132400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 -- 172.21.15.16:0/2561233375 learned_addr learned my addr 172.21.15.16:0/2561233375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ee925700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb132940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 -- 172.21.15.16:0/2561233375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 msgr2=0x555fbb131ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb131ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 -- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 msgr2=0x555fbb132940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb132940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 -- 172.21.15.16:0/2561233375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d8010c20 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb131ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb132400 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005d90 tx=0x7ff2d800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a990c33817e4b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2561233375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d802b070 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2561233375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff2d8017070 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fbafcd270 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2561233375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d8012d80 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2fc542b80 10 client.17088 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.743+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fbaea1b60 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2561233375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555fbafcd270 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2561233375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2d8026a50 con 0x555fbb12e1f0 2024-05-08T23:26:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2fc542b80 10 client.17088 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2fc542b80 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2fc542b80 20 client.17088 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1424f0 0x555fbb144910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2fc542b80 4 client.17088 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fbb0f3810 con 0x555fbb1424f0 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.744+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1424f0 0x555fbb144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2d1ffb700 10 client.17088 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1424f0 0x555fbb144910 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4295cd53e9d04b server_cookie=f566b917006c1381 in_seq=0 out_seq=0 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2561233375 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fbb0f3810 con 0x555fbb1424f0 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2d1ffb700 10 client.17088 handle_command_reply: tid=0 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 client.17088 shutdown 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 msgr2=0x555fbb132400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb132400 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005d90 tx=0x7ff2d800bed0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 shutdown_connections 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1424f0 0x555fbb144910 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb132940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb131ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2d1ffb700 0 client.17088 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2561233375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb132400 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 >> 172.21.15.16:0/2561233375 conn(0x555fbb082210 msgr2=0x555fbb11e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.745+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 shutdown_connections 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.746+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2561233375 wait complete. 2024-05-08T23:26:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.746+0000 7ff2fc542b80 20 client.17088 trim_cache size 0 max 16384 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 Processor -- start 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 -- start start 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb11ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb139a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb12e950 con 0x555fbb12e1f0 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb120640 con 0x555fbb12eb10 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb149b00 con 0x555fbb128ca0 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2ed923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2ed923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48062/0 (socket says 172.21.15.16:48062) 2024-05-08T23:26:50.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2ed923700 1 -- 172.21.15.16:0/2660569183 learned_addr learned my addr 172.21.15.16:0/2660569183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.764+0000 7ff2ee925700 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb139a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb11ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ed923700 1 -- 172.21.15.16:0/2660569183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 msgr2=0x555fbb11ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb11ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ed923700 1 -- 172.21.15.16:0/2660569183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 msgr2=0x555fbb139a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb139a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ed923700 1 -- 172.21.15.16:0/2660569183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fbb03a560 con 0x555fbb12e1f0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ee925700 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb139a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb139550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800b510 tx=0x7ff2d8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1c23089c12dac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2dd5a5700 1 -- 172.21.15.16:0/2660569183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d802b030 con 0x555fbb12e1f0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2dd5a5700 1 -- 172.21.15.16:0/2660569183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d801a4f0 con 0x555fbb12e1f0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.765+0000 7ff2dd5a5700 1 -- 172.21.15.16:0/2660569183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8004690 con 0x555fbb12e1f0 2024-05-08T23:26:50.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2660569183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 msgr2=0x555fbb139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb139550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800b510 tx=0x7ff2d8005a10 comp rx=0 tx=0).stop 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2660569183 shutdown_connections 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12eb10 0x555fbb139a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb11ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2660569183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12e1f0 0x555fbb139550 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2660569183 >> 172.21.15.16:0/2660569183 conn(0x555fbb082210 msgr2=0x555fbb129080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2660569183 shutdown_connections 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2660569183 wait complete. 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- start start 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 Processor -- start 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb135530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12e1f0 0x555fbb135a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb197a30 con 0x555fbb12eb10 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb197ba0 con 0x555fbb12e1f0 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb197d60 con 0x555fbb128ca0 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.766+0000 7ff2fc542b80 10 client.0 _ll_register_callbacks cb 0x555fbb019b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb135fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48074/0 (socket says 172.21.15.16:48074) 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 -- 172.21.15.16:0/2500566070 learned_addr learned my addr 172.21.15.16:0/2500566070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb135530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ed923700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12e1f0 0x555fbb135a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 -- 172.21.15.16:0/2500566070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 msgr2=0x555fbb135530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb135530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 -- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12e1f0 msgr2=0x555fbb135a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:50.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12e1f0 0x555fbb135a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 -- 172.21.15.16:0/2500566070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fbaea1b60 con 0x555fbb12eb10 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2ee925700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb135fb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0007bc0 tx=0x7ff2e0007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7108dedc36f9d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2500566070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000e040 con 0x555fbb12eb10 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2500566070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff2e0003e10 con 0x555fbb12eb10 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.767+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2d8015070 con 0x555fbb12eb10 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2500566070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e0004bb0 con 0x555fbb12eb10 2024-05-08T23:26:50.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 10 client.17094 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fbaef9cd0 con 0x555fbb12eb10 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2500566070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d8015070 con 0x555fbb12eb10 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2500566070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2e0017620 con 0x555fbb12eb10 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 10 client.17094 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 20 client.17094 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1994f0 0x555fbb19b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 4 client.17094 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555fbb06cec0 con 0x555fbb1994f0 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.768+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1994f0 0x555fbb19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2ee124700 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1994f0 0x555fbb19b910 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cfd0b321de0d5f5 server_cookie=c53ebe379f7cbd5b in_seq=0 out_seq=0 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2d1ffb700 10 client.17094 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:50.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2d1ffb700 1 -- 172.21.15.16:0/2500566070 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555fbb06cec0 con 0x555fbb1994f0 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2d1ffb700 10 client.17094 handle_command_reply: tid=0 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 client.17094 shutdown 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 msgr2=0x555fbb135fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb135fb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0007bc0 tx=0x7ff2e0007bf0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 shutdown_connections 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fbb12eb10 0x555fbb135fb0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb128ca0 0x555fbb135530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555fbb1994f0 0x555fbb19b910 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 --2- 172.21.15.16:0/2500566070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555fbb12e1f0 0x555fbb135a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 >> 172.21.15.16:0/2500566070 conn(0x555fbb082210 msgr2=0x555fbb12cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2d1ffb700 0 client.17094 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 shutdown_connections 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 1 -- 172.21.15.16:0/2500566070 wait complete. 2024-05-08T23:26:50.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.769+0000 7ff2fc542b80 20 client.17094 trim_cache size 0 max 16384 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:26:50.777 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff2d40a1310 msgr2=0x7ff2d40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff2d40a1310 0x7ff2d40a37d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff2e81988b0 tx=0x7ff2d800b410 comp rx=0 tx=0).stop 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 msgr2=0x7ff2e8197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8197360 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4004730 tx=0x7ff2e4004760 comp rx=0 tx=0).stop 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 shutdown_connections 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff2d40a1310 0x7ff2d40a37d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e819bc40 0x7ff2e819c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2e8101160 0x7ff2e8197360 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 --2- 172.21.15.16:0/1583606398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e8106280 0x7ff2e81978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 >> 172.21.15.16:0/1583606398 conn(0x7ff2e80fc9c0 msgr2=0x7ff2e80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 shutdown_connections 2024-05-08T23:26:50.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.770+0000 7ff2f0388700 1 -- 172.21.15.16:0/1583606398 wait complete. 2024-05-08T23:26:50.783 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:26:50.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:26:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 -- 172.21.15.16:0/2772940840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102000 msgr2=0x7f896c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 --2- 172.21.15.16:0/2772940840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102000 0x7f896c1023e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8960009990 tx=0x7f8960009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 -- 172.21.15.16:0/2772940840 shutdown_connections 2024-05-08T23:26:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 --2- 172.21.15.16:0/2772940840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c10b3d0 0x7f896c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 --2- 172.21.15.16:0/2772940840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102000 0x7f896c1023e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 --2- 172.21.15.16:0/2772940840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f896c102920 0x7f896c10ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 -- 172.21.15.16:0/2772940840 >> 172.21.15.16:0/2772940840 conn(0x7f896c0fb7e0 msgr2=0x7f896c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 -- 172.21.15.16:0/2772940840 shutdown_connections 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.920+0000 7f8972d63700 1 -- 172.21.15.16:0/2772940840 wait complete. 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 Processor -- start 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 -- start start 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c102000 0x7f896c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 0x7f896c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f896c10b3d0 0x7f896c19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c19c140 con 0x7f896c102920 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c19c2b0 con 0x7f896c10b3d0 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f8972d63700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c19c470 con 0x7f896c102000 2024-05-08T23:26:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f896bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 0x7f896c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f896bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 0x7f896c197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48092/0 (socket says 172.21.15.16:48092) 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.921+0000 7f896bfff700 1 -- 172.21.15.16:0/1077792516 learned_addr learned my addr 172.21.15.16:0/1077792516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8970aff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c102000 0x7f896c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8971300700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f896c10b3d0 0x7f896c19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f896bfff700 1 -- 172.21.15.16:0/1077792516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c102000 msgr2=0x7f896c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f896bfff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c102000 0x7f896c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f896bfff700 1 -- 172.21.15.16:0/1077792516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f896c10b3d0 msgr2=0x7f896c19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f896bfff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f896c10b3d0 0x7f896c19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f896bfff700 1 -- 172.21.15.16:0/1077792516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600095f0 con 0x7f896c102920 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8970aff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c102000 0x7f896c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f896bfff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 0x7f896c197820 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f895800e9a0 tx=0x7f895800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958009790 con 0x7f896c102920 2024-05-08T23:26:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f895800cbd0 con 0x7f896c102920 2024-05-08T23:26:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c19c750 con 0x7f896c102920 2024-05-08T23:26:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958018680 con 0x7f896c102920 2024-05-08T23:26:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.922+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c1a3fb0 con 0x7f896c102920 2024-05-08T23:26:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.923+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c106550 con 0x7f896c102920 2024-05-08T23:26:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.925+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f895800cd40 con 0x7f896c102920 2024-05-08T23:26:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.926+0000 7f8969ffb700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f895c0a1310 0x7f895c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.926+0000 7f8970aff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f895c0a1310 0x7f895c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.926+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f8958014070 con 0x7f896c102920 2024-05-08T23:26:50.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.926+0000 7f8970aff700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f895c0a1310 0x7f895c0a37d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8960006010 tx=0x7f8960005ab0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:50.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:50.931+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f89580963d0 con 0x7f896c102920 2024-05-08T23:26:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.201+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f896c110c20 con 0x7f896c102920 2024-05-08T23:26:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.202+0000 7f8969ffb700 1 -- 172.21.15.16:0/1077792516 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f89580999f0 con 0x7f896c102920 2024-05-08T23:26:51.202 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:26:51.202 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f895c0a1310 msgr2=0x7f895c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f895c0a1310 0x7f895c0a37d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8960006010 tx=0x7f8960005ab0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 msgr2=0x7f896c197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 0x7f896c197820 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f895800e9a0 tx=0x7f895800ed60 comp rx=0 tx=0).stop 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 shutdown_connections 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f895c0a1310 0x7f895c0a37d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f896c10b3d0 0x7f896c19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f896c102000 0x7f896c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 --2- 172.21.15.16:0/1077792516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f896c102920 0x7f896c197820 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 >> 172.21.15.16:0/1077792516 conn(0x7f896c0fb7e0 msgr2=0x7f896c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 shutdown_connections 2024-05-08T23:26:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.205+0000 7f8972d63700 1 -- 172.21.15.16:0/1077792516 wait complete. 2024-05-08T23:26:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:26:51.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 -- 172.21.15.16:0/576167313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 msgr2=0x7fa968100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 --2- 172.21.15.16:0/576167313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968100440 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa958009990 tx=0x7fa958009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 -- 172.21.15.16:0/576167313 shutdown_connections 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 --2- 172.21.15.16:0/576167313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa968105160 0x7fa96810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 --2- 172.21.15.16:0/576167313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968100440 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 --2- 172.21.15.16:0/576167313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa968100a10 0x7fa968104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 -- 172.21.15.16:0/576167313 >> 172.21.15.16:0/576167313 conn(0x7fa968076f40 msgr2=0x7fa968077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 -- 172.21.15.16:0/576167313 shutdown_connections 2024-05-08T23:26:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.359+0000 7fa96ca15700 1 -- 172.21.15.16:0/576167313 wait complete. 2024-05-08T23:26:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 Processor -- start 2024-05-08T23:26:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 -- start start 2024-05-08T23:26:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa968105160 0x7fa968196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa96819ab00 0x7fa96819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819b580 con 0x7fa968100060 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819b6f0 con 0x7fa968105160 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96ca15700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96819b8b0 con 0x7fa96819ab00 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.360+0000 7fa96659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48104/0 (socket says 172.21.15.16:48104) 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 -- 172.21.15.16:0/939737116 learned_addr learned my addr 172.21.15.16:0/939737116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa966d9d700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa96819ab00 0x7fa96819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa965d9b700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa968105160 0x7fa968196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 -- 172.21.15.16:0/939737116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa96819ab00 msgr2=0x7fa96819af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa96819ab00 0x7fa96819af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 -- 172.21.15.16:0/939737116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa968105160 msgr2=0x7fa968196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa968105160 0x7fa968196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 -- 172.21.15.16:0/939737116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580095f0 con 0x7fa968100060 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa966d9d700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa96819ab00 0x7fa96819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96659c700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968196220 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa958000c00 tx=0x7fa958005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa9577fe700 1 -- 172.21.15.16:0/939737116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95801d070 con 0x7fa968100060 2024-05-08T23:26:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9681a2db0 con 0x7fa968100060 2024-05-08T23:26:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa9577fe700 1 -- 172.21.15.16:0/939737116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa958021410 con 0x7fa968100060 2024-05-08T23:26:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa9577fe700 1 -- 172.21.15.16:0/939737116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958021ea0 con 0x7fa968100060 2024-05-08T23:26:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.361+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9681a31d0 con 0x7fa968100060 2024-05-08T23:26:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.364+0000 7fa9577fe700 1 -- 172.21.15.16:0/939737116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa958021580 con 0x7fa968100060 2024-05-08T23:26:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.364+0000 7fa9577fe700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa9500a1290 0x7fa9500a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.364+0000 7fa9577fe700 1 -- 172.21.15.16:0/939737116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fa9580ce2c0 con 0x7fa968100060 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.364+0000 7fa965d9b700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa9500a1290 0x7fa9500a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.365+0000 7fa965d9b700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa9500a1290 0x7fa9500a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa968197770 tx=0x7fa95c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.365+0000 7fa978bcfb80 1 Processor -- start 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.365+0000 7fa978bcfb80 1 -- start start 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 0x55cbca48f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca49a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca49cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca3e5150 con 0x55cbca48f060 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca3e2b10 con 0x55cbca49a860 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca3de400 con 0x55cbca48f980 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 0x55cbca48f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 0x55cbca48f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48118/0 (socket says 172.21.15.16:48118) 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 -- 172.21.15.16:0/2764838105 learned_addr learned my addr 172.21.15.16:0/2764838105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa965d9b700 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca49a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa966d9d700 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca49cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 -- 172.21.15.16:0/2764838105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 msgr2=0x55cbca49a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca49a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 -- 172.21.15.16:0/2764838105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 msgr2=0x55cbca49cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca49cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 -- 172.21.15.16:0/2764838105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbca25f3b0 con 0x55cbca48f060 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa96659c700 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 0x55cbca48f440 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa958097b30 tx=0x7fa958005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ceddd3b74fac87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.366+0000 7fa954ff9700 1 -- 172.21.15.16:0/2764838105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95809eef0 con 0x55cbca48f060 2024-05-08T23:26:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa954ff9700 1 -- 172.21.15.16:0/2764838105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9580a34b0 con 0x55cbca48f060 2024-05-08T23:26:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa954ff9700 1 -- 172.21.15.16:0/2764838105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95809b560 con 0x55cbca48f060 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 -- 172.21.15.16:0/2764838105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 msgr2=0x55cbca48f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 0x55cbca48f440 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa958097b30 tx=0x7fa958005c80 comp rx=0 tx=0).stop 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 -- 172.21.15.16:0/2764838105 shutdown_connections 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca49cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f060 0x55cbca48f440 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/2764838105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca49a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 -- 172.21.15.16:0/2764838105 >> 172.21.15.16:0/2764838105 conn(0x55cbca3ec210 msgr2=0x55cbca48b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 -- 172.21.15.16:0/2764838105 shutdown_connections 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.367+0000 7fa978bcfb80 1 -- 172.21.15.16:0/2764838105 wait complete. 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 -- start start 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 Processor -- start 2024-05-08T23:26:51.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca49ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca49b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca49a860 0x55cbca4a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca37a560 con 0x55cbca48f980 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca37a200 con 0x55cbca48f060 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca2616a0 con 0x55cbca49a860 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa978bcfb80 10 client.0 _ll_register_callbacks cb 0x55cbca383b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa965d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca49b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa965d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca49b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48126/0 (socket says 172.21.15.16:48126) 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa965d9b700 1 -- 172.21.15.16:0/216439576 learned_addr learned my addr 172.21.15.16:0/216439576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.368+0000 7fa96659c700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca49ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa966d9d700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca49a860 0x55cbca4a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa96659c700 1 -- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca49a860 msgr2=0x55cbca4a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa96659c700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca49a860 0x55cbca4a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa96659c700 1 -- 172.21.15.16:0/216439576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 msgr2=0x55cbca49b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa96659c700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca49b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa96659c700 1 -- 172.21.15.16:0/216439576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9681a31d0 con 0x55cbca48f060 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa965d9b700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca49b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa96659c700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca49ae30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa958097b30 tx=0x7fa958099670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f468f379daf19c1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa94dffb700 10 client.0 ms_handle_connect on v2:172.21.15.155:3300/0 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa94dffb700 1 -- 172.21.15.16:0/216439576 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa958021a60 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa94dffb700 1 -- 172.21.15.16:0/216439576 <== mon.2 v2:172.21.15.155:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fa95809a5b0 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbca337270 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa94dffb700 1 -- 172.21.15.16:0/216439576 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa95809a770 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa978bcfb80 10 client.26711 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.369+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cbca20bb60 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa94dffb700 1 -- 172.21.15.16:0/216439576 <== mon.2 v2:172.21.15.155:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cbca337270 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa94dffb700 1 -- 172.21.15.16:0/216439576 <== mon.2 v2:172.21.15.155:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa95809b320 con 0x55cbca48f060 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa978bcfb80 10 client.26711 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa978bcfb80 10 client.26711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:26:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa978bcfb80 20 client.26711 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca4a9c90 0x55cbca4ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa978bcfb80 4 client.26711 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbca45d810 con 0x55cbca4a9c90 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.370+0000 7fa965d9b700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca4a9c90 0x55cbca4ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.371+0000 7fa94dffb700 10 client.26711 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.371+0000 7fa965d9b700 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca4a9c90 0x55cbca4ac0b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=516625c14cda4887 server_cookie=86dde1c786aeb00d in_seq=0 out_seq=0 2024-05-08T23:26:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.371+0000 7fa94dffb700 1 -- 172.21.15.16:0/216439576 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbca45d810 con 0x55cbca4a9c90 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.371+0000 7fa94dffb700 10 client.26711 handle_command_reply: tid=0 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.371+0000 7fa978bcfb80 1 client.26711 shutdown 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 msgr2=0x55cbca49ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca49ae30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa958097b30 tx=0x7fa958099670 comp rx=0 tx=0).stop 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 shutdown_connections 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca49a860 0x55cbca4a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca4a9c90 0x55cbca4ac0b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca49ae30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/216439576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca49b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa94dffb700 0 client.26711 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 >> 172.21.15.16:0/216439576 conn(0x55cbca3ec210 msgr2=0x55cbca491060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 shutdown_connections 2024-05-08T23:26:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 1 -- 172.21.15.16:0/216439576 wait complete. 2024-05-08T23:26:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.372+0000 7fa978bcfb80 20 client.26711 trim_cache size 0 max 16384 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 Processor -- start 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 -- start start 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f060 0x55cbca49fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca4a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca493fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca48ae80 con 0x55cbca48f980 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca4b1280 con 0x55cbca49a860 2024-05-08T23:26:51.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.386+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca48fd60 con 0x55cbca48f060 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca4a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca4a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48138/0 (socket says 172.21.15.16:48138) 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 -- 172.21.15.16:0/4094492759 learned_addr learned my addr 172.21.15.16:0/4094492759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa96659c700 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f060 0x55cbca49fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa966d9d700 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca493fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 -- 172.21.15.16:0/4094492759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f060 msgr2=0x55cbca49fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f060 0x55cbca49fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 -- 172.21.15.16:0/4094492759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 msgr2=0x55cbca493fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca493fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 -- 172.21.15.16:0/4094492759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbca4854e0 con 0x55cbca48f980 2024-05-08T23:26:51.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa966d9d700 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca493fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa965d9b700 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca4a0160 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x55cbca4ac980 tx=0x7fa95c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5dafe2b0198182 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa9557fa700 1 -- 172.21.15.16:0/4094492759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c025040 con 0x55cbca48f980 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa9557fa700 1 -- 172.21.15.16:0/4094492759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa95c016070 con 0x55cbca48f980 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.387+0000 7fa9557fa700 1 -- 172.21.15.16:0/4094492759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0038e0 con 0x55cbca48f980 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 -- 172.21.15.16:0/4094492759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 msgr2=0x55cbca4a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca4a0160 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x55cbca4ac980 tx=0x7fa95c014000 comp rx=0 tx=0).stop 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 -- 172.21.15.16:0/4094492759 shutdown_connections 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca49a860 0x55cbca493fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f060 0x55cbca49fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/4094492759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca48f980 0x55cbca4a0160 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 -- 172.21.15.16:0/4094492759 >> 172.21.15.16:0/4094492759 conn(0x55cbca3ec210 msgr2=0x55cbca48b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 -- 172.21.15.16:0/4094492759 shutdown_connections 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 -- 172.21.15.16:0/4094492759 wait complete. 2024-05-08T23:26:51.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.388+0000 7fa978bcfb80 1 -- start start 2024-05-08T23:26:51.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 Processor -- start 2024-05-08T23:26:51.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca4f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca4f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 0x55cbca4f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca4ff1f0 con 0x55cbca49a860 2024-05-08T23:26:51.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca4ff330 con 0x55cbca48f060 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbca4ff4f0 con 0x55cbca48f980 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa978bcfb80 10 client.0 _ll_register_callbacks cb 0x55cbca383b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 0x55cbca4f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 0x55cbca4f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48154/0 (socket says 172.21.15.16:48154) 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 -- 172.21.15.16:0/642126786 learned_addr learned my addr 172.21.15.16:0/642126786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa965d9b700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca4f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa96659c700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca4f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 -- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 msgr2=0x55cbca4f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca4f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 -- 172.21.15.16:0/642126786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 msgr2=0x55cbca4f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca4f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 -- 172.21.15.16:0/642126786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c0126f0 con 0x55cbca49a860 2024-05-08T23:26:51.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa966d9d700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 0x55cbca4f4b60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa960007b70 tx=0x7fa960007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb6de5ebea953d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.389+0000 7fa94dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa94dffb700 1 -- 172.21.15.16:0/642126786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960003c00 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9580a34b0 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa94dffb700 1 -- 172.21.15.16:0/642126786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fa960003d60 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa94dffb700 1 -- 172.21.15.16:0/642126786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa960007e90 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 10 client.17118 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cbca3a4560 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa94dffb700 1 -- 172.21.15.16:0/642126786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9580a34b0 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa94dffb700 1 -- 172.21.15.16:0/642126786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9600044c0 con 0x55cbca49a860 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 10 client.17118 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 10 client.17118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 20 client.17118 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca500d50 0x55cbca503170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 4 client.17118 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:26:51.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.390+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbca3d6ec0 con 0x55cbca500d50 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa96659c700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca500d50 0x55cbca503170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa96659c700 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca500d50 0x55cbca503170 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=add9da873bceff8a server_cookie=c4ca084ec4a9ec89 in_seq=0 out_seq=0 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa94dffb700 10 client.17118 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa94dffb700 1 -- 172.21.15.16:0/642126786 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbca3d6ec0 con 0x55cbca500d50 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa94dffb700 10 client.17118 handle_command_reply: tid=0 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa978bcfb80 1 client.17118 shutdown 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 msgr2=0x55cbca4f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.391+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 0x55cbca4f4b60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa960007b70 tx=0x7fa960007c50 comp rx=0 tx=0).stop 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 shutdown_connections 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbca49a860 0x55cbca4f4b60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55cbca500d50 0x55cbca503170 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55cbca48f060 0x55cbca4f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 --2- 172.21.15.16:0/642126786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbca48f980 0x55cbca4f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 >> 172.21.15.16:0/642126786 conn(0x55cbca3ec210 msgr2=0x55cbca493fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa94dffb700 0 client.17118 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:26:51.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 shutdown_connections 2024-05-08T23:26:51.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 1 -- 172.21.15.16:0/642126786 wait complete. 2024-05-08T23:26:51.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa978bcfb80 20 client.17118 trim_cache size 0 max 16384 2024-05-08T23:26:51.398 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa9500a1290 msgr2=0x7fa9500a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa9500a1290 0x7fa9500a3750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa968197770 tx=0x7fa95c009260 comp rx=0 tx=0).stop 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 msgr2=0x7fa968196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968196220 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa958000c00 tx=0x7fa958005f00 comp rx=0 tx=0).stop 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 shutdown_connections 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa9500a1290 0x7fa9500a3750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa96819ab00 0x7fa96819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa968100060 0x7fa968196220 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 --2- 172.21.15.16:0/939737116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa968105160 0x7fa968196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 >> 172.21.15.16:0/939737116 conn(0x7fa968076f40 msgr2=0x7fa9680fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 shutdown_connections 2024-05-08T23:26:51.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.392+0000 7fa96ca15700 1 -- 172.21.15.16:0/939737116 wait complete. 2024-05-08T23:26:51.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:251/600: open fd = 3 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:252/600: open fd = 3 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:253/600: open fd = 3 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.473 INFO:tasks.workunit.client.0.smithi016.stdout:254/600: open fd = 3 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:255/600: open fd = 3 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:256/600: open fd = 3 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:257/600: open fd = 3 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.474 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:258/600: open fd = 3 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:259/600: open fd = 3 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:260/600: open fd = 3 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:261/600: open fd = 3 2024-05-08T23:26:51.475 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:262/600: open fd = 3 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:263/600: open fd = 3 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:264/600: open fd = 3 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.476 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:265/600: open fd = 3 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:266/600: open fd = 3 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:267/600: open fd = 3 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:268/600: open fd = 3 2024-05-08T23:26:51.477 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:269/600: open fd = 3 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:270/600: open fd = 3 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:271/600: open fd = 3 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.478 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:272/600: open fd = 3 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:273/600: open fd = 3 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:274/600: open fd = 3 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:275/600: open fd = 3 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.479 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:276/600: open fd = 3 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:277/600: open fd = 3 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:278/600: open fd = 3 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:279/600: open fd = 3 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.480 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:280/600: open fd = 3 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:281/600: open fd = 3 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:282/600: open fd = 3 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.481 INFO:tasks.workunit.client.0.smithi016.stdout:283/600: open fd = 3 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:284/600: open fd = 3 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:285/600: open fd = 3 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:286/600: open fd = 3 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.482 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:287/600: open fd = 3 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:288/600: open fd = 3 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:289/600: open fd = 3 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:290/600: open fd = 3 2024-05-08T23:26:51.483 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:291/600: open fd = 3 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:292/600: open fd = 3 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:293/600: open fd = 3 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.484 INFO:tasks.workunit.client.0.smithi016.stdout:294/600: open fd = 3 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:295/600: open fd = 3 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:296/600: open fd = 3 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:297/600: open fd = 3 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.485 INFO:tasks.workunit.client.0.smithi016.stdout:298/600: open fd = 3 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:299/600: open fd = 3 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:300/600: open fd = 3 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:301/600: open fd = 3 2024-05-08T23:26:51.486 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:302/600: open fd = 3 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:303/600: open fd = 3 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:304/600: open fd = 3 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.487 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:305/600: open fd = 3 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:306/600: open fd = 3 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:307/600: open fd = 3 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:308/600: open fd = 3 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.488 INFO:tasks.workunit.client.0.smithi016.stdout:309/600: open fd = 3 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:310/600: open fd = 3 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:311/600: open fd = 3 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:312/600: open fd = 3 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.489 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:26:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 -- 172.21.15.16:0/3003588654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230101180 msgr2=0x7fd230101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 --2- 172.21.15.16:0/3003588654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230101180 0x7fd230101560 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd218009990 tx=0x7fd218009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 -- 172.21.15.16:0/3003588654 shutdown_connections 2024-05-08T23:26:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 --2- 172.21.15.16:0/3003588654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd23010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 --2- 172.21.15.16:0/3003588654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230101180 0x7fd230101560 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 --2- 172.21.15.16:0/3003588654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101b30 0x7fd230105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 -- 172.21.15.16:0/3003588654 >> 172.21.15.16:0/3003588654 conn(0x7fd2300fc9e0 msgr2=0x7fd2300fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 -- 172.21.15.16:0/3003588654 shutdown_connections 2024-05-08T23:26:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.544+0000 7fd2351b5700 1 -- 172.21.15.16:0/3003588654 wait complete. 2024-05-08T23:26:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.545+0000 7fd2351b5700 1 Processor -- start 2024-05-08T23:26:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.545+0000 7fd2351b5700 1 -- start start 2024-05-08T23:26:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.545+0000 7fd2351b5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101180 0x7fd230199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.545+0000 7fd2351b5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd230199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.545+0000 7fd2351b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 0x7fd23019e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd2351b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23019e8d0 con 0x7fd23019de50 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd2351b5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23019ea40 con 0x7fd230101180 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd2351b5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23019ec00 con 0x7fd2301062a0 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 0x7fd23019e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd230199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 0x7fd23019e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48172/0 (socket says 172.21.15.16:48172) 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd230199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:35996/0 (socket says 172.21.15.16:35996) 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22f59e700 1 -- 172.21.15.16:0/2294675755 learned_addr learned my addr 172.21.15.16:0/2294675755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101180 0x7fd230199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 -- 172.21.15.16:0/2294675755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101180 msgr2=0x7fd230199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101180 0x7fd230199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 -- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 msgr2=0x7fd23019e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 0x7fd23019e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 -- 172.21.15.16:0/2294675755 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180095f0 con 0x7fd2301062a0 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22f59e700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 0x7fd23019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101180 0x7fd230199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.546+0000 7fd22e59c700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd230199ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd23006a5d0 tx=0x7fd22000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.547+0000 7fd227fff700 1 -- 172.21.15.16:0/2294675755 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220009930 con 0x7fd2301062a0 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.547+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2301a6160 con 0x7fd2301062a0 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.547+0000 7fd227fff700 1 -- 172.21.15.16:0/2294675755 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd220010460 con 0x7fd2301062a0 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.547+0000 7fd227fff700 1 -- 172.21.15.16:0/2294675755 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000b5d0 con 0x7fd2301062a0 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.547+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2301a6690 con 0x7fd2301062a0 2024-05-08T23:26:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.549+0000 7fd227fff700 1 -- 172.21.15.16:0/2294675755 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd220009a90 con 0x7fd2301062a0 2024-05-08T23:26:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.550+0000 7fd227fff700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd21c0a1290 0x7fd21c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.550+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd21c0a1290 0x7fd21c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.550+0000 7fd227fff700 1 -- 172.21.15.16:0/2294675755 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fd2200ccdc0 con 0x7fd2301062a0 2024-05-08T23:26:51.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 Processor -- start 2024-05-08T23:26:51.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 -- start start 2024-05-08T23:26:51.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd21c0a1290 0x7fd21c0a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd21800b5c0 tx=0x7fd218005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df72890 0x562c1df72cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dec9150 con 0x562c1df78720 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dec6b10 con 0x562c1df72890 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd24136fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dec2400 con 0x562c1df77e00 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.551+0000 7fd22e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df72350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48178/0 (socket says 172.21.15.16:48178) 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 -- 172.21.15.16:0/3831704784 learned_addr learned my addr 172.21.15.16:0/3831704784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22f59e700 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df72890 0x562c1df72cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 -- 172.21.15.16:0/3831704784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 msgr2=0x562c1df781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 -- 172.21.15.16:0/3831704784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df72890 msgr2=0x562c1df72cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df72890 0x562c1df72cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 -- 172.21.15.16:0/3831704784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1dd433b0 con 0x562c1df78720 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22f59e700 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df72890 0x562c1df72cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd22e59c700 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df72350 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd22000dfd0 tx=0x7fd220099530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce9fc3417b32a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd2257fa700 1 -- 172.21.15.16:0/3831704784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220010ea0 con 0x562c1df78720 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd2257fa700 1 -- 172.21.15.16:0/3831704784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd22009d900 con 0x562c1df78720 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.552+0000 7fd2257fa700 1 -- 172.21.15.16:0/3831704784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220099d30 con 0x562c1df78720 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 -- 172.21.15.16:0/3831704784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 msgr2=0x562c1df72350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df72350 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd22000dfd0 tx=0x7fd220099530 comp rx=0 tx=0).stop 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 -- 172.21.15.16:0/3831704784 shutdown_connections 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df72890 0x562c1df72cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 --2- 172.21.15.16:0/3831704784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df72350 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 -- 172.21.15.16:0/3831704784 >> 172.21.15.16:0/3831704784 conn(0x562c1ded0210 msgr2=0x562c1df6cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 -- 172.21.15.16:0/3831704784 shutdown_connections 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 -- 172.21.15.16:0/3831704784 wait complete. 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.553+0000 7fd24136fb80 1 -- start start 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 Processor -- start 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 0x562c1df7bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df7c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df7c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1de5e560 con 0x562c1df72890 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1de5e200 con 0x562c1df78720 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dd456a0 con 0x562c1df77e00 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd24136fb80 10 client.0 _ll_register_callbacks cb 0x562c1de67b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:51.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 0x562c1df7bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 0x562c1df7bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48186/0 (socket says 172.21.15.16:48186) 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 -- 172.21.15.16:0/771298865 learned_addr learned my addr 172.21.15.16:0/771298865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22e59c700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df7c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22f59e700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df7c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 -- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 msgr2=0x562c1df7c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df7c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 -- 172.21.15.16:0/771298865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 msgr2=0x562c1df7c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df7c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.554+0000 7fd22ed9d700 1 -- 172.21.15.16:0/771298865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2301a6690 con 0x562c1df72890 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 0x562c1df7bb30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd218015000 tx=0x7fd218009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0dd83245a26069a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd2167fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd2167fc700 1 -- 172.21.15.16:0/771298865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800fd40 con 0x562c1df72890 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c1de1b270 con 0x562c1df72890 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd2167fc700 1 -- 172.21.15.16:0/771298865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fd218024410 con 0x562c1df72890 2024-05-08T23:26:51.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd2167fc700 1 -- 172.21.15.16:0/771298865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd21800fea0 con 0x562c1df72890 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 10 client.17130 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562c1dcefb60 con 0x562c1df72890 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd2167fc700 1 -- 172.21.15.16:0/771298865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562c1de1b270 con 0x562c1df72890 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd2167fc700 1 -- 172.21.15.16:0/771298865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd218030580 con 0x562c1df72890 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 10 client.17130 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 20 client.17130 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1df8c0a0 0x562c1df8e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 4 client.17130 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.555+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c1df41810 con 0x562c1df8c0a0 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.556+0000 7fd22e59c700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1df8c0a0 0x562c1df8e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.556+0000 7fd2167fc700 10 client.17130 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:51.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.556+0000 7fd22e59c700 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1df8c0a0 0x562c1df8e4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b329c8322baa131 server_cookie=d875c088d3933120 in_seq=0 out_seq=0 2024-05-08T23:26:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd2167fc700 1 -- 172.21.15.16:0/771298865 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c1df41810 con 0x562c1df8c0a0 2024-05-08T23:26:51.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd2167fc700 10 client.17130 handle_command_reply: tid=0 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 client.17130 shutdown 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 msgr2=0x562c1df7bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 0x562c1df7bb30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd218015000 tx=0x7fd218009ca0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 shutdown_connections 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1df8c0a0 0x562c1df8e4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df7c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df72890 0x562c1df7bb30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd2167fc700 0 client.17130 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 --2- 172.21.15.16:0/771298865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df77e00 0x562c1df7c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 >> 172.21.15.16:0/771298865 conn(0x562c1ded0210 msgr2=0x562c1df6c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 shutdown_connections 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.557+0000 7fd24136fb80 1 -- 172.21.15.16:0/771298865 wait complete. 2024-05-08T23:26:51.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.558+0000 7fd24136fb80 20 client.17130 trim_cache size 0 max 16384 2024-05-08T23:26:51.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.571+0000 7fd24136fb80 1 Processor -- start 2024-05-08T23:26:51.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.571+0000 7fd24136fb80 1 -- start start 2024-05-08T23:26:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df87b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df77e00 0x562c1df6e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df77330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd24136fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1df726d0 con 0x562c1df78720 2024-05-08T23:26:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd24136fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1df93690 con 0x562c1df77e00 2024-05-08T23:26:51.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd24136fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1df7a040 con 0x562c1df72890 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df77330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df77330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48194/0 (socket says 172.21.15.16:48194) 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 -- 172.21.15.16:0/2038949612 learned_addr learned my addr 172.21.15.16:0/2038949612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22e59c700 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df77e00 0x562c1df6e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df87b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 -- 172.21.15.16:0/2038949612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 msgr2=0x562c1df87b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df87b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 -- 172.21.15.16:0/2038949612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df77e00 msgr2=0x562c1df6e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df77e00 0x562c1df6e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22f59e700 1 -- 172.21.15.16:0/2038949612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1de88560 con 0x562c1df78720 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.572+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df87b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.573+0000 7fd22f59e700 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df77330 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd228007850 tx=0x7fd2280091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cbc9ba73262e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.573+0000 7fd225ffb700 1 -- 172.21.15.16:0/2038949612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228017680 con 0x562c1df78720 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.573+0000 7fd225ffb700 1 -- 172.21.15.16:0/2038949612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd22800f040 con 0x562c1df78720 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.573+0000 7fd225ffb700 1 -- 172.21.15.16:0/2038949612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228015a70 con 0x562c1df78720 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.573+0000 7fd24136fb80 1 -- 172.21.15.16:0/2038949612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 msgr2=0x562c1df77330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.573+0000 7fd24136fb80 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df77330 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd228007850 tx=0x7fd2280091b0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- 172.21.15.16:0/2038949612 shutdown_connections 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df78720 0x562c1df77330 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df87b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 --2- 172.21.15.16:0/2038949612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df77e00 0x562c1df6e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- 172.21.15.16:0/2038949612 >> 172.21.15.16:0/2038949612 conn(0x562c1ded0210 msgr2=0x562c1df76c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- 172.21.15.16:0/2038949612 shutdown_connections 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- 172.21.15.16:0/2038949612 wait complete. 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- start start 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 Processor -- start 2024-05-08T23:26:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df80e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 0x562c1df813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df81950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dfe1730 con 0x562c1df77e00 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dfe18a0 con 0x562c1df78720 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1dfe1a60 con 0x562c1df72890 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd24136fb80 10 client.0 _ll_register_callbacks cb 0x562c1de67b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd22e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 0x562c1df813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd22e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 0x562c1df813d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48210/0 (socket says 172.21.15.16:48210) 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.574+0000 7fd22e59c700 1 -- 172.21.15.16:0/1460679600 learned_addr learned my addr 172.21.15.16:0/1460679600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df80e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22f59e700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df81950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22e59c700 1 -- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 msgr2=0x562c1df80e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22e59c700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df80e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22e59c700 1 -- 172.21.15.16:0/1460679600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 msgr2=0x562c1df81950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22e59c700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df81950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22e59c700 1 -- 172.21.15.16:0/1460679600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd228007500 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df80e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd22e59c700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 0x562c1df813d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd220005950 tx=0x7fd22009a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b036af19eb6bad75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd2167fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd2167fc700 1 -- 172.21.15.16:0/1460679600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220013050 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd2167fc700 1 -- 172.21.15.16:0/1460679600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fd2200131b0 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd22009d900 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd2167fc700 1 -- 172.21.15.16:0/1460679600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2200b2250 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd24136fb80 10 client.17136 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.575+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562c1dd47cd0 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd2167fc700 1 -- 172.21.15.16:0/1460679600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd22009d900 con 0x562c1df77e00 2024-05-08T23:26:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd2167fc700 1 -- 172.21.15.16:0/1460679600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd220099d30 con 0x562c1df77e00 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd24136fb80 10 client.17136 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd24136fb80 10 client.17136 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd24136fb80 20 client.17136 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd24136fb80 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1dfe2de0 0x562c1dfe5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd24136fb80 4 client.17136 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c1debaec0 con 0x562c1dfe2de0 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1dfe2de0 0x562c1dfe5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd2167fc700 10 client.17136 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.576+0000 7fd22ed9d700 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1dfe2de0 0x562c1dfe5200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b08052f8afa8443 server_cookie=e2408d696f095590 in_seq=0 out_seq=0 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2167fc700 1 -- 172.21.15.16:0/1460679600 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c1debaec0 con 0x562c1dfe2de0 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2167fc700 10 client.17136 handle_command_reply: tid=0 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 client.17136 shutdown 2024-05-08T23:26:51.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 msgr2=0x562c1df813d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 0x562c1df813d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd220005950 tx=0x7fd22009a310 comp rx=0 tx=0).stop 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 shutdown_connections 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x562c1dfe2de0 0x562c1dfe5200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562c1df78720 0x562c1df81950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562c1df72890 0x562c1df80e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 --2- 172.21.15.16:0/1460679600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c1df77e00 0x562c1df813d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2167fc700 0 client.17136 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 >> 172.21.15.16:0/1460679600 conn(0x562c1ded0210 msgr2=0x562c1df7c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 shutdown_connections 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 1 -- 172.21.15.16:0/1460679600 wait complete. 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd24136fb80 20 client.17136 trim_cache size 0 max 16384 2024-05-08T23:26:51.583 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:26:51.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd21c0a1290 msgr2=0x7fd21c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2351b5700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd21c0a1290 0x7fd21c0a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd21800b5c0 tx=0x7fd218005fb0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 msgr2=0x7fd230199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.577+0000 7fd2351b5700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd230199ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd23006a5d0 tx=0x7fd22000dc30 comp rx=0 tx=0).stop 2024-05-08T23:26:51.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 shutdown_connections 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd21c0a1290 0x7fd21c0a3750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23019de50 0x7fd23019e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd230101180 0x7fd230199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 --2- 172.21.15.16:0/2294675755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2301062a0 0x7fd230199ab0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 >> 172.21.15.16:0/2294675755 conn(0x7fd2300fc9e0 msgr2=0x7fd2300fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 shutdown_connections 2024-05-08T23:26:51.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.578+0000 7fd2351b5700 1 -- 172.21.15.16:0/2294675755 wait complete. 2024-05-08T23:26:51.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:26:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 -- 172.21.15.16:0/2757099562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7381062a0 msgr2=0x7fa73810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 --2- 172.21.15.16:0/2757099562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7381062a0 0x7fa73810de20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa73000b0a0 tx=0x7fa73000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 -- 172.21.15.16:0/2757099562 shutdown_connections 2024-05-08T23:26:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 --2- 172.21.15.16:0/2757099562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7381062a0 0x7fa73810de20 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 --2- 172.21.15.16:0/2757099562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa738101180 0x7fa738101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 --2- 172.21.15.16:0/2757099562 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa738101b30 0x7fa738105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 -- 172.21.15.16:0/2757099562 >> 172.21.15.16:0/2757099562 conn(0x7fa7380fc9e0 msgr2=0x7fa7380fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 -- 172.21.15.16:0/2757099562 shutdown_connections 2024-05-08T23:26:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.733+0000 7fa7401a2700 1 -- 172.21.15.16:0/2757099562 wait complete. 2024-05-08T23:26:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 Processor -- start 2024-05-08T23:26:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 -- start start 2024-05-08T23:26:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa738101180 0x7fa7380732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7381062a0 0x7fa7380737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 0x7fa7381a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381a5c20 con 0x7fa738073d20 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381a5d90 con 0x7fa7381062a0 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa7401a2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7381a5f50 con 0x7fa738101180 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa73e73f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 0x7fa7381a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa73e73f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 0x7fa7381a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48228/0 (socket says 172.21.15.16:48228) 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa73df3e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa738101180 0x7fa7380732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa73e73f700 1 -- 172.21.15.16:0/1547129576 learned_addr learned my addr 172.21.15.16:0/1547129576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.734+0000 7fa73d73d700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7381062a0 0x7fa7380737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.735+0000 7fa73e73f700 1 -- 172.21.15.16:0/1547129576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa738101180 msgr2=0x7fa7380732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.735+0000 7fa73e73f700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa738101180 0x7fa7380732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.735+0000 7fa73e73f700 1 -- 172.21.15.16:0/1547129576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7381062a0 msgr2=0x7fa7380737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.735+0000 7fa73e73f700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7381062a0 0x7fa7380737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.735+0000 7fa73e73f700 1 -- 172.21.15.16:0/1547129576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009d00 con 0x7fa738073d20 2024-05-08T23:26:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.735+0000 7fa73e73f700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 0x7fa7381a55e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa730006130 tx=0x7fa730009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.737+0000 7fa72effd700 1 -- 172.21.15.16:0/1547129576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000e040 con 0x7fa738073d20 2024-05-08T23:26:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.737+0000 7fa72effd700 1 -- 172.21.15.16:0/1547129576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa73001b450 con 0x7fa738073d20 2024-05-08T23:26:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.737+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7381a61d0 con 0x7fa738073d20 2024-05-08T23:26:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.737+0000 7fa72effd700 1 -- 172.21.15.16:0/1547129576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7300052a0 con 0x7fa738073d20 2024-05-08T23:26:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.737+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7381a66a0 con 0x7fa738073d20 2024-05-08T23:26:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.739+0000 7fa72effd700 1 -- 172.21.15.16:0/1547129576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa73001b5c0 con 0x7fa738073d20 2024-05-08T23:26:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.740+0000 7fa72effd700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa7240a1310 0x7fa7240a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.740+0000 7fa72effd700 1 -- 172.21.15.16:0/1547129576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fa7300ce230 con 0x7fa738073d20 2024-05-08T23:26:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.740+0000 7fa73df3e700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa7240a1310 0x7fa7240a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.741+0000 7fa73df3e700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa7240a1310 0x7fa7240a37d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa734009c60 tx=0x7fa734009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.741+0000 7fa74c35cb80 1 Processor -- start 2024-05-08T23:26:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 -- start start 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590910a150 con 0x5559091b8e00 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555909107b10 con 0x5559091b9720 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555909103400 con 0x5559091b3890 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091b91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48240/0 (socket says 172.21.15.16:48240) 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 -- 172.21.15.16:0/3213713902 learned_addr learned my addr 172.21.15.16:0/3213713902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73d73d700 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73e73f700 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 -- 172.21.15.16:0/3213713902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 msgr2=0x5559091b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 -- 172.21.15.16:0/3213713902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 msgr2=0x5559091b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73df3e700 1 -- 172.21.15.16:0/3213713902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555908f843b0 con 0x5559091b8e00 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.742+0000 7fa73e73f700 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.743+0000 7fa73df3e700 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091b91e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa73400e4e0 tx=0x7fa734014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d619ace106fe1fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.743+0000 7fa723fff700 1 -- 172.21.15.16:0/3213713902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734026430 con 0x5559091b8e00 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.743+0000 7fa723fff700 1 -- 172.21.15.16:0/3213713902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa734021070 con 0x5559091b8e00 2024-05-08T23:26:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.743+0000 7fa723fff700 1 -- 172.21.15.16:0/3213713902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73401d670 con 0x5559091b8e00 2024-05-08T23:26:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3213713902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 msgr2=0x5559091b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091b91e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa73400e4e0 tx=0x7fa734014000 comp rx=0 tx=0).stop 2024-05-08T23:26:51.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3213713902 shutdown_connections 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091b91e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3213713902 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3213713902 >> 172.21.15.16:0/3213713902 conn(0x555909111210 msgr2=0x5559091addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3213713902 shutdown_connections 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3213713902 wait complete. 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 -- start start 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.744+0000 7fa74c35cb80 1 Processor -- start 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b8e00 0x5559091bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x5559091bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590909f560 con 0x5559091b9720 2024-05-08T23:26:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590909f200 con 0x5559091b8e00 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555908f866a0 con 0x5559091b3890 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa74c35cb80 10 client.0 _ll_register_callbacks cb 0x5559090a8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x5559091bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x5559091bd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48250/0 (socket says 172.21.15.16:48250) 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 -- 172.21.15.16:0/2783361238 learned_addr learned my addr 172.21.15.16:0/2783361238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73df3e700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73d73d700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b8e00 0x5559091bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 -- 172.21.15.16:0/2783361238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 msgr2=0x5559091bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091bcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 -- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b8e00 msgr2=0x5559091bd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b8e00 0x5559091bd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 -- 172.21.15.16:0/2783361238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa734017070 con 0x5559091b9720 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa73e73f700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x5559091bd610 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa730097530 tx=0x7fa730019f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330877a995af0b24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.745+0000 7fa7217fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa7217fa700 1 -- 172.21.15.16:0/2783361238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73001bac0 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa7217fa700 1 -- 172.21.15.16:0/2783361238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fa73009ee60 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55590905c270 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa7217fa700 1 -- 172.21.15.16:0/2783361238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7300a3420 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 10 client.17148 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555908f30b60 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa7217fa700 1 -- 172.21.15.16:0/2783361238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55590905c270 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa7217fa700 1 -- 172.21.15.16:0/2783361238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa73009a520 con 0x5559091b9720 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 10 client.17148 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 20 client.17148 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5559091ccf60 0x5559091cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 4 client.17148 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555909182810 con 0x5559091ccf60 2024-05-08T23:26:51.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.746+0000 7fa73df3e700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5559091ccf60 0x5559091cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.747+0000 7fa7217fa700 10 client.17148 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.747+0000 7fa73df3e700 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5559091ccf60 0x5559091cf380 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7971fa15eded512d server_cookie=5c498002c9c1ef21 in_seq=0 out_seq=0 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa7217fa700 1 -- 172.21.15.16:0/2783361238 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555909182810 con 0x5559091ccf60 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa7217fa700 10 client.17148 handle_command_reply: tid=0 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 client.17148 shutdown 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 msgr2=0x5559091bd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x5559091bd610 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa730097530 tx=0x7fa730019f60 comp rx=0 tx=0).stop 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 shutdown_connections 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x5559091bd610 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5559091ccf60 0x5559091cf380 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/2783361238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b8e00 0x5559091bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa7217fa700 0 client.17148 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:51.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 >> 172.21.15.16:0/2783361238 conn(0x555909111210 msgr2=0x5559091ad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 shutdown_connections 2024-05-08T23:26:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 1 -- 172.21.15.16:0/2783361238 wait complete. 2024-05-08T23:26:51.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.748+0000 7fa74c35cb80 20 client.17148 trim_cache size 0 max 16384 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 Processor -- start 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 -- start start 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590905c270 con 0x5559091b8e00 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559091aece0 con 0x5559091b9720 2024-05-08T23:26:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.762+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559091be1e0 con 0x5559091b3890 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091bb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48264/0 (socket says 172.21.15.16:48264) 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 -- 172.21.15.16:0/3104578571 learned_addr learned my addr 172.21.15.16:0/3104578571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73df3e700 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73e73f700 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 -- 172.21.15.16:0/3104578571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 msgr2=0x5559091add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 -- 172.21.15.16:0/3104578571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 msgr2=0x5559091bb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091bb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 -- 172.21.15.16:0/3104578571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559090c9560 con 0x5559091b8e00 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa73d73d700 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091bb460 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa738074d50 tx=0x7fa728009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ff3edb1af5a539 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa72cff9700 1 -- 172.21.15.16:0/3104578571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7280045b0 con 0x5559091b8e00 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa72cff9700 1 -- 172.21.15.16:0/3104578571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa728017040 con 0x5559091b8e00 2024-05-08T23:26:51.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.763+0000 7fa72cff9700 1 -- 172.21.15.16:0/3104578571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728010c60 con 0x5559091b8e00 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3104578571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 msgr2=0x5559091bb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091bb460 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa738074d50 tx=0x7fa728009b10 comp rx=0 tx=0).stop 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3104578571 shutdown_connections 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b9720 0x5559091bb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b3890 0x5559091add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/3104578571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b8e00 0x5559091bb460 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3104578571 >> 172.21.15.16:0/3104578571 conn(0x555909111210 msgr2=0x5559091aa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3104578571 shutdown_connections 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 -- 172.21.15.16:0/3104578571 wait complete. 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 -- start start 2024-05-08T23:26:51.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.764+0000 7fa74c35cb80 1 Processor -- start 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b3890 0x5559091c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b8e00 0x5559091c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x555909222460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559092229a0 con 0x5559091b9720 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555909222ae0 con 0x5559091b3890 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555909222ca0 con 0x5559091b8e00 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa74c35cb80 10 client.0 _ll_register_callbacks cb 0x5559090a8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73e73f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x555909222460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73d73d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b8e00 0x5559091c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73e73f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x555909222460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48274/0 (socket says 172.21.15.16:48274) 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73e73f700 1 -- 172.21.15.16:0/1855850258 learned_addr learned my addr 172.21.15.16:0/1855850258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73df3e700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b3890 0x5559091c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73d73d700 1 -- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b3890 msgr2=0x5559091c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73d73d700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b3890 0x5559091c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73d73d700 1 -- 172.21.15.16:0/1855850258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 msgr2=0x555909222460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73d73d700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x555909222460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73d73d700 1 -- 172.21.15.16:0/1855850258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7280041d0 con 0x5559091b8e00 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.765+0000 7fa73e73f700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x555909222460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa73d73d700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b8e00 0x5559091c2940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa728005920 tx=0x7fa728010b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a670159d778d558 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa7217fa700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa7217fa700 1 -- 172.21.15.16:0/1855850258 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728005250 con 0x5559091b8e00 2024-05-08T23:26:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa734021070 con 0x5559091b8e00 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa7217fa700 1 -- 172.21.15.16:0/1855850258 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fa7280053b0 con 0x5559091b8e00 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa7217fa700 1 -- 172.21.15.16:0/1855850258 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7280055b0 con 0x5559091b8e00 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 10 client.26848 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555908f88cd0 con 0x5559091b8e00 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa7217fa700 1 -- 172.21.15.16:0/1855850258 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa734021070 con 0x5559091b8e00 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa7217fa700 1 -- 172.21.15.16:0/1855850258 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa728003980 con 0x5559091b8e00 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 10 client.26848 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 10 client.26848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 20 client.26848 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555909224410 0x555909226830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 4 client.26848 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.766+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559090fbec0 con 0x555909224410 2024-05-08T23:26:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa73df3e700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555909224410 0x555909226830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa73df3e700 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555909224410 0x555909226830 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287fc32995cd5356 server_cookie=c59f6def355a6f49 in_seq=0 out_seq=0 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa7217fa700 10 client.26848 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa7217fa700 1 -- 172.21.15.16:0/1855850258 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559090fbec0 con 0x555909224410 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa7217fa700 10 client.26848 handle_command_reply: tid=0 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa74c35cb80 1 client.26848 shutdown 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b8e00 msgr2=0x5559091c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.767+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b8e00 0x5559091c2940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa728005920 tx=0x7fa728010b50 comp rx=0 tx=0).stop 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 shutdown_connections 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559091b9720 0x555909222460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555909224410 0x555909226830 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5559091b3890 0x5559091c2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7217fa700 0 client.26848 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 --2- 172.21.15.16:0/1855850258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559091b8e00 0x5559091c2940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 >> 172.21.15.16:0/1855850258 conn(0x555909111210 msgr2=0x5559091bb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 shutdown_connections 2024-05-08T23:26:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 1 -- 172.21.15.16:0/1855850258 wait complete. 2024-05-08T23:26:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa74c35cb80 20 client.26848 trim_cache size 0 max 16384 2024-05-08T23:26:51.774 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa7240a1310 msgr2=0x7fa7240a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa7240a1310 0x7fa7240a37d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa734009c60 tx=0x7fa734009260 comp rx=0 tx=0).stop 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 msgr2=0x7fa7381a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 0x7fa7381a55e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa730006130 tx=0x7fa730009500 comp rx=0 tx=0).stop 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 shutdown_connections 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fa7240a1310 0x7fa7240a37d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738073d20 0x7fa7381a55e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa738101180 0x7fa7380732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 --2- 172.21.15.16:0/1547129576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7381062a0 0x7fa7380737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.768+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 >> 172.21.15.16:0/1547129576 conn(0x7fa7380fc9e0 msgr2=0x7fa7380fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.769+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 shutdown_connections 2024-05-08T23:26:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:51.769+0000 7fa7401a2700 1 -- 172.21.15.16:0/1547129576 wait complete. 2024-05-08T23:26:51.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:51 smithi016 ceph-mon[42541]: pgmap v223: 129 pgs: 129 active+clean; 9.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:51.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:51 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1077792516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:26:52.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:51 smithi189 ceph-mon[46919]: pgmap v223: 129 pgs: 129 active+clean; 9.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:52.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:51 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1077792516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:26:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:51 smithi155 ceph-mon[47105]: pgmap v223: 129 pgs: 129 active+clean; 9.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-05-08T23:26:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:51 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1077792516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:26:52.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:26:52.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:26:52.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 -- 172.21.15.16:0/2024711665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a60068c90 msgr2=0x7f1a60105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 --2- 172.21.15.16:0/2024711665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a60068c90 0x7f1a60105b60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1a480099c0 tx=0x7f1a48009cd0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 -- 172.21.15.16:0/2024711665 shutdown_connections 2024-05-08T23:26:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 --2- 172.21.15.16:0/2024711665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a60106290 0x7f1a6010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 --2- 172.21.15.16:0/2024711665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a600682e0 0x7f1a600686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 --2- 172.21.15.16:0/2024711665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a60068c90 0x7f1a60105b60 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 -- 172.21.15.16:0/2024711665 >> 172.21.15.16:0/2024711665 conn(0x7f1a60076f40 msgr2=0x7f1a60077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 -- 172.21.15.16:0/2024711665 shutdown_connections 2024-05-08T23:26:52.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.933+0000 7f1a65204700 1 -- 172.21.15.16:0/2024711665 wait complete. 2024-05-08T23:26:52.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.934+0000 7f1a65204700 1 Processor -- start 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.934+0000 7f1a65204700 1 -- start start 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 0x7f1a60197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a60106290 0x7f1a601977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6019bb40 0x7f1a6019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6019c5c0 con 0x7f1a600682e0 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6019c730 con 0x7f1a6019bb40 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6019c8f0 con 0x7f1a60106290 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 0x7f1a60197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 0x7f1a60197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48298/0 (socket says 172.21.15.16:48298) 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/919797234 learned_addr learned my addr 172.21.15.16:0/919797234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5659c700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a60106290 0x7f1a601977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6019bb40 0x7f1a6019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/919797234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a60106290 msgr2=0x7f1a601977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a60106290 0x7f1a601977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/919797234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6019bb40 msgr2=0x7f1a6019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6019bb40 0x7f1a6019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/919797234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480095f0 con 0x7f1a600682e0 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6019bb40 0x7f1a6019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 0x7f1a60197260 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c9b0 tx=0x7f1a5000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5cd99700 1 -- 172.21.15.16:0/919797234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007b00 con 0x7f1a600682e0 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5cd99700 1 -- 172.21.15.16:0/919797234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a50007c60 con 0x7f1a600682e0 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a601a3e50 con 0x7f1a600682e0 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.935+0000 7f1a5cd99700 1 -- 172.21.15.16:0/919797234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50018660 con 0x7f1a600682e0 2024-05-08T23:26:52.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.936+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a601a4270 con 0x7f1a600682e0 2024-05-08T23:26:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.938+0000 7f1a5cd99700 1 -- 172.21.15.16:0/919797234 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1a5000f450 con 0x7f1a600682e0 2024-05-08T23:26:52.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.938+0000 7f1a5cd99700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1a4c0a1290 0x7f1a4c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.939+0000 7f1a5cd99700 1 -- 172.21.15.16:0/919797234 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1a500cd100 con 0x7f1a600682e0 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.939+0000 7f1a5659c700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1a4c0a1290 0x7f1a4c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.939+0000 7f1a5659c700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1a4c0a1290 0x7f1a4c0a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b580 tx=0x7f1a48005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.939+0000 7f1a713beb80 1 Processor -- start 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.939+0000 7f1a713beb80 1 -- start start 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a713beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79de01d0 con 0x559c79e8ee80 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a713beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79dddb90 con 0x559c79e89910 2024-05-08T23:26:52.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a713beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79dd9480 con 0x559c79e8f7a0 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e8f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48300/0 (socket says 172.21.15.16:48300) 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/2668844069 learned_addr learned my addr 172.21.15.16:0/2668844069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5659c700 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/2668844069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 msgr2=0x559c79e893d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e893d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/2668844069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 msgr2=0x559c79e89d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e89d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5ed9d700 1 -- 172.21.15.16:0/2668844069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c79c5a430 con 0x559c79e8ee80 2024-05-08T23:26:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.940+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.941+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e8f260 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1a50096880 tx=0x7f1a50099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7f1ba887e90d57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.941+0000 7f1a5559a700 1 -- 172.21.15.16:0/2668844069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50099e10 con 0x559c79e8ee80 2024-05-08T23:26:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.941+0000 7f1a5559a700 1 -- 172.21.15.16:0/2668844069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a5009dc40 con 0x559c79e8ee80 2024-05-08T23:26:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.941+0000 7f1a5559a700 1 -- 172.21.15.16:0/2668844069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5009a070 con 0x559c79e8ee80 2024-05-08T23:26:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.941+0000 7f1a713beb80 1 -- 172.21.15.16:0/2668844069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 msgr2=0x559c79e8f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.941+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e8f260 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1a50096880 tx=0x7f1a50099840 comp rx=0 tx=0).stop 2024-05-08T23:26:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- 172.21.15.16:0/2668844069 shutdown_connections 2024-05-08T23:26:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e8f260 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2668844069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e893d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- 172.21.15.16:0/2668844069 >> 172.21.15.16:0/2668844069 conn(0x559c79de7290 msgr2=0x559c79e83e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- 172.21.15.16:0/2668844069 shutdown_connections 2024-05-08T23:26:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- 172.21.15.16:0/2668844069 wait complete. 2024-05-08T23:26:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- start start 2024-05-08T23:26:52.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 Processor -- start 2024-05-08T23:26:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e930f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79d755e0 con 0x559c79e8ee80 2024-05-08T23:26:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79d75280 con 0x559c79e89910 2024-05-08T23:26:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79c5c720 con 0x559c79e8f7a0 2024-05-08T23:26:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.942+0000 7f1a713beb80 10 client.0 _ll_register_callbacks cb 0x559c79d7ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e930f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e930f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48310/0 (socket says 172.21.15.16:48310) 2024-05-08T23:26:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 -- 172.21.15.16:0/1337081741 learned_addr learned my addr 172.21.15.16:0/1337081741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e92bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 -- 172.21.15.16:0/1337081741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 msgr2=0x559c79e93630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e93630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 -- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 msgr2=0x559c79e92bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e92bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 -- 172.21.15.16:0/1337081741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a601a4270 con 0x559c79e8ee80 2024-05-08T23:26:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e92bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:26:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a5659c700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e930f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1a6006a6d0 tx=0x7f1a48009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec2e0061cf97d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a467fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:52.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a467fc700 1 -- 172.21.15.16:0/1337081741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48003b10 con 0x559c79e8ee80 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a467fc700 1 -- 172.21.15.16:0/1337081741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1a48003c70 con 0x559c79e8ee80 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.943+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c79d322f0 con 0x559c79e8ee80 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a467fc700 1 -- 172.21.15.16:0/1337081741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a48003e30 con 0x559c79e8ee80 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 10 client.17166 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c79c06be0 con 0x559c79e8ee80 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a467fc700 1 -- 172.21.15.16:0/1337081741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c79d322f0 con 0x559c79e8ee80 2024-05-08T23:26:52.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a467fc700 1 -- 172.21.15.16:0/1337081741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1a4802f4c0 con 0x559c79e8ee80 2024-05-08T23:26:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 10 client.17166 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 20 client.17166 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79ea31a0 0x559c79ea55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 4 client.17166 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c79e58890 con 0x559c79ea31a0 2024-05-08T23:26:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.944+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79ea31a0 0x559c79ea55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.945+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79ea31a0 0x559c79ea55c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68305eccdeac5e9 server_cookie=d0c34b11c7e14fd5 in_seq=0 out_seq=0 2024-05-08T23:26:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.945+0000 7f1a467fc700 10 client.17166 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:52.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.945+0000 7f1a467fc700 1 -- 172.21.15.16:0/1337081741 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c79e58890 con 0x559c79ea31a0 2024-05-08T23:26:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.945+0000 7f1a467fc700 10 client.17166 handle_command_reply: tid=0 2024-05-08T23:26:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.945+0000 7f1a713beb80 1 client.17166 shutdown 2024-05-08T23:26:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 msgr2=0x559c79e930f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e930f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1a6006a6d0 tx=0x7f1a48009cd0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 shutdown_connections 2024-05-08T23:26:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8f7a0 0x559c79e93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79ea31a0 0x559c79ea55c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e92bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 --2- 172.21.15.16:0/1337081741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8ee80 0x559c79e930f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a467fc700 0 client.17166 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 >> 172.21.15.16:0/1337081741 conn(0x559c79de7290 msgr2=0x559c79e83620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 shutdown_connections 2024-05-08T23:26:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 1 -- 172.21.15.16:0/1337081741 wait complete. 2024-05-08T23:26:52.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.946+0000 7f1a713beb80 20 client.17166 trim_cache size 0 max 16384 2024-05-08T23:26:52.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.964+0000 7f1a713beb80 1 Processor -- start 2024-05-08T23:26:52.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.964+0000 7f1a713beb80 1 -- start start 2024-05-08T23:26:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8ee80 0x559c79e9a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e83da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a713beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79e80e40 con 0x559c79e8f7a0 2024-05-08T23:26:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a713beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79e9fce0 con 0x559c79e89910 2024-05-08T23:26:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a713beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79e8a2b0 con 0x559c79e8ee80 2024-05-08T23:26:52.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e83da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e83da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48320/0 (socket says 172.21.15.16:48320) 2024-05-08T23:26:52.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 -- 172.21.15.16:0/4099445505 learned_addr learned my addr 172.21.15.16:0/4099445505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:52.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5659c700 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8ee80 0x559c79e9a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 -- 172.21.15.16:0/4099445505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8ee80 msgr2=0x559c79e9a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8ee80 0x559c79e9a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 -- 172.21.15.16:0/4099445505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 msgr2=0x559c79e9eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e9eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 -- 172.21.15.16:0/4099445505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c79e92710 con 0x559c79e8f7a0 2024-05-08T23:26:52.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.965+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e83da0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1a58008700 tx=0x7f1a5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2a831b3b637391 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a55d9b700 1 -- 172.21.15.16:0/4099445505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a580075f0 con 0x559c79e8f7a0 2024-05-08T23:26:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a55d9b700 1 -- 172.21.15.16:0/4099445505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a58007be0 con 0x559c79e8f7a0 2024-05-08T23:26:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a55d9b700 1 -- 172.21.15.16:0/4099445505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58017c10 con 0x559c79e8f7a0 2024-05-08T23:26:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 -- 172.21.15.16:0/4099445505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 msgr2=0x559c79e83da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e83da0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1a58008700 tx=0x7f1a5800b040 comp rx=0 tx=0).stop 2024-05-08T23:26:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 -- 172.21.15.16:0/4099445505 shutdown_connections 2024-05-08T23:26:52.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e83da0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e89910 0x559c79e9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 --2- 172.21.15.16:0/4099445505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e8ee80 0x559c79e9a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 -- 172.21.15.16:0/4099445505 >> 172.21.15.16:0/4099445505 conn(0x559c79de7290 msgr2=0x559c79e91480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 -- 172.21.15.16:0/4099445505 shutdown_connections 2024-05-08T23:26:52.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.966+0000 7f1a713beb80 1 -- 172.21.15.16:0/4099445505 wait complete. 2024-05-08T23:26:52.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 -- start start 2024-05-08T23:26:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 Processor -- start 2024-05-08T23:26:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e89910 0x559c79e8ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e8ee80 0x559c79e8b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e8b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79efd690 con 0x559c79e8f7a0 2024-05-08T23:26:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79efd7d0 con 0x559c79e8ee80 2024-05-08T23:26:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c79efd940 con 0x559c79e89910 2024-05-08T23:26:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a713beb80 10 client.0 _ll_register_callbacks cb 0x559c79d7ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:52.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a5f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e8b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a5f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e8b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48324/0 (socket says 172.21.15.16:48324) 2024-05-08T23:26:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a5f59e700 1 -- 172.21.15.16:0/2346928550 learned_addr learned my addr 172.21.15.16:0/2346928550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e89910 0x559c79e8ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.967+0000 7f1a5659c700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e8ee80 0x559c79e8b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5f59e700 1 -- 172.21.15.16:0/2346928550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e89910 msgr2=0x559c79e8ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e89910 0x559c79e8ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5f59e700 1 -- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e8ee80 msgr2=0x559c79e8b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e8ee80 0x559c79e8b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5f59e700 1 -- 172.21.15.16:0/2346928550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a58013040 con 0x559c79e8f7a0 2024-05-08T23:26:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5659c700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e8ee80 0x559c79e8b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a5f59e700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e8b5a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1a58007f80 tx=0x7f1a58017420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb807436eeb6b29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a467fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:52.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a467fc700 1 -- 172.21.15.16:0/2346928550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800e040 con 0x559c79e8f7a0 2024-05-08T23:26:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a467fc700 1 -- 172.21.15.16:0/2346928550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1a580075f0 con 0x559c79e8f7a0 2024-05-08T23:26:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a5009dc40 con 0x559c79e8f7a0 2024-05-08T23:26:52.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a467fc700 1 -- 172.21.15.16:0/2346928550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a58004ea0 con 0x559c79e8f7a0 2024-05-08T23:26:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a713beb80 10 client.17172 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c79d9f5e0 con 0x559c79e8f7a0 2024-05-08T23:26:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.968+0000 7f1a467fc700 1 -- 172.21.15.16:0/2346928550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a5009dc40 con 0x559c79e8f7a0 2024-05-08T23:26:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a467fc700 1 -- 172.21.15.16:0/2346928550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1a580050d0 con 0x559c79e8f7a0 2024-05-08T23:26:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a713beb80 10 client.17172 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a713beb80 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a713beb80 20 client.17172 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79eff120 0x559c79f01540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a713beb80 4 client.17172 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:26:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559c79dd1f40 con 0x559c79eff120 2024-05-08T23:26:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79eff120 0x559c79f01540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a467fc700 10 client.17172 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.969+0000 7f1a5ed9d700 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79eff120 0x559c79f01540 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91084da92c264e29 server_cookie=572d23ec4c6fc7e2 in_seq=0 out_seq=0 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.970+0000 7f1a467fc700 1 -- 172.21.15.16:0/2346928550 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559c79dd1f40 con 0x559c79eff120 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.970+0000 7f1a467fc700 10 client.17172 handle_command_reply: tid=0 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.970+0000 7f1a713beb80 1 client.17172 shutdown 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 msgr2=0x559c79e8b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e8b5a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1a58007f80 tx=0x7f1a58017420 comp rx=0 tx=0).stop 2024-05-08T23:26:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 shutdown_connections 2024-05-08T23:26:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c79e8f7a0 0x559c79e8b5a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c79e89910 0x559c79e8ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559c79eff120 0x559c79f01540 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 --2- 172.21.15.16:0/2346928550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559c79e8ee80 0x559c79e8b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a467fc700 0 client.17172 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 >> 172.21.15.16:0/2346928550 conn(0x559c79de7290 msgr2=0x559c79e8dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 shutdown_connections 2024-05-08T23:26:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 1 -- 172.21.15.16:0/2346928550 wait complete. 2024-05-08T23:26:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.971+0000 7f1a713beb80 20 client.17172 trim_cache size 0 max 16384 2024-05-08T23:26:52.995 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:26:52.995 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:26:52.995 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "bd6efb39-0a11-4d5c-bbce-882a84db7c5b", 2024-05-08T23:26:52.995 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:26:52.996 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:26:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1a4c0a1290 msgr2=0x7f1a4c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1a4c0a1290 0x7f1a4c0a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800b580 tx=0x7f1a48005fb0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 msgr2=0x7f1a60197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 0x7f1a60197260 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c9b0 tx=0x7f1a5000cd70 comp rx=0 tx=0).stop 2024-05-08T23:26:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 shutdown_connections 2024-05-08T23:26:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1a4c0a1290 0x7f1a4c0a3750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6019bb40 0x7f1a6019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a600682e0 0x7f1a60197260 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 --2- 172.21.15.16:0/919797234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a60106290 0x7f1a601977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 >> 172.21.15.16:0/919797234 conn(0x7f1a60076f40 msgr2=0x7f1a600fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 shutdown_connections 2024-05-08T23:26:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:52.972+0000 7f1a65204700 1 -- 172.21.15.16:0/919797234 wait complete. 2024-05-08T23:26:53.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:26:53.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 -- 172.21.15.16:0/1248548131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 msgr2=0x7f03fc10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 --2- 172.21.15.16:0/1248548131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc10ccc0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f03f400b0d0 tx=0x7f03f400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 -- 172.21.15.16:0/1248548131 shutdown_connections 2024-05-08T23:26:53.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 --2- 172.21.15.16:0/1248548131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc10ccc0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 --2- 172.21.15.16:0/1248548131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 0x7f03fc106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 --2- 172.21.15.16:0/1248548131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03fc1000f0 0x7f03fc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 -- 172.21.15.16:0/1248548131 >> 172.21.15.16:0/1248548131 conn(0x7f03fc076ba0 msgr2=0x7f03fc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 -- 172.21.15.16:0/1248548131 shutdown_connections 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 -- 172.21.15.16:0/1248548131 wait complete. 2024-05-08T23:26:53.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.141+0000 7f04027ef700 1 Processor -- start 2024-05-08T23:26:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 -- start start 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03fc1000f0 0x7f03fc101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 0x7f03fc102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1a56e0 con 0x7f03fc100ab0 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1a5850 con 0x7f03fc1000f0 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f04027ef700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1a5a10 con 0x7f03fc105f90 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48332/0 (socket says 172.21.15.16:48332) 2024-05-08T23:26:53.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 -- 172.21.15.16:0/1001117624 learned_addr learned my addr 172.21.15.16:0/1001117624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fbfff700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03fc1000f0 0x7f03fc101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f0400d8c700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 0x7f03fc102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 -- 172.21.15.16:0/1001117624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 msgr2=0x7f03fc102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 0x7f03fc102990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 -- 172.21.15.16:0/1001117624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03fc1000f0 msgr2=0x7f03fc101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.142+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03fc1000f0 0x7f03fc101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f03fb7fe700 1 -- 172.21.15.16:0/1001117624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009d00 con 0x7f03fc100ab0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f0400d8c700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 0x7f03fc102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc102450 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f03f000d950 tx=0x7f03f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f03f97fa700 1 -- 172.21.15.16:0/1001117624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0009940 con 0x7f03fc100ab0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f03f97fa700 1 -- 172.21.15.16:0/1001117624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03f0010460 con 0x7f03fc100ab0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc1a5cf0 con 0x7f03fc100ab0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f03f97fa700 1 -- 172.21.15.16:0/1001117624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000f620 con 0x7f03fc100ab0 2024-05-08T23:26:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.143+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc1a61f0 con 0x7f03fc100ab0 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.145+0000 7f03f97fa700 1 -- 172.21.15.16:0/1001117624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f03f00105d0 con 0x7f03fc100ab0 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.146+0000 7f03f97fa700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f03e40a1310 0x7f03e40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.146+0000 7f03f97fa700 1 -- 172.21.15.16:0/1001117624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f03f00ccec0 con 0x7f03fc100ab0 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.146+0000 7f03fbfff700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f03e40a1310 0x7f03e40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f03fbfff700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f03e40a1310 0x7f03e40a37d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03fc067bb0 tx=0x7f03ec00b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 Processor -- start 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 -- start start 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7225e00 0x560cc72261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7220350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc7220cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7177150 con 0x560cc7220890 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7174b10 con 0x560cc7225e00 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7170400 con 0x560cc7226720 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f0400d8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc7220cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f0400d8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc7220cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48340/0 (socket says 172.21.15.16:48340) 2024-05-08T23:26:53.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f0400d8c700 1 -- 172.21.15.16:0/2474354259 learned_addr learned my addr 172.21.15.16:0/2474354259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f03fbfff700 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7225e00 0x560cc72261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.147+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7220350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f0400d8c700 1 -- 172.21.15.16:0/2474354259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 msgr2=0x560cc7220350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f0400d8c700 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7220350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f0400d8c700 1 -- 172.21.15.16:0/2474354259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7225e00 msgr2=0x560cc72261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f0400d8c700 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7225e00 0x560cc72261e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f0400d8c700 1 -- 172.21.15.16:0/2474354259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc6ff13b0 con 0x560cc7220890 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f0400d8c700 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc7220cd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009cb0 tx=0x7f03f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1751c1e194ab11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f03e27fc700 1 -- 172.21.15.16:0/2474354259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400e070 con 0x560cc7220890 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f03e27fc700 1 -- 172.21.15.16:0/2474354259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03f40090d0 con 0x560cc7220890 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.148+0000 7f03e27fc700 1 -- 172.21.15.16:0/2474354259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f40074e0 con 0x560cc7220890 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 -- 172.21.15.16:0/2474354259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 msgr2=0x560cc7220cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc7220cd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009cb0 tx=0x7f03f400cd70 comp rx=0 tx=0).stop 2024-05-08T23:26:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 -- 172.21.15.16:0/2474354259 shutdown_connections 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc7220cd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7225e00 0x560cc72261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/2474354259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7220350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 -- 172.21.15.16:0/2474354259 >> 172.21.15.16:0/2474354259 conn(0x560cc717e210 msgr2=0x560cc721adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 -- 172.21.15.16:0/2474354259 shutdown_connections 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 -- 172.21.15.16:0/2474354259 wait complete. 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.149+0000 7f040e9a9b80 1 -- start start 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 Processor -- start 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc722c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7229a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 0x560cc722a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc710c560 con 0x560cc7226720 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc710c200 con 0x560cc7220890 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6ff36a0 con 0x560cc7225e00 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f040e9a9b80 10 client.0 _ll_register_callbacks cb 0x560cc7115b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f0400d8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 0x560cc722a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f0400d8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 0x560cc722a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48356/0 (socket says 172.21.15.16:48356) 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f0400d8c700 1 -- 172.21.15.16:0/3832590772 learned_addr learned my addr 172.21.15.16:0/3832590772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc722c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.150+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7229a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f0400d8c700 1 -- 172.21.15.16:0/3832590772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 msgr2=0x560cc7229a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f0400d8c700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7229a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f0400d8c700 1 -- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 msgr2=0x560cc722c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f0400d8c700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc722c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f0400d8c700 1 -- 172.21.15.16:0/3832590772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc10d550 con 0x560cc7226720 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f0400d8c700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 0x560cc722a0b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009a20 tx=0x7f03f4007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a025b71d97a7358 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f03db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f03db7fe700 1 -- 172.21.15.16:0/3832590772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400e040 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f03db7fe700 1 -- 172.21.15.16:0/3832590772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f03f40074a0 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc70c9270 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f03db7fe700 1 -- 172.21.15.16:0/3832590772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f4004e80 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f040e9a9b80 10 client.17184 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560cc6f9db60 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.151+0000 7f03db7fe700 1 -- 172.21.15.16:0/3832590772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560cc70c9270 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f03db7fe700 1 -- 172.21.15.16:0/3832590772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03f400b100 con 0x560cc7226720 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f040e9a9b80 10 client.17184 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f040e9a9b80 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f040e9a9b80 20 client.17184 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7239e40 0x560cc723c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f040e9a9b80 4 client.17184 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cc71ef810 con 0x560cc7239e40 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7239e40 0x560cc723c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7239e40 0x560cc723c230 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5809257423403a server_cookie=1ac4265e9c561192 in_seq=0 out_seq=0 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.152+0000 7f03db7fe700 10 client.17184 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f03db7fe700 1 -- 172.21.15.16:0/3832590772 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cc71ef810 con 0x560cc7239e40 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f03db7fe700 10 client.17184 handle_command_reply: tid=0 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 client.17184 shutdown 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 msgr2=0x560cc722a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 0x560cc722a0b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009a20 tx=0x7f03f4007e20 comp rx=0 tx=0).stop 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 shutdown_connections 2024-05-08T23:26:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7226720 0x560cc722a0b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7239e40 0x560cc723c230 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc722c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3832590772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7229a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f03db7fe700 0 client.17184 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 >> 172.21.15.16:0/3832590772 conn(0x560cc717e210 msgr2=0x560cc721a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 shutdown_connections 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.153+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3832590772 wait complete. 2024-05-08T23:26:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.154+0000 7f040e9a9b80 20 client.17184 trim_cache size 0 max 16384 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 Processor -- start 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 -- start start 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc7220350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 0x560cc721ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7224ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc70c9270 con 0x560cc7225e00 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc72212a0 con 0x560cc7220890 2024-05-08T23:26:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc7217dc0 con 0x560cc7226720 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 0x560cc721ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 0x560cc721ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48366/0 (socket says 172.21.15.16:48366) 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 -- 172.21.15.16:0/3928894082 learned_addr learned my addr 172.21.15.16:0/3928894082 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc7220350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f0400d8c700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7224ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 -- 172.21.15.16:0/3928894082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 msgr2=0x560cc7224ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7224ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 -- 172.21.15.16:0/3928894082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 msgr2=0x560cc7220350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc7220350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 -- 172.21.15.16:0/3928894082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc7136560 con 0x560cc7225e00 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f0400d8c700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7224ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:26:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc7220350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.172+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 0x560cc721ad20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009520 tx=0x7f03f0099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1658a77ed356c049 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f03e2ffd700 1 -- 172.21.15.16:0/3928894082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0099bd0 con 0x560cc7225e00 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f03e2ffd700 1 -- 172.21.15.16:0/3928894082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03f009da00 con 0x560cc7225e00 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f03e2ffd700 1 -- 172.21.15.16:0/3928894082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f009a070 con 0x560cc7225e00 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3928894082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 msgr2=0x560cc721ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 0x560cc721ad20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009520 tx=0x7f03f0099ad0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3928894082 shutdown_connections 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7226720 0x560cc7224ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7220890 0x560cc7220350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3928894082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7225e00 0x560cc721ad20 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3928894082 >> 172.21.15.16:0/3928894082 conn(0x560cc717e210 msgr2=0x560cc7235bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3928894082 shutdown_connections 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.173+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3928894082 wait complete. 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 -- start start 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 Processor -- start 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc72310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7231610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7226720 0x560cc722f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc722fd80 con 0x560cc7220890 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc722fef0 con 0x560cc7226720 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc72300b0 con 0x560cc7225e00 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f040e9a9b80 10 client.0 _ll_register_callbacks cb 0x560cc7115b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f03fbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc72310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f03fbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc72310d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48376/0 (socket says 172.21.15.16:48376) 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f03fbfff700 1 -- 172.21.15.16:0/3077243854 learned_addr learned my addr 172.21.15.16:0/3077243854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f0400d8c700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7226720 0x560cc722f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.174+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7231610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03fbfff700 1 -- 172.21.15.16:0/3077243854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 msgr2=0x560cc7231610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7231610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03fbfff700 1 -- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7226720 msgr2=0x560cc722f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7226720 0x560cc722f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03fbfff700 1 -- 172.21.15.16:0/3077243854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc1a61f0 con 0x560cc7220890 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03fbfff700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc72310d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x560cc723cb40 tx=0x7f03ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acda5747b1dd4ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03db7fe700 1 -- 172.21.15.16:0/3077243854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec024040 con 0x560cc7220890 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03db7fe700 1 -- 172.21.15.16:0/3077243854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f03ec017070 con 0x560cc7220890 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03f40090d0 con 0x560cc7220890 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03db7fe700 1 -- 172.21.15.16:0/3077243854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ec004220 con 0x560cc7220890 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f040e9a9b80 10 client.17190 fetch_fsmap learned FSMap version 25 2024-05-08T23:26:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560cc6ff5cd0 con 0x560cc7220890 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.175+0000 7f03db7fe700 1 -- 172.21.15.16:0/3077243854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03f40090d0 con 0x560cc7220890 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f03db7fe700 1 -- 172.21.15.16:0/3077243854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f03ec003860 con 0x560cc7220890 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f040e9a9b80 10 client.17190 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f040e9a9b80 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f040e9a9b80 20 client.17190 populate_metadata read hostname 'smithi016' 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7291700 0x560cc7293b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f040e9a9b80 4 client.17190 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:26:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cc7168ec0 con 0x560cc7291700 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7291700 0x560cc7293b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f03db7fe700 10 client.17190 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.176+0000 7f03fb7fe700 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7291700 0x560cc7293b20 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c70592b7654351 server_cookie=cc8ab6489b632cc1 in_seq=0 out_seq=0 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f03db7fe700 1 -- 172.21.15.16:0/3077243854 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560cc7168ec0 con 0x560cc7291700 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f03db7fe700 10 client.17190 handle_command_reply: tid=0 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 client.17190 shutdown 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 msgr2=0x560cc72310d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc72310d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x560cc723cb40 tx=0x7f03ec0047c0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 shutdown_connections 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560cc7291700 0x560cc7293b20 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560cc7226720 0x560cc722f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560cc7220890 0x560cc72310d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f03db7fe700 0 client.17190 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 --2- 172.21.15.16:0/3077243854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cc7225e00 0x560cc7231610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 >> 172.21.15.16:0/3077243854 conn(0x560cc717e210 msgr2=0x560cc7226b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 shutdown_connections 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 1 -- 172.21.15.16:0/3077243854 wait complete. 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.177+0000 7f040e9a9b80 20 client.17190 trim_cache size 0 max 16384 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:26:53.186 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout: "bd6efb39-0a11-4d5c-bbce-882a84db7c5b": { 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "bd6efb39-0a11-4d5c-bbce-882a84db7c5b", 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:26:53.187 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f03e40a1310 msgr2=0x7f03e40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f03e40a1310 0x7f03e40a37d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f03fc067bb0 tx=0x7f03ec00b410 comp rx=0 tx=0).stop 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 msgr2=0x7f03fc102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc102450 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f03f000d950 tx=0x7f03f000dc60 comp rx=0 tx=0).stop 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 shutdown_connections 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f03e40a1310 0x7f03e40a37d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03fc105f90 0x7f03fc102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03fc1000f0 0x7f03fc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 --2- 172.21.15.16:0/1001117624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03fc100ab0 0x7f03fc102450 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:26:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 >> 172.21.15.16:0/1001117624 conn(0x7f03fc076ba0 msgr2=0x7f03fc104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:26:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 shutdown_connections 2024-05-08T23:26:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:26:53.178+0000 7f04027ef700 1 -- 172.21.15.16:0/1001117624 wait complete. 2024-05-08T23:26:53.191 INFO:tasks.cephfs.filesystem:scrub status for tag:bd6efb39-0a11-4d5c-bbce-882a84db7c5b - {'path': '/', 'tag': 'bd6efb39-0a11-4d5c-bbce-882a84db7c5b', 'options': 'recursive,force'} 2024-05-08T23:26:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:53 smithi016 ceph-mon[42541]: pgmap v224: 129 pgs: 129 active+clean; 9.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:53 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:26:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:53 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:53 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:26:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:53 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:54.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:53 smithi189 ceph-mon[46919]: pgmap v224: 129 pgs: 129 active+clean; 9.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:53 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:26:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:53 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:53 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:26:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:53 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:53 smithi155 ceph-mon[47105]: pgmap v224: 129 pgs: 129 active+clean; 9.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 10 op/s 2024-05-08T23:26:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:53 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:26:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:53 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:53 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:26:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:53 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:26:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:55 smithi016 ceph-mon[42541]: pgmap v225: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-05-08T23:26:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:55 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:26:56.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:55 smithi189 ceph-mon[46919]: pgmap v225: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-05-08T23:26:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:55 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:26:56.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:55 smithi155 ceph-mon[47105]: pgmap v225: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 16 op/s 2024-05-08T23:26:56.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:55 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:26:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:57 smithi016 ceph-mon[42541]: pgmap v226: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-08T23:26:58.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:57 smithi189 ceph-mon[46919]: pgmap v226: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-08T23:26:58.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:57 smithi155 ceph-mon[47105]: pgmap v226: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-08T23:27:00.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:26:59 smithi016 ceph-mon[42541]: pgmap v227: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-08T23:27:00.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:26:59 smithi189 ceph-mon[46919]: pgmap v227: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-08T23:27:00.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:26:59 smithi155 ceph-mon[47105]: pgmap v227: 129 pgs: 129 active+clean; 9.4 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-05-08T23:27:02.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:01 smithi016 ceph-mon[42541]: pgmap v228: 129 pgs: 129 active+clean; 9.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-05-08T23:27:02.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:01 smithi189 ceph-mon[46919]: pgmap v228: 129 pgs: 129 active+clean; 9.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-05-08T23:27:02.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:01 smithi155 ceph-mon[47105]: pgmap v228: 129 pgs: 129 active+clean; 9.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-05-08T23:27:04.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:03 smithi016 ceph-mon[42541]: pgmap v229: 129 pgs: 129 active+clean; 9.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-05-08T23:27:04.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:03 smithi189 ceph-mon[46919]: pgmap v229: 129 pgs: 129 active+clean; 9.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-05-08T23:27:04.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:03 smithi155 ceph-mon[47105]: pgmap v229: 129 pgs: 129 active+clean; 9.6 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 11 op/s 2024-05-08T23:27:06.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:05 smithi016 ceph-mon[42541]: pgmap v230: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 98 KiB/s wr, 33 op/s 2024-05-08T23:27:06.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:05 smithi189 ceph-mon[46919]: pgmap v230: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 98 KiB/s wr, 33 op/s 2024-05-08T23:27:06.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:05 smithi155 ceph-mon[47105]: pgmap v230: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 98 KiB/s wr, 33 op/s 2024-05-08T23:27:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:06 smithi016 ceph-mon[42541]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:27:07.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:06 smithi189 ceph-mon[46919]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:27:07.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:06 smithi155 ceph-mon[47105]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T23:27:08.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:07 smithi016 ceph-mon[42541]: pgmap v232: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-05-08T23:27:08.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:07 smithi189 ceph-mon[46919]: pgmap v232: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-05-08T23:27:08.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:07 smithi155 ceph-mon[47105]: pgmap v232: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-05-08T23:27:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:09 smithi016 ceph-mon[42541]: pgmap v233: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-05-08T23:27:10.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:09 smithi189 ceph-mon[46919]: pgmap v233: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-05-08T23:27:10.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:09 smithi155 ceph-mon[47105]: pgmap v233: 129 pgs: 129 active+clean; 9.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 96 KiB/s wr, 32 op/s 2024-05-08T23:27:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:11 smithi016 ceph-mon[42541]: pgmap v234: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 99 KiB/s wr, 32 op/s 2024-05-08T23:27:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:11 smithi189 ceph-mon[46919]: pgmap v234: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 99 KiB/s wr, 32 op/s 2024-05-08T23:27:12.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:11 smithi155 ceph-mon[47105]: pgmap v234: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 99 KiB/s wr, 32 op/s 2024-05-08T23:27:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:13 smithi016 ceph-mon[42541]: pgmap v235: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 99 KiB/s wr, 32 op/s 2024-05-08T23:27:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:13 smithi016 ceph-mon[42541]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:27:14.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:13 smithi189 ceph-mon[46919]: pgmap v235: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 99 KiB/s wr, 32 op/s 2024-05-08T23:27:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:13 smithi189 ceph-mon[46919]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:27:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:13 smithi155 ceph-mon[47105]: pgmap v235: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 99 KiB/s wr, 32 op/s 2024-05-08T23:27:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:13 smithi155 ceph-mon[47105]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T23:27:16.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:15 smithi016 ceph-mon[42541]: pgmap v237: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-05-08T23:27:16.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:15 smithi189 ceph-mon[46919]: pgmap v237: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-05-08T23:27:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:15 smithi155 ceph-mon[47105]: pgmap v237: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-05-08T23:27:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:17 smithi016 ceph-mon[42541]: pgmap v238: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-05-08T23:27:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:17 smithi189 ceph-mon[46919]: pgmap v238: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-05-08T23:27:18.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:17 smithi155 ceph-mon[47105]: pgmap v238: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-05-08T23:27:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:19 smithi016 ceph-mon[42541]: pgmap v239: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-05-08T23:27:20.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:19 smithi189 ceph-mon[46919]: pgmap v239: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-05-08T23:27:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:19 smithi155 ceph-mon[47105]: pgmap v239: 129 pgs: 129 active+clean; 10 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-05-08T23:27:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:21 smithi016 ceph-mon[42541]: pgmap v240: 129 pgs: 129 active+clean; 10 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-05-08T23:27:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:21 smithi189 ceph-mon[46919]: pgmap v240: 129 pgs: 129 active+clean; 10 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-05-08T23:27:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:21 smithi155 ceph-mon[47105]: pgmap v240: 129 pgs: 129 active+clean; 10 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-05-08T23:27:23.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 -- 172.21.15.16:0/530924538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4105f90 msgr2=0x7fbdd4106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/530924538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4105f90 0x7fbdd4106370 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00099c0 tx=0x7fbdd0009cd0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 -- 172.21.15.16:0/530924538 shutdown_connections 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/530924538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdd4100ab0 0x7fbdd410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/530924538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4105f90 0x7fbdd4106370 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/530924538 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd41000f0 0x7fbdd4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 -- 172.21.15.16:0/530924538 >> 172.21.15.16:0/530924538 conn(0x7fbdd4076ba0 msgr2=0x7fbdd4076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 -- 172.21.15.16:0/530924538 shutdown_connections 2024-05-08T23:27:23.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.340+0000 7fbddb8e5700 1 -- 172.21.15.16:0/530924538 wait complete. 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 Processor -- start 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 -- start start 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd41000f0 0x7fbdd4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 0x7fbdd4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdd4105f90 0x7fbdd419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd419c170 con 0x7fbdd4100ab0 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd419c2e0 con 0x7fbdd41000f0 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbddb8e5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd419c4a0 con 0x7fbdd4105f90 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbdd8e80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 0x7fbdd4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbdd8e80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 0x7fbdd4197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43826/0 (socket says 172.21.15.16:43826) 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbdd8e80700 1 -- 172.21.15.16:0/2551186609 learned_addr learned my addr 172.21.15.16:0/2551186609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.341+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdd4105f90 0x7fbdd419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd9681700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd41000f0 0x7fbdd4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd8e80700 1 -- 172.21.15.16:0/2551186609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdd4105f90 msgr2=0x7fbdd419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdd4105f90 0x7fbdd419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd8e80700 1 -- 172.21.15.16:0/2551186609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd41000f0 msgr2=0x7fbdd4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd41000f0 0x7fbdd4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd8e80700 1 -- 172.21.15.16:0/2551186609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00095f0 con 0x7fbdd4100ab0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 0x7fbdd4197790 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c920 tx=0x7fbdc400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdca7fc700 1 -- 172.21.15.16:0/2551186609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4007b00 con 0x7fbdd4100ab0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdca7fc700 1 -- 172.21.15.16:0/2551186609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbdc400ce70 con 0x7fbdd4100ab0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd419c780 con 0x7fbdd4100ab0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbdca7fc700 1 -- 172.21.15.16:0/2551186609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4005440 con 0x7fbdd4100ab0 2024-05-08T23:27:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.342+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd41a3fc0 con 0x7fbdd4100ab0 2024-05-08T23:27:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.344+0000 7fbdca7fc700 1 -- 172.21.15.16:0/2551186609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbdc40055a0 con 0x7fbdd4100ab0 2024-05-08T23:27:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.345+0000 7fbdca7fc700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbdc00a1310 0x7fbdc00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.345+0000 7fbdca7fc700 1 -- 172.21.15.16:0/2551186609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fbdc40ccbe0 con 0x7fbdd4100ab0 2024-05-08T23:27:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.345+0000 7fbdd9681700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbdc00a1310 0x7fbdc00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbdd9681700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbdc00a1310 0x7fbdc00a37d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b580 tx=0x7fbdd0005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 Processor -- start 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 -- start start 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d8791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d873350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d7ca150 con 0x55ef6d879720 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d7c7b10 con 0x55ef6d878e00 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d7c3400 con 0x55ef6d873890 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.346+0000 7fbdd8e80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d873350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d873350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43830/0 (socket says 172.21.15.16:43830) 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 -- 172.21.15.16:0/3037247486 learned_addr learned my addr 172.21.15.16:0/3037247486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd9681700 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d8791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 -- 172.21.15.16:0/3037247486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 msgr2=0x55ef6d873cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 -- 172.21.15.16:0/3037247486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 msgr2=0x55ef6d8791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d8791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 -- 172.21.15.16:0/3037247486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef6d6443b0 con 0x55ef6d879720 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d873350 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40c9ff0 tx=0x7fbdc4099330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292ef188ed2b9a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdbf7fe700 1 -- 172.21.15.16:0/3037247486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4099950 con 0x55ef6d879720 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdbf7fe700 1 -- 172.21.15.16:0/3037247486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbdc409d780 con 0x55ef6d879720 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.347+0000 7fbdbf7fe700 1 -- 172.21.15.16:0/3037247486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4099e10 con 0x55ef6d879720 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3037247486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 msgr2=0x55ef6d873350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d873350 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40c9ff0 tx=0x7fbdc4099330 comp rx=0 tx=0).stop 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3037247486 shutdown_connections 2024-05-08T23:27:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d8791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3037247486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d873350 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3037247486 >> 172.21.15.16:0/3037247486 conn(0x55ef6d7d1210 msgr2=0x55ef6d86ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3037247486 shutdown_connections 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3037247486 wait complete. 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.348+0000 7fbde7a9fb80 1 -- start start 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 Processor -- start 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d87cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d87d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d87d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d75f560 con 0x55ef6d879720 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d75f200 con 0x55ef6d878e00 2024-05-08T23:27:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d6466a0 con 0x55ef6d873890 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbde7a9fb80 10 client.0 _ll_register_callbacks cb 0x55ef6d768b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbdd9e82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d87d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbdd9e82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d87d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43836/0 (socket says 172.21.15.16:43836) 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbdd9681700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d87cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbdd9e82700 1 -- 172.21.15.16:0/4248299699 learned_addr learned my addr 172.21.15.16:0/4248299699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.349+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d87d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9e82700 1 -- 172.21.15.16:0/4248299699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 msgr2=0x55ef6d87cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d87cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9e82700 1 -- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 msgr2=0x55ef6d87d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d87d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9e82700 1 -- 172.21.15.16:0/4248299699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd41a3fc0 con 0x55ef6d879720 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9681700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d87cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d87d580 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc011e80 tx=0x7fbdcc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699fe36d59f1ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdbcff9700 1 -- 172.21.15.16:0/4248299699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00e040 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdbcff9700 1 -- 172.21.15.16:0/4248299699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fbdcc01a070 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef6d71c270 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdbcff9700 1 -- 172.21.15.16:0/4248299699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdcc01e420 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbde7a9fb80 10 client.17202 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ef6d5f0b60 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.350+0000 7fbdbcff9700 1 -- 172.21.15.16:0/4248299699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ef6d71c270 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbdbcff9700 1 -- 172.21.15.16:0/4248299699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdcc016520 con 0x55ef6d879720 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbde7a9fb80 10 client.17202 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbde7a9fb80 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbde7a9fb80 20 client.17202 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d88d2a0 0x55ef6d88f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbde7a9fb80 4 client.17202 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef6d842810 con 0x55ef6d88d2a0 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbdd9681700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d88d2a0 0x55ef6d88f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbdbcff9700 10 client.17202 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.351+0000 7fbdd9681700 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d88d2a0 0x55ef6d88f6c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da415878872efb9 server_cookie=5004a4f001bfd60d in_seq=0 out_seq=0 2024-05-08T23:27:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbdbcff9700 1 -- 172.21.15.16:0/4248299699 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef6d842810 con 0x55ef6d88d2a0 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbdbcff9700 10 client.17202 handle_command_reply: tid=0 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 client.17202 shutdown 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 msgr2=0x55ef6d87d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d87d580 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc011e80 tx=0x7fbdcc014600 comp rx=0 tx=0).stop 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 shutdown_connections 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d87d580 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d87cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d88d2a0 0x55ef6d88f6c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/4248299699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d87d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 >> 172.21.15.16:0/4248299699 conn(0x55ef6d7d1210 msgr2=0x55ef6d86d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbdbcff9700 0 client.17202 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 shutdown_connections 2024-05-08T23:27:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.352+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/4248299699 wait complete. 2024-05-08T23:27:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.353+0000 7fbde7a9fb80 20 client.17202 trim_cache size 0 max 16384 2024-05-08T23:27:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 Processor -- start 2024-05-08T23:27:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 -- start start 2024-05-08T23:27:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d888c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d86dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d71c270 con 0x55ef6d879720 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d86eb30 con 0x55ef6d878e00 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d88a9b0 con 0x55ef6d873890 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d86dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d86dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43852/0 (socket says 172.21.15.16:43852) 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 -- 172.21.15.16:0/3338736687 learned_addr learned my addr 172.21.15.16:0/3338736687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9681700 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d888c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 -- 172.21.15.16:0/3338736687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 msgr2=0x55ef6d873350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 -- 172.21.15.16:0/3338736687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 msgr2=0x55ef6d888c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d888c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.371+0000 7fbdd9e82700 1 -- 172.21.15.16:0/3338736687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef6d789560 con 0x55ef6d879720 2024-05-08T23:27:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d888c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d86dd20 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc000f80 tx=0x7fbdcc011290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7a9fa3e525d08b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbdbffff700 1 -- 172.21.15.16:0/3338736687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00e070 con 0x55ef6d879720 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbdbffff700 1 -- 172.21.15.16:0/3338736687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbdcc00f040 con 0x55ef6d879720 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbdbffff700 1 -- 172.21.15.16:0/3338736687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc005310 con 0x55ef6d879720 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3338736687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 msgr2=0x55ef6d86dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.372+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d86dd20 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc000f80 tx=0x7fbdcc011290 comp rx=0 tx=0).stop 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3338736687 shutdown_connections 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d86dd20 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d873890 0x55ef6d873350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/3338736687 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d878e00 0x55ef6d888c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3338736687 >> 172.21.15.16:0/3338736687 conn(0x55ef6d7d1210 msgr2=0x55ef6d87b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3338736687 shutdown_connections 2024-05-08T23:27:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/3338736687 wait complete. 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- start start 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 Processor -- start 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d873890 0x55ef6d882150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d878e00 0x55ef6d882690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d882bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d8e2820 con 0x55ef6d879720 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d8e2990 con 0x55ef6d873890 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef6d8e2b50 con 0x55ef6d878e00 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.373+0000 7fbde7a9fb80 10 client.0 _ll_register_callbacks cb 0x55ef6d768b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d882bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd8e80700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d878e00 0x55ef6d882690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d882bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43868/0 (socket says 172.21.15.16:43868) 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 -- 172.21.15.16:0/1800014008 learned_addr learned my addr 172.21.15.16:0/1800014008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9681700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d873890 0x55ef6d882150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 -- 172.21.15.16:0/1800014008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d878e00 msgr2=0x55ef6d882690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d878e00 0x55ef6d882690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 -- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d873890 msgr2=0x55ef6d882150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d873890 0x55ef6d882150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 -- 172.21.15.16:0/1800014008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef6d5f0b60 con 0x55ef6d879720 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd8e80700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d878e00 0x55ef6d882690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9681700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d873890 0x55ef6d882150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdd9e82700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d882bd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc016010 tx=0x7fbdcc009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ade86f1636b97f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdbcff9700 1 -- 172.21.15.16:0/1800014008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc03c070 con 0x55ef6d879720 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbdbcff9700 1 -- 172.21.15.16:0/1800014008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fbdcc01a040 con 0x55ef6d879720 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.374+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdc409d780 con 0x55ef6d879720 2024-05-08T23:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbdbcff9700 1 -- 172.21.15.16:0/1800014008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdcc00e040 con 0x55ef6d879720 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 10 client.17208 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ef6d648cd0 con 0x55ef6d879720 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbdbcff9700 1 -- 172.21.15.16:0/1800014008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbdc409d780 con 0x55ef6d879720 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbdbcff9700 1 -- 172.21.15.16:0/1800014008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdcc03f030 con 0x55ef6d879720 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 10 client.17208 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 20 client.17208 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d8e3ed0 0x55ef6d8e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 4 client.17208 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef6d7bbec0 con 0x55ef6d8e3ed0 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.375+0000 7fbdd9681700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d8e3ed0 0x55ef6d8e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbdbcff9700 10 client.17208 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbdd9681700 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d8e3ed0 0x55ef6d8e62c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca341d4a1e10b54a server_cookie=d836b782ffbc8323 in_seq=0 out_seq=0 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbdbcff9700 1 -- 172.21.15.16:0/1800014008 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ef6d7bbec0 con 0x55ef6d8e3ed0 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbdbcff9700 10 client.17208 handle_command_reply: tid=0 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 client.17208 shutdown 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 msgr2=0x55ef6d882bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d882bd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc016010 tx=0x7fbdcc009a80 comp rx=0 tx=0).stop 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 shutdown_connections 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef6d879720 0x55ef6d882bd0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55ef6d8e3ed0 0x55ef6d8e62c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ef6d873890 0x55ef6d882150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 --2- 172.21.15.16:0/1800014008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef6d878e00 0x55ef6d882690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbdbcff9700 0 client.17208 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 >> 172.21.15.16:0/1800014008 conn(0x55ef6d7d1210 msgr2=0x55ef6d877be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 shutdown_connections 2024-05-08T23:27:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 1 -- 172.21.15.16:0/1800014008 wait complete. 2024-05-08T23:27:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.376+0000 7fbde7a9fb80 20 client.17208 trim_cache size 0 max 16384 2024-05-08T23:27:23.385 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:27:23.385 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:27:23.385 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:27:23.385 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbdc00a1310 msgr2=0x7fbdc00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbdc00a1310 0x7fbdc00a37d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b580 tx=0x7fbdd0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 msgr2=0x7fbdd4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 0x7fbdd4197790 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400c920 tx=0x7fbdc400cc30 comp rx=0 tx=0).stop 2024-05-08T23:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 shutdown_connections 2024-05-08T23:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbdc00a1310 0x7fbdc00a37d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbdd4105f90 0x7fbdd419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbdd41000f0 0x7fbdd4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 --2- 172.21.15.16:0/2551186609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd4100ab0 0x7fbdd4197790 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 >> 172.21.15.16:0/2551186609 conn(0x7fbdd4076ba0 msgr2=0x7fbdd40693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 shutdown_connections 2024-05-08T23:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.377+0000 7fbddb8e5700 1 -- 172.21.15.16:0/2551186609 wait complete. 2024-05-08T23:27:23.391 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:23.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 -- 172.21.15.16:0/2360425572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c10b3c0 msgr2=0x7f1b6c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 --2- 172.21.15.16:0/2360425572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c10b3c0 0x7f1b6c10d880 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400b0a0 tx=0x7f1b6400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 -- 172.21.15.16:0/2360425572 shutdown_connections 2024-05-08T23:27:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 --2- 172.21.15.16:0/2360425572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c10b3c0 0x7f1b6c10d880 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 --2- 172.21.15.16:0/2360425572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b6c101fa0 0x7f1b6c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 --2- 172.21.15.16:0/2360425572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 -- 172.21.15.16:0/2360425572 >> 172.21.15.16:0/2360425572 conn(0x7f1b6c0fb7e0 msgr2=0x7f1b6c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 -- 172.21.15.16:0/2360425572 shutdown_connections 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.527+0000 7f1b74782700 1 -- 172.21.15.16:0/2360425572 wait complete. 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 Processor -- start 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 -- start start 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b6c101fa0 0x7f1b6c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6c10b3c0 0x7f1b6c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19c2b0 con 0x7f1b6c1028c0 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19c420 con 0x7f1b6c101fa0 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b74782700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c19c5e0 con 0x7f1b6c10b3c0 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b71d1d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b71d1d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43894/0 (socket says 172.21.15.16:43894) 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b71d1d700 1 -- 172.21.15.16:0/2775412980 learned_addr learned my addr 172.21.15.16:0/2775412980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b7251e700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b6c101fa0 0x7f1b6c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b72d1f700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6c10b3c0 0x7f1b6c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b71d1d700 1 -- 172.21.15.16:0/2775412980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6c10b3c0 msgr2=0x7f1b6c19bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.528+0000 7f1b71d1d700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6c10b3c0 0x7f1b6c19bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b71d1d700 1 -- 172.21.15.16:0/2775412980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b6c101fa0 msgr2=0x7f1b6c1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b71d1d700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b6c101fa0 0x7f1b6c1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b71d1d700 1 -- 172.21.15.16:0/2775412980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009d00 con 0x7f1b6c1028c0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b71d1d700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c197920 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00d750 tx=0x7f1b5c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c004020 con 0x7f1b6c1028c0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b5c007590 con 0x7f1b6c1028c0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c19c8c0 con 0x7f1b6c1028c0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c005380 con 0x7f1b6c1028c0 2024-05-08T23:27:23.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.529+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c1a4100 con 0x7f1b6c1028c0 2024-05-08T23:27:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.530+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c1064f0 con 0x7f1b6c1028c0 2024-05-08T23:27:23.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.531+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1b5c004750 con 0x7f1b6c1028c0 2024-05-08T23:27:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.532+0000 7f1b637fe700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1b580a1310 0x7f1b580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.532+0000 7f1b7251e700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1b580a1310 0x7f1b580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.532+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1b5c0ccc80 con 0x7f1b6c1028c0 2024-05-08T23:27:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.533+0000 7f1b7251e700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1b580a1310 0x7f1b580a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b68005950 tx=0x7f1b6800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.536+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1b5c095c90 con 0x7f1b6c1028c0 2024-05-08T23:27:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.808+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b6c198690 con 0x7f1b6c1028c0 2024-05-08T23:27:23.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.809+0000 7f1b637fe700 1 -- 172.21.15.16:0/2775412980 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f1b5c014090 con 0x7f1b6c1028c0 2024-05-08T23:27:23.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:27:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:27:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1b580a1310 msgr2=0x7f1b580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1b580a1310 0x7f1b580a37d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1b68005950 tx=0x7f1b6800b410 comp rx=0 tx=0).stop 2024-05-08T23:27:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 msgr2=0x7f1b6c197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c197920 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00d750 tx=0x7f1b5c00da60 comp rx=0 tx=0).stop 2024-05-08T23:27:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 shutdown_connections 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1b580a1310 0x7f1b580a37d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b6c10b3c0 0x7f1b6c19bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b6c101fa0 0x7f1b6c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 --2- 172.21.15.16:0/2775412980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c1028c0 0x7f1b6c197920 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.811+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 >> 172.21.15.16:0/2775412980 conn(0x7f1b6c0fb7e0 msgr2=0x7f1b6c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.812+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 shutdown_connections 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.812+0000 7f1b74782700 1 -- 172.21.15.16:0/2775412980 wait complete. 2024-05-08T23:27:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:27:23.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:27:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 -- 172.21.15.16:0/1485718370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100a10 msgr2=0x7f0008104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 --2- 172.21.15.16:0/1485718370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100a10 0x7f0008104a30 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efff0009990 tx=0x7efff0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 -- 172.21.15.16:0/1485718370 shutdown_connections 2024-05-08T23:27:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 --2- 172.21.15.16:0/1485718370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 0x7f000810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 --2- 172.21.15.16:0/1485718370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0008100060 0x7f0008100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 --2- 172.21.15.16:0/1485718370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100a10 0x7f0008104a30 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 -- 172.21.15.16:0/1485718370 >> 172.21.15.16:0/1485718370 conn(0x7f0008076f40 msgr2=0x7f0008077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 -- 172.21.15.16:0/1485718370 shutdown_connections 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.964+0000 7f000cc64700 1 -- 172.21.15.16:0/1485718370 wait complete. 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 Processor -- start 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 -- start start 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 0x7f0008198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 0x7f0008198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0008198ba0 0x7f00081a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00081a3880 con 0x7f0008100060 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00081a39f0 con 0x7f0008105160 2024-05-08T23:27:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000cc64700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00081a3bb0 con 0x7f0008198ba0 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 0x7f0008198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 0x7f0008198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43920/0 (socket says 172.21.15.16:43920) 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.965+0000 7f000659c700 1 -- 172.21.15.16:0/2884652607 learned_addr learned my addr 172.21.15.16:0/2884652607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f0006d9d700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0008198ba0 0x7f00081a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 0x7f0008198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000659c700 1 -- 172.21.15.16:0/2884652607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0008198ba0 msgr2=0x7f00081a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000659c700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0008198ba0 0x7f00081a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000659c700 1 -- 172.21.15.16:0/2884652607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 msgr2=0x7f0008198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000659c700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 0x7f0008198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000659c700 1 -- 172.21.15.16:0/2884652607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff00095f0 con 0x7f0008100060 2024-05-08T23:27:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 0x7f0008198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000659c700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 0x7f0008198120 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7efff800d950 tx=0x7efff800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7effff7fe700 1 -- 172.21.15.16:0/2884652607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8009940 con 0x7f0008100060 2024-05-08T23:27:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7effff7fe700 1 -- 172.21.15.16:0/2884652607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff8010460 con 0x7f0008100060 2024-05-08T23:27:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00081a3e90 con 0x7f0008100060 2024-05-08T23:27:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7effff7fe700 1 -- 172.21.15.16:0/2884652607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800f620 con 0x7f0008100060 2024-05-08T23:27:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.966+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00081a4310 con 0x7f0008100060 2024-05-08T23:27:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.968+0000 7effff7fe700 1 -- 172.21.15.16:0/2884652607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efff80105d0 con 0x7f0008100060 2024-05-08T23:27:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.969+0000 7effff7fe700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efff40a1290 0x7efff40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.969+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efff40a1290 0x7efff40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.969+0000 7effff7fe700 1 -- 172.21.15.16:0/2884652607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7efff80ccf10 con 0x7f0008100060 2024-05-08T23:27:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.970+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efff40a1290 0x7efff40a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff000b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.970+0000 7f0018e1eb80 1 Processor -- start 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.970+0000 7f0018e1eb80 1 -- start start 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23ae00 0x55d70b23b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 0x55d70b235350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b235cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b18c150 con 0x55d70b23b720 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b189b10 con 0x55d70b23ae00 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b185400 con 0x55d70b235890 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 0x55d70b235350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 0x55d70b235350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43924/0 (socket says 172.21.15.16:43924) 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 -- 172.21.15.16:0/3639588108 learned_addr learned my addr 172.21.15.16:0/3639588108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f000659c700 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23ae00 0x55d70b23b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0006d9d700 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b235cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 -- 172.21.15.16:0/3639588108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 msgr2=0x55d70b235cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b235cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 -- 172.21.15.16:0/3639588108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23ae00 msgr2=0x55d70b23b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23ae00 0x55d70b23b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 -- 172.21.15.16:0/3639588108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70b0063b0 con 0x55d70b23b720 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.971+0000 7f0005d9b700 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 0x55d70b235350 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efff0009990 tx=0x7efff001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde40579d2ba775c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7efffcff9700 1 -- 172.21.15.16:0/3639588108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff002b070 con 0x55d70b23b720 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7efffcff9700 1 -- 172.21.15.16:0/3639588108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff001e040 con 0x55d70b23b720 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7efffcff9700 1 -- 172.21.15.16:0/3639588108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff00049d0 con 0x55d70b23b720 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 -- 172.21.15.16:0/3639588108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 msgr2=0x55d70b235350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 0x55d70b235350 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efff0009990 tx=0x7efff001f000 comp rx=0 tx=0).stop 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 -- 172.21.15.16:0/3639588108 shutdown_connections 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b235cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23ae00 0x55d70b23b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/3639588108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23b720 0x55d70b235350 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.972+0000 7f0018e1eb80 1 -- 172.21.15.16:0/3639588108 >> 172.21.15.16:0/3639588108 conn(0x55d70b193210 msgr2=0x55d70b22fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 -- 172.21.15.16:0/3639588108 shutdown_connections 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 -- 172.21.15.16:0/3639588108 wait complete. 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 -- start start 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 Processor -- start 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b235890 0x55d70b241750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b23ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23b720 0x55d70b23f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b121560 con 0x55d70b23ae00 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b121200 con 0x55d70b235890 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b0086a0 con 0x55d70b23b720 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b23ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0018e1eb80 10 client.0 _ll_register_callbacks cb 0x55d70b12ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b23ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43938/0 (socket says 172.21.15.16:43938) 2024-05-08T23:27:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0005d9b700 1 -- 172.21.15.16:0/1531830453 learned_addr learned my addr 172.21.15.16:0/1531830453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.973+0000 7f0006d9d700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23b720 0x55d70b23f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f000659c700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b235890 0x55d70b241750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0005d9b700 1 -- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23b720 msgr2=0x55d70b23f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0005d9b700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23b720 0x55d70b23f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0005d9b700 1 -- 172.21.15.16:0/1531830453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b235890 msgr2=0x55d70b241750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0005d9b700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b235890 0x55d70b241750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0005d9b700 1 -- 172.21.15.16:0/1531830453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000810d550 con 0x55d70b23ae00 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0005d9b700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b23ea40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efff0009350 tx=0x7efff0004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc623457b52ffe75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7effedffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7effedffb700 1 -- 172.21.15.16:0/1531830453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff002b030 con 0x55d70b23ae00 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7effedffb700 1 -- 172.21.15.16:0/1531830453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7efff002c070 con 0x55d70b23ae00 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d70b0de270 con 0x55d70b23ae00 2024-05-08T23:27:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7effedffb700 1 -- 172.21.15.16:0/1531830453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff0004a60 con 0x55d70b23ae00 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0018e1eb80 10 client.17226 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.974+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d70afb2b60 con 0x55d70b23ae00 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7effedffb700 1 -- 172.21.15.16:0/1531830453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d70b0de270 con 0x55d70b23ae00 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7effedffb700 1 -- 172.21.15.16:0/1531830453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efff0030bb0 con 0x55d70b23ae00 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f0018e1eb80 10 client.17226 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f0018e1eb80 10 client.17226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f0018e1eb80 20 client.17226 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b24ecd0 0x55d70b2510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f0018e1eb80 4 client.17226 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d70b204810 con 0x55d70b24ecd0 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.975+0000 7f000659c700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b24ecd0 0x55d70b2510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.976+0000 7f000659c700 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b24ecd0 0x55d70b2510f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8267802528804b00 server_cookie=653b91993ebb114f in_seq=0 out_seq=0 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.976+0000 7effedffb700 10 client.17226 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.976+0000 7effedffb700 1 -- 172.21.15.16:0/1531830453 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d70b204810 con 0x55d70b24ecd0 2024-05-08T23:27:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.976+0000 7effedffb700 10 client.17226 handle_command_reply: tid=0 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.976+0000 7f0018e1eb80 1 client.17226 shutdown 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 msgr2=0x55d70b23ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b23ea40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efff0009350 tx=0x7efff0004f40 comp rx=0 tx=0).stop 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 shutdown_connections 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23b720 0x55d70b23f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b24ecd0 0x55d70b2510f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b235890 0x55d70b241750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/1531830453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b23ea40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7effedffb700 0 client.17226 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 >> 172.21.15.16:0/1531830453 conn(0x55d70b193210 msgr2=0x55d70b22f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 shutdown_connections 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 1 -- 172.21.15.16:0/1531830453 wait complete. 2024-05-08T23:27:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.977+0000 7f0018e1eb80 20 client.17226 trim_cache size 0 max 16384 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.991+0000 7f0018e1eb80 1 Processor -- start 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.991+0000 7f0018e1eb80 1 -- start start 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 0x55d70b235350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23ae00 0x55d70b22fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b246100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b22aad0 con 0x55d70b235890 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b23f9c0 con 0x55d70b23b720 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b22a800 con 0x55d70b23ae00 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 0x55d70b235350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 0x55d70b235350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43952/0 (socket says 172.21.15.16:43952) 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 -- 172.21.15.16:0/352959378 learned_addr learned my addr 172.21.15.16:0/352959378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 -- 172.21.15.16:0/352959378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23ae00 msgr2=0x55d70b22fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0005d9b700 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23ae00 0x55d70b22fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0006d9d700 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b246100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23ae00 0x55d70b22fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 -- 172.21.15.16:0/352959378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 msgr2=0x55d70b246100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b246100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 -- 172.21.15.16:0/352959378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d70b235f40 con 0x55d70b235890 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f0005d9b700 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23ae00 0x55d70b22fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7f000659c700 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 0x55d70b235350 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efff8099920 tx=0x7efff8083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f3c6a96b247d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7efffd7fa700 1 -- 172.21.15.16:0/352959378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8014070 con 0x55d70b235890 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7efffd7fa700 1 -- 172.21.15.16:0/352959378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff80a20a0 con 0x55d70b235890 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.992+0000 7efffd7fa700 1 -- 172.21.15.16:0/352959378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8099110 con 0x55d70b235890 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 -- 172.21.15.16:0/352959378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 msgr2=0x55d70b235350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 0x55d70b235350 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7efff8099920 tx=0x7efff8083040 comp rx=0 tx=0).stop 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 -- 172.21.15.16:0/352959378 shutdown_connections 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b246100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b235890 0x55d70b235350 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/352959378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b23ae00 0x55d70b22fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 -- 172.21.15.16:0/352959378 >> 172.21.15.16:0/352959378 conn(0x55d70b193210 msgr2=0x55d70b241380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 -- 172.21.15.16:0/352959378 shutdown_connections 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 -- 172.21.15.16:0/352959378 wait complete. 2024-05-08T23:27:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.993+0000 7f0018e1eb80 1 -- start start 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 Processor -- start 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b246110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b246650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b236e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b237560 con 0x55d70b23ae00 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b2376d0 con 0x55d70b23b720 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d70b237890 con 0x55d70b235890 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0018e1eb80 10 client.0 _ll_register_callbacks cb 0x55d70b12ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b246650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b246650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43964/0 (socket says 172.21.15.16:43964) 2024-05-08T23:27:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0005d9b700 1 -- 172.21.15.16:0/2263259212 learned_addr learned my addr 172.21.15.16:0/2263259212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0006d9d700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b236e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f000659c700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b246110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0005d9b700 1 -- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 msgr2=0x55d70b246110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b246110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.994+0000 7f0005d9b700 1 -- 172.21.15.16:0/2263259212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 msgr2=0x55d70b236e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b236e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0005d9b700 1 -- 172.21.15.16:0/2263259212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00081a4310 con 0x55d70b23ae00 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0005d9b700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b246650 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efff000b790 tx=0x7efff000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94054d418046034 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7effedffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7effedffb700 1 -- 172.21.15.16:0/2263259212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff002b040 con 0x55d70b23ae00 2024-05-08T23:27:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7effedffb700 1 -- 172.21.15.16:0/2263259212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7efff000fcf0 con 0x55d70b23ae00 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efff001e040 con 0x55d70b23ae00 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7effedffb700 1 -- 172.21.15.16:0/2263259212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff00059e0 con 0x55d70b23ae00 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 10 client.17232 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d70b14b560 con 0x55d70b23ae00 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7effedffb700 1 -- 172.21.15.16:0/2263259212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff001e040 con 0x55d70b23ae00 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7effedffb700 1 -- 172.21.15.16:0/2263259212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efff0030760 con 0x55d70b23ae00 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 10 client.17232 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 10 client.17232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 20 client.17232 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.995+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b2a6090 0x55d70b2a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f0018e1eb80 4 client.17232 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:27:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d70b17dec0 con 0x55d70b2a6090 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f000659c700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b2a6090 0x55d70b2a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f000659c700 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b2a6090 0x55d70b2a84b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bba4c4d38e5bed4d server_cookie=7f957e1e60c5729 in_seq=0 out_seq=0 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7effedffb700 10 client.17232 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7effedffb700 1 -- 172.21.15.16:0/2263259212 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d70b17dec0 con 0x55d70b2a6090 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7effedffb700 10 client.17232 handle_command_reply: tid=0 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f0018e1eb80 1 client.17232 shutdown 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 msgr2=0x55d70b246650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.996+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b246650 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7efff000b790 tx=0x7efff000f6a0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 shutdown_connections 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55d70b2a6090 0x55d70b2a84b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55d70b23b720 0x55d70b236e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d70b235890 0x55d70b246110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7effedffb700 0 client.17232 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 --2- 172.21.15.16:0/2263259212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d70b23ae00 0x55d70b246650 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 >> 172.21.15.16:0/2263259212 conn(0x55d70b193210 msgr2=0x55d70b23c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 shutdown_connections 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 1 -- 172.21.15.16:0/2263259212 wait complete. 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f0018e1eb80 20 client.17232 trim_cache size 0 max 16384 2024-05-08T23:27:24.005 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:27:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efff40a1290 msgr2=0x7efff40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efff40a1290 0x7efff40a3750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff000b540 comp rx=0 tx=0).stop 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 msgr2=0x7f0008198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 0x7f0008198120 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7efff800d950 tx=0x7efff800dc60 comp rx=0 tx=0).stop 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 shutdown_connections 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efff40a1290 0x7efff40a3750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.997+0000 7f000cc64700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0008198ba0 0x7f00081a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.998+0000 7f000cc64700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0008100060 0x7f0008198120 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.998+0000 7f000cc64700 1 --2- 172.21.15.16:0/2884652607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0008105160 0x7f0008198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.998+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 >> 172.21.15.16:0/2884652607 conn(0x7f0008076f40 msgr2=0x7f00080fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.998+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 shutdown_connections 2024-05-08T23:27:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:23.998+0000 7f000cc64700 1 -- 172.21.15.16:0/2884652607 wait complete. 2024-05-08T23:27:24.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:27:24.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:23 smithi016 ceph-mon[42541]: pgmap v241: 129 pgs: 129 active+clean; 10 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-05-08T23:27:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 -- 172.21.15.16:0/4264154093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0106280 msgr2=0x7f8fd010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/4264154093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0106280 0x7f8fd010de00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b0a0 tx=0x7f8fc800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 -- 172.21.15.16:0/4264154093 shutdown_connections 2024-05-08T23:27:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/4264154093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0106280 0x7f8fd010de00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/4264154093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0101160 0x7f8fd0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/4264154093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0101b10 0x7f8fd0105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 -- 172.21.15.16:0/4264154093 >> 172.21.15.16:0/4264154093 conn(0x7f8fd00fc9e0 msgr2=0x7f8fd00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 -- 172.21.15.16:0/4264154093 shutdown_connections 2024-05-08T23:27:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.151+0000 7f8fd49ee700 1 -- 172.21.15.16:0/4264154093 wait complete. 2024-05-08T23:27:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 Processor -- start 2024-05-08T23:27:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 -- start start 2024-05-08T23:27:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0101160 0x7f8fd01974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0106280 0x7f8fd0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 0x7f8fd019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019c7d0 con 0x7f8fd019bd50 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019c940 con 0x7f8fd0106280 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fd49ee700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019cb00 con 0x7f8fd0101160 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 0x7f8fd019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fced9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 0x7f8fd019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43984/0 (socket says 172.21.15.16:43984) 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fced9d700 1 -- 172.21.15.16:0/2139870495 learned_addr learned my addr 172.21.15.16:0/2139870495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fce59c700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0101160 0x7f8fd01974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0106280 0x7f8fd0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fced9d700 1 -- 172.21.15.16:0/2139870495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0101160 msgr2=0x7f8fd01974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.152+0000 7f8fced9d700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0101160 0x7f8fd01974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fced9d700 1 -- 172.21.15.16:0/2139870495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0106280 msgr2=0x7f8fd0197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fced9d700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0106280 0x7f8fd0197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fced9d700 1 -- 172.21.15.16:0/2139870495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc8009d00 con 0x7f8fd019bd50 2024-05-08T23:27:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0106280 0x7f8fd0197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fced9d700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 0x7f8fd019c1d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80094e0 tx=0x7f8fc8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fbf7fe700 1 -- 172.21.15.16:0/2139870495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800e040 con 0x7f8fd019bd50 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd01a4000 con 0x7f8fd019bd50 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fbf7fe700 1 -- 172.21.15.16:0/2139870495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fc8007e20 con 0x7f8fd019bd50 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fbf7fe700 1 -- 172.21.15.16:0/2139870495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8005040 con 0x7f8fd019bd50 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.153+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01a4430 con 0x7f8fd019bd50 2024-05-08T23:27:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.157+0000 7f8fe0ba8b80 1 Processor -- start 2024-05-08T23:27:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fbf7fe700 1 -- 172.21.15.16:0/2139870495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8fc801b450 con 0x7f8fd019bd50 2024-05-08T23:27:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 -- start start 2024-05-08T23:27:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00ed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ed720 0x555ef00e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef003e150 con 0x555ef00e7890 2024-05-08T23:27:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef003bb10 con 0x555ef00ed720 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef0037400 con 0x555ef00ece00 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fbf7fe700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8fb80a1290 0x7f8fb80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43992/0 (socket says 172.21.15.16:43992) 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.158+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/4171003588 learned_addr learned my addr 172.21.15.16:0/4171003588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00ed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fce59c700 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ed720 0x555ef00e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fbf7fe700 1 -- 172.21.15.16:0/2139870495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f8fc80ce4b0 con 0x7f8fd019bd50 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fced9d700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8fb80a1290 0x7f8fb80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/4171003588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 msgr2=0x555ef00ed1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00ed1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/4171003588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ed720 msgr2=0x555ef00e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ed720 0x555ef00e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/4171003588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eefeb83b0 con 0x555ef00e7890 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40149e0 tx=0x7f8fc400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0b2d9a719515e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fced9d700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8fb80a1290 0x7f8fb80a3750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80cb9b0 tx=0x7f8fc8086040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fbcff9700 1 -- 172.21.15.16:0/4171003588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4014310 con 0x555ef00e7890 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fbcff9700 1 -- 172.21.15.16:0/4171003588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fc401a020 con 0x555ef00e7890 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.159+0000 7f8fbcff9700 1 -- 172.21.15.16:0/4171003588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4003bb0 con 0x555ef00e7890 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/4171003588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 msgr2=0x555ef00e7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40149e0 tx=0x7f8fc400b500 comp rx=0 tx=0).stop 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/4171003588 shutdown_connections 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7cd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00ed1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/4171003588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ed720 0x555ef00e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/4171003588 >> 172.21.15.16:0/4171003588 conn(0x555ef0045210 msgr2=0x555ef00e1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/4171003588 shutdown_connections 2024-05-08T23:27:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.160+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/4171003588 wait complete. 2024-05-08T23:27:24.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 -- start start 2024-05-08T23:27:24.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 Processor -- start 2024-05-08T23:27:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eeffd3560 con 0x555ef00ed720 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eeffd3200 con 0x555ef00e7890 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eefeba6a0 con 0x555ef00ece00 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fe0ba8b80 10 client.0 _ll_register_callbacks cb 0x555eeffdcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43994/0 (socket says 172.21.15.16:43994) 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2558235068 learned_addr learned my addr 172.21.15.16:0/2558235068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fce59c700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.161+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 msgr2=0x555ef00f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2558235068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 msgr2=0x555ef00f0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2558235068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40146c0 con 0x555ef00ed720 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fce59c700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f1600 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400bd70 tx=0x7f8fc40145e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dff6e6d01b2aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/2558235068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4011b70 con 0x555ef00ed720 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/2558235068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f8fc4000e40 con 0x555ef00ed720 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eeff90270 con 0x555ef00ed720 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/2558235068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc4011cd0 con 0x555ef00ed720 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fe0ba8b80 10 client.17244 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555eefe64b60 con 0x555ef00ed720 2024-05-08T23:27:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.162+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/2558235068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555eeff90270 con 0x555ef00ed720 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/2558235068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fc401f420 con 0x555ef00ed720 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fe0ba8b80 10 client.17244 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fe0ba8b80 10 client.17244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fe0ba8b80 20 client.17244 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef01010e0 0x555ef0103500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fe0ba8b80 4 client.17244 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ef00b6810 con 0x555ef01010e0 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fce59c700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef01010e0 0x555ef0103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fb5ffb700 10 client.17244 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.163+0000 7f8fce59c700 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef01010e0 0x555ef0103500 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d46cafaf055dfb5 server_cookie=9cb0b48a37797bce in_seq=0 out_seq=0 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/2558235068 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ef00b6810 con 0x555ef01010e0 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fb5ffb700 10 client.17244 handle_command_reply: tid=0 2024-05-08T23:27:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 client.17244 shutdown 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 msgr2=0x555ef00f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f1600 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400bd70 tx=0x7f8fc40145e0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 shutdown_connections 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f1600 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef01010e0 0x555ef0103500 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2558235068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fb5ffb700 0 client.17244 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.164+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 >> 172.21.15.16:0/2558235068 conn(0x555ef0045210 msgr2=0x555ef00e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.165+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 shutdown_connections 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.165+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2558235068 wait complete. 2024-05-08T23:27:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.165+0000 7f8fe0ba8b80 20 client.17244 trim_cache size 0 max 16384 2024-05-08T23:27:24.172 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:23 smithi189 ceph-mon[46919]: pgmap v241: 129 pgs: 129 active+clean; 10 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-05-08T23:27:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 Processor -- start 2024-05-08T23:27:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 -- start start 2024-05-08T23:27:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ece00 0x555ef00ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ed720 0x555ef00e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef00f21b0 con 0x555ef00e7890 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef00dedf0 con 0x555ef00ece00 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef0108790 con 0x555ef00ed720 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44010/0 (socket says 172.21.15.16:44010) 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2685899713 learned_addr learned my addr 172.21.15.16:0/2685899713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fce59c700 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ece00 0x555ef00ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2685899713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ed720 msgr2=0x555ef00e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ed720 0x555ef00e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2685899713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ece00 msgr2=0x555ef00ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ece00 0x555ef00ec4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.179+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/2685899713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eefffd560 con 0x555ef00e7890 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7350 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005de0 tx=0x7f8fc400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0376834ea5a5254 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fbd7fa700 1 -- 172.21.15.16:0/2685899713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400bba0 con 0x555ef00e7890 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fbd7fa700 1 -- 172.21.15.16:0/2685899713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fc4014320 con 0x555ef00e7890 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fbd7fa700 1 -- 172.21.15.16:0/2685899713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4012450 con 0x555ef00e7890 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2685899713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 msgr2=0x555ef00e7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7350 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4005de0 tx=0x7f8fc400be20 comp rx=0 tx=0).stop 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2685899713 shutdown_connections 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ed720 0x555ef00e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00e7890 0x555ef00e7350 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/2685899713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00ece00 0x555ef00ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2685899713 >> 172.21.15.16:0/2685899713 conn(0x555ef0045210 msgr2=0x555ef00edbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2685899713 shutdown_connections 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.180+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/2685899713 wait complete. 2024-05-08T23:27:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 -- start start 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 Processor -- start 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef01565e0 con 0x555ef00ed720 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef0156720 con 0x555ef00e7890 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ef0156860 con 0x555ef00ece00 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fe0ba8b80 10 client.0 _ll_register_callbacks cb 0x555eeffdcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fcdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44022/0 (socket says 172.21.15.16:44022) 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/1579850079 learned_addr learned my addr 172.21.15.16:0/1579850079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fce59c700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.181+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 msgr2=0x555ef00f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/1579850079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 msgr2=0x555ef00f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fcdd9b700 1 -- 172.21.15.16:0/1579850079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eefe64b60 con 0x555ef00ed720 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fcdd9b700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f66f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8fc401f8a0 tx=0x7f8fc400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f02ab1e747c369 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/1579850079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401fa70 con 0x555ef00ed720 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/1579850079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f8fc401b030 con 0x555ef00ed720 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc401a020 con 0x555ef00ed720 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/1579850079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc400bba0 con 0x555ef00ed720 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fe0ba8b80 10 client.17250 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555eefebccd0 con 0x555ef00ed720 2024-05-08T23:27:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.182+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/1579850079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fc401a020 con 0x555ef00ed720 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/1579850079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8fc4004710 con 0x555ef00ed720 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 10 client.17250 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 10 client.17250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 20 client.17250 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef0157b30 0x555ef0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 4 client.17250 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555ef002fec0 con 0x555ef0157b30 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fce59c700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef0157b30 0x555ef0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fce59c700 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef0157b30 0x555ef0159f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=877bf15096524949 server_cookie=d2ffd3fbe770a3c4 in_seq=0 out_seq=0 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fb5ffb700 10 client.17250 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fb5ffb700 1 -- 172.21.15.16:0/1579850079 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555ef002fec0 con 0x555ef0157b30 2024-05-08T23:27:24.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fb5ffb700 10 client.17250 handle_command_reply: tid=0 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.183+0000 7f8fe0ba8b80 1 client.17250 shutdown 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 msgr2=0x555ef00f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f66f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8fc401f8a0 tx=0x7f8fc400be90 comp rx=0 tx=0).stop 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 shutdown_connections 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ef00ed720 0x555ef00f66f0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x555ef0157b30 0x555ef0159f20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555ef00e7890 0x555ef00f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 --2- 172.21.15.16:0/1579850079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ef00ece00 0x555ef00f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 >> 172.21.15.16:0/1579850079 conn(0x555ef0045210 msgr2=0x555ef00f10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fb5ffb700 0 client.17250 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 shutdown_connections 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 1 -- 172.21.15.16:0/1579850079 wait complete. 2024-05-08T23:27:24.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fe0ba8b80 20 client.17250 trim_cache size 0 max 16384 2024-05-08T23:27:24.192 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8fb80a1290 msgr2=0x7f8fb80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8fb80a1290 0x7f8fb80a3750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80cb9b0 tx=0x7f8fc8086040 comp rx=0 tx=0).stop 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 msgr2=0x7f8fd019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 0x7f8fd019c1d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80094e0 tx=0x7f8fc8009510 comp rx=0 tx=0).stop 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 shutdown_connections 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8fb80a1290 0x7f8fb80a3750 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd019bd50 0x7f8fd019c1d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fd0101160 0x7f8fd01974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 --2- 172.21.15.16:0/2139870495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fd0106280 0x7f8fd0197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.184+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 >> 172.21.15.16:0/2139870495 conn(0x7f8fd00fc9e0 msgr2=0x7f8fd00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.185+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 shutdown_connections 2024-05-08T23:27:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.185+0000 7f8fd49ee700 1 -- 172.21.15.16:0/2139870495 wait complete. 2024-05-08T23:27:24.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:27:24.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:23 smithi155 ceph-mon[47105]: pgmap v241: 129 pgs: 129 active+clean; 10 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 12 op/s 2024-05-08T23:27:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 -- 172.21.15.16:0/1044317934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc069420 msgr2=0x7fbcdc069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 --2- 172.21.15.16:0/1044317934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc069420 0x7fbcdc069800 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009990 tx=0x7fbcd0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 -- 172.21.15.16:0/1044317934 shutdown_connections 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 --2- 172.21.15.16:0/1044317934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcdc1062a0 0x7fbcdc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 --2- 172.21.15.16:0/1044317934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc069420 0x7fbcdc069800 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 --2- 172.21.15.16:0/1044317934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcdc069dd0 0x7fbcdc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 -- 172.21.15.16:0/1044317934 >> 172.21.15.16:0/1044317934 conn(0x7fbcdc076ba0 msgr2=0x7fbcdc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 -- 172.21.15.16:0/1044317934 shutdown_connections 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.338+0000 7fbce2db7700 1 -- 172.21.15.16:0/1044317934 wait complete. 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 Processor -- start 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 -- start start 2024-05-08T23:27:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcdc069420 0x7fbcdc1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcdc1062a0 0x7fbcdc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 0x7fbcdc19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc19c610 con 0x7fbcdc19bb90 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc19c780 con 0x7fbcdc1062a0 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce2db7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc19c940 con 0x7fbcdc069420 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 0x7fbcdc19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 0x7fbcdc19c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44032/0 (socket says 172.21.15.16:44032) 2024-05-08T23:27:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 -- 172.21.15.16:0/469059282 learned_addr learned my addr 172.21.15.16:0/469059282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce0b53700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcdc069420 0x7fbcdc1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcdc1062a0 0x7fbcdc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 -- 172.21.15.16:0/469059282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcdc069420 msgr2=0x7fbcdc1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcdc069420 0x7fbcdc1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 -- 172.21.15.16:0/469059282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcdc1062a0 msgr2=0x7fbcdc1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcdc1062a0 0x7fbcdc1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.339+0000 7fbce1354700 1 -- 172.21.15.16:0/469059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd00095f0 con 0x7fbcdc19bb90 2024-05-08T23:27:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.340+0000 7fbce1354700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 0x7fbcdc19c010 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4007ee0 tx=0x7fbcd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.340+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/469059282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400ddd0 con 0x7fbcdc19bb90 2024-05-08T23:27:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.340+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/469059282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd400f040 con 0x7fbcdc19bb90 2024-05-08T23:27:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.340+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/469059282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40047d0 con 0x7fbcdc19bb90 2024-05-08T23:27:24.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.340+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc1a3ea0 con 0x7fbcdc19bb90 2024-05-08T23:27:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.340+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc1a4330 con 0x7fbcdc19bb90 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.342+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/469059282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbcd4003a00 con 0x7fbcdc19bb90 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.343+0000 7fbcd9ffb700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbccc0a1290 0x7fbccc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.343+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/469059282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fbcd40cc930 con 0x7fbcdc19bb90 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.343+0000 7fbce0b53700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbccc0a1290 0x7fbccc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbce0b53700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbccc0a1290 0x7fbccc0a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000b2e0 tx=0x7fbcd0005040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 Processor -- start 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 -- start start 2024-05-08T23:27:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a525150 con 0x56236a5ce890 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a522b10 con 0x56236a5d4720 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.344+0000 7fbceef71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a51e400 con 0x56236a5d3e00 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce0b53700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44038/0 (socket says 172.21.15.16:44038) 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 -- 172.21.15.16:0/557164596 learned_addr learned my addr 172.21.15.16:0/557164596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 -- 172.21.15.16:0/557164596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 msgr2=0x56236a5d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 -- 172.21.15.16:0/557164596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 msgr2=0x56236a5ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 -- 172.21.15.16:0/557164596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236a39f3b0 con 0x56236a5ce890 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbce1354700 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5cecd0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4096110 tx=0x7fbcd4099590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a67a132a995306 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbcc6ffd700 1 -- 172.21.15.16:0/557164596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4082ea0 con 0x56236a5ce890 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbcc6ffd700 1 -- 172.21.15.16:0/557164596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd4099da0 con 0x56236a5ce890 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.345+0000 7fbcc6ffd700 1 -- 172.21.15.16:0/557164596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4003d40 con 0x56236a5ce890 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 -- 172.21.15.16:0/557164596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 msgr2=0x56236a5cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5cecd0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4096110 tx=0x7fbcd4099590 comp rx=0 tx=0).stop 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 -- 172.21.15.16:0/557164596 shutdown_connections 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5cecd0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 --2- 172.21.15.16:0/557164596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 -- 172.21.15.16:0/557164596 >> 172.21.15.16:0/557164596 conn(0x56236a52c210 msgr2=0x56236a5c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 -- 172.21.15.16:0/557164596 shutdown_connections 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 -- 172.21.15.16:0/557164596 wait complete. 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.346+0000 7fbceef71b80 1 -- start start 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 Processor -- start 2024-05-08T23:27:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d3e00 0x56236a5d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d4720 0x56236a5d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a4ba560 con 0x56236a5ce890 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a4ba200 con 0x56236a5d3e00 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a3a16a0 con 0x56236a5d4720 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbceef71b80 10 client.0 _ll_register_callbacks cb 0x56236a4c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbce0b53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbce0b53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5d7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44048/0 (socket says 172.21.15.16:44048) 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.347+0000 7fbce0b53700 1 -- 172.21.15.16:0/1165253820 learned_addr learned my addr 172.21.15.16:0/1165253820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce1354700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d4720 0x56236a5d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d3e00 0x56236a5d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce0b53700 1 -- 172.21.15.16:0/1165253820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d4720 msgr2=0x56236a5d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce0b53700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d4720 0x56236a5d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce0b53700 1 -- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d3e00 msgr2=0x56236a5d8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce0b53700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d3e00 0x56236a5d8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce0b53700 1 -- 172.21.15.16:0/1165253820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc1a4330 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbce0b53700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5d7bd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009990 tx=0x7fbcd0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cca6dd588b8b175 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbcbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbcbbfff700 1 -- 172.21.15.16:0/1165253820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000be80 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56236a477270 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbcbbfff700 1 -- 172.21.15.16:0/1165253820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fbcd0004e50 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbcbbfff700 1 -- 172.21.15.16:0/1165253820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd000f460 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbceef71b80 10 client.17262 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.348+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56236a34bb60 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbcbbfff700 1 -- 172.21.15.16:0/1165253820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56236a477270 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbcbbfff700 1 -- 172.21.15.16:0/1165253820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcd002f070 con 0x56236a5ce890 2024-05-08T23:27:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbceef71b80 10 client.17262 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbceef71b80 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbceef71b80 20 client.17262 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbceef71b80 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a5e83c0 0x56236a5ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbceef71b80 4 client.17262 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56236a59d810 con 0x56236a5e83c0 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.349+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a5e83c0 0x56236a5ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.350+0000 7fbcbbfff700 10 client.17262 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.350+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a5e83c0 0x56236a5ea7e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77d1eff710a6d59 server_cookie=97cc3ec2652a12f9 in_seq=0 out_seq=0 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.350+0000 7fbcbbfff700 1 -- 172.21.15.16:0/1165253820 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56236a59d810 con 0x56236a5e83c0 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.350+0000 7fbcbbfff700 10 client.17262 handle_command_reply: tid=0 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.350+0000 7fbceef71b80 1 client.17262 shutdown 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 msgr2=0x56236a5d7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5d7bd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009990 tx=0x7fbcd0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 shutdown_connections 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d4720 0x56236a5d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5d7bd0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a5e83c0 0x56236a5ea7e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 --2- 172.21.15.16:0/1165253820 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d3e00 0x56236a5d8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbcbbfff700 0 client.17262 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 >> 172.21.15.16:0/1165253820 conn(0x56236a52c210 msgr2=0x56236a5c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 shutdown_connections 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 1 -- 172.21.15.16:0/1165253820 wait complete. 2024-05-08T23:27:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.351+0000 7fbceef71b80 20 client.17262 trim_cache size 0 max 16384 2024-05-08T23:27:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 Processor -- start 2024-05-08T23:27:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 -- start start 2024-05-08T23:27:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5e4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a5ca680 con 0x56236a5ce890 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a5c5df0 con 0x56236a5d4720 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbceef71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a5efa70 con 0x56236a5d3e00 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44058/0 (socket says 172.21.15.16:44058) 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 -- 172.21.15.16:0/595334159 learned_addr learned my addr 172.21.15.16:0/595334159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce1354700 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 -- 172.21.15.16:0/595334159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 msgr2=0x56236a5e4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5e4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 -- 172.21.15.16:0/595334159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 msgr2=0x56236a5e3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5e3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbce0b53700 1 -- 172.21.15.16:0/595334159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236a4e4560 con 0x56236a5ce890 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.365+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5e4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbce0b53700 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5ce350 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005d80 tx=0x7fbcd0005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328426a539fa9375 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbcc77fe700 1 -- 172.21.15.16:0/595334159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000be80 con 0x56236a5ce890 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbcc77fe700 1 -- 172.21.15.16:0/595334159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd0028070 con 0x56236a5ce890 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbcc77fe700 1 -- 172.21.15.16:0/595334159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000fb50 con 0x56236a5ce890 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 -- 172.21.15.16:0/595334159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 msgr2=0x56236a5ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5ce350 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005d80 tx=0x7fbcd0005980 comp rx=0 tx=0).stop 2024-05-08T23:27:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 -- 172.21.15.16:0/595334159 shutdown_connections 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5d4720 0x56236a5e3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5ce890 0x56236a5ce350 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 --2- 172.21.15.16:0/595334159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5e4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 -- 172.21.15.16:0/595334159 >> 172.21.15.16:0/595334159 conn(0x56236a52c210 msgr2=0x56236a5d64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 -- 172.21.15.16:0/595334159 shutdown_connections 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.366+0000 7fbceef71b80 1 -- 172.21.15.16:0/595334159 wait complete. 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 -- start start 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 Processor -- start 2024-05-08T23:27:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5ce890 0x56236a5df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5dd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 0x56236a5dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a5ddd70 con 0x56236a5d4720 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a5ddee0 con 0x56236a5ce890 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a63d8c0 con 0x56236a5d3e00 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbceef71b80 10 client.0 _ll_register_callbacks cb 0x56236a4c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbce1354700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 0x56236a5dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbce1354700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 0x56236a5dd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44064/0 (socket says 172.21.15.16:44064) 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.367+0000 7fbce1354700 1 -- 172.21.15.16:0/3106183982 learned_addr learned my addr 172.21.15.16:0/3106183982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce0b53700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5ce890 0x56236a5df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbcdbfff700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce1354700 1 -- 172.21.15.16:0/3106183982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 msgr2=0x56236a5dd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce1354700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5dd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce1354700 1 -- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5ce890 msgr2=0x56236a5df4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce1354700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5ce890 0x56236a5df4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce1354700 1 -- 172.21.15.16:0/3106183982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236a34bb60 con 0x56236a5d4720 2024-05-08T23:27:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbce1354700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 0x56236a5dd690 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4096110 tx=0x7fbcd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3288fc2634a10c1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbcbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbcbbfff700 1 -- 172.21.15.16:0/3106183982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4082ea0 con 0x56236a5d4720 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbcbbfff700 1 -- 172.21.15.16:0/3106183982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fbcd40a1b20 con 0x56236a5d4720 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcd4099da0 con 0x56236a5d4720 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbcbbfff700 1 -- 172.21.15.16:0/3106183982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd409d4d0 con 0x56236a5d4720 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbceef71b80 10 client.17268 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56236a3a3cd0 con 0x56236a5d4720 2024-05-08T23:27:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.368+0000 7fbcbbfff700 1 -- 172.21.15.16:0/3106183982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcd4065020 con 0x56236a5d4720 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbcbbfff700 1 -- 172.21.15.16:0/3106183982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcd40b4070 con 0x56236a5d4720 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbceef71b80 10 client.17268 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbceef71b80 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbceef71b80 20 client.17268 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbceef71b80 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a63eac0 0x56236a640ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbceef71b80 4 client.17268 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56236a516ec0 con 0x56236a63eac0 2024-05-08T23:27:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.369+0000 7fbce0b53700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a63eac0 0x56236a640ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbcbbfff700 10 client.17268 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbce0b53700 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a63eac0 0x56236a640ee0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b5e206d73e5d07 server_cookie=b88b3fe399f4acbf in_seq=0 out_seq=0 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbcbbfff700 1 -- 172.21.15.16:0/3106183982 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56236a516ec0 con 0x56236a63eac0 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbcbbfff700 10 client.17268 handle_command_reply: tid=0 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 client.17268 shutdown 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 msgr2=0x56236a5dd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 0x56236a5dd690 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4096110 tx=0x7fbcd4004ad0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 shutdown_connections 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a5d4720 0x56236a5dd690 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56236a63eac0 0x56236a640ee0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56236a5ce890 0x56236a5df4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 --2- 172.21.15.16:0/3106183982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56236a5d3e00 0x56236a5dd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbcbbfff700 0 client.17268 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 >> 172.21.15.16:0/3106183982 conn(0x56236a52c210 msgr2=0x56236a5d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 shutdown_connections 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 1 -- 172.21.15.16:0/3106183982 wait complete. 2024-05-08T23:27:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.370+0000 7fbceef71b80 20 client.17268 trim_cache size 0 max 16384 2024-05-08T23:27:24.382 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:27:24.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbccc0a1290 msgr2=0x7fbccc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbccc0a1290 0x7fbccc0a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000b2e0 tx=0x7fbcd0005040 comp rx=0 tx=0).stop 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 msgr2=0x7fbcdc19c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 0x7fbcdc19c010 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4007ee0 tx=0x7fbcd400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 shutdown_connections 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fbccc0a1290 0x7fbccc0a3750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc19bb90 0x7fbcdc19c010 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbcdc069420 0x7fbcdc1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 --2- 172.21.15.16:0/469059282 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbcdc1062a0 0x7fbcdc1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 >> 172.21.15.16:0/469059282 conn(0x7fbcdc076ba0 msgr2=0x7fbcdc107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 shutdown_connections 2024-05-08T23:27:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:24.371+0000 7fbce2db7700 1 -- 172.21.15.16:0/469059282 wait complete. 2024-05-08T23:27:25.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:24 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2775412980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:27:25.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:24 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2775412980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:27:25.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:24 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2775412980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:27:25.385 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:27:25.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:27:25.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 -- 172.21.15.16:0/1464219939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174100050 msgr2=0x7f0174100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 --2- 172.21.15.16:0/1464219939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174100050 0x7f0174100430 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0164009990 tx=0x7f0164009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 -- 172.21.15.16:0/1464219939 shutdown_connections 2024-05-08T23:27:25.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 --2- 172.21.15.16:0/1464219939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0174105170 0x7f017410ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 --2- 172.21.15.16:0/1464219939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174100050 0x7f0174100430 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 --2- 172.21.15.16:0/1464219939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0174100a00 0x7f0174104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 -- 172.21.15.16:0/1464219939 >> 172.21.15.16:0/1464219939 conn(0x7f0174076fa0 msgr2=0x7f01740773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 -- 172.21.15.16:0/1464219939 shutdown_connections 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.534+0000 7f01796ee700 1 -- 172.21.15.16:0/1464219939 wait complete. 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 Processor -- start 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 -- start start 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0174100050 0x7f0174197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 0x7f0174197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017419bc30 0x7f017419c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017419c6b0 con 0x7f0174105170 2024-05-08T23:27:25.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017419c820 con 0x7f0174100050 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f01796ee700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017419c9e0 con 0x7f017419bc30 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f016a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 0x7f0174197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f016a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 0x7f0174197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44070/0 (socket says 172.21.15.16:44070) 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.535+0000 7f016a7fc700 1 -- 172.21.15.16:0/3019584521 learned_addr learned my addr 172.21.15.16:0/3019584521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f01737fe700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017419bc30 0x7f017419c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0174100050 0x7f0174197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f016a7fc700 1 -- 172.21.15.16:0/3019584521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017419bc30 msgr2=0x7f017419c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017419bc30 0x7f017419c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f016a7fc700 1 -- 172.21.15.16:0/3019584521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0174100050 msgr2=0x7f0174197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0174100050 0x7f0174197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f016a7fc700 1 -- 172.21.15.16:0/3019584521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01640095f0 con 0x7f0174105170 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 0x7f0174197890 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c9e0 tx=0x7f015c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f0170ff9700 1 -- 172.21.15.16:0/3019584521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c007aa0 con 0x7f0174105170 2024-05-08T23:27:25.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f0170ff9700 1 -- 172.21.15.16:0/3019584521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015c00ce80 con 0x7f0174105170 2024-05-08T23:27:25.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01741a3f40 con 0x7f0174105170 2024-05-08T23:27:25.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f0170ff9700 1 -- 172.21.15.16:0/3019584521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c018400 con 0x7f0174105170 2024-05-08T23:27:25.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.536+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01741a43c0 con 0x7f0174105170 2024-05-08T23:27:25.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.539+0000 7f0170ff9700 1 -- 172.21.15.16:0/3019584521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f015c00f450 con 0x7f0174105170 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.539+0000 7f0170ff9700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01600a1290 0x7f01600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.539+0000 7f0170ff9700 1 -- 172.21.15.16:0/3019584521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f015c0ccfc0 con 0x7f0174105170 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.539+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01600a1290 0x7f01600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01600a1290 0x7f01600a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f016400b5c0 tx=0x7f0164005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 Processor -- start 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 -- start start 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeae80 0x5645deaeb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deae53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deae5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dea3c1d0 con 0x5645deaeb7a0 2024-05-08T23:27:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dea39b90 con 0x5645deaeae80 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.540+0000 7f01858a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645dea35480 con 0x5645deae5910 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deae53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deae53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44072/0 (socket says 172.21.15.16:44072) 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 -- 172.21.15.16:0/3015853462 learned_addr learned my addr 172.21.15.16:0/3015853462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeae80 0x5645deaeb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f01737fe700 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deae5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 -- 172.21.15.16:0/3015853462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 msgr2=0x5645deae5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deae5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 -- 172.21.15.16:0/3015853462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeae80 msgr2=0x5645deaeb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeae80 0x5645deaeb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 -- 172.21.15.16:0/3015853462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645de8b6430 con 0x5645deaeb7a0 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deae53d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f015c096230 tx=0x7f015c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad2aae3ae587f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f01697fa700 1 -- 172.21.15.16:0/3015853462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c0b1070 con 0x5645deaeb7a0 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f01697fa700 1 -- 172.21.15.16:0/3015853462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015c099d30 con 0x5645deaeb7a0 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.541+0000 7f01697fa700 1 -- 172.21.15.16:0/3015853462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c096440 con 0x5645deaeb7a0 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 -- 172.21.15.16:0/3015853462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 msgr2=0x5645deae53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deae53d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f015c096230 tx=0x7f015c007e80 comp rx=0 tx=0).stop 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 -- 172.21.15.16:0/3015853462 shutdown_connections 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deae5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeae80 0x5645deaeb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3015853462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deae53d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 -- 172.21.15.16:0/3015853462 >> 172.21.15.16:0/3015853462 conn(0x5645dea43290 msgr2=0x5645deadfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 -- 172.21.15.16:0/3015853462 shutdown_connections 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.542+0000 7f01858a8b80 1 -- 172.21.15.16:0/3015853462 wait complete. 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 -- start start 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 Processor -- start 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deaeeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 0x5645deaef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeb7a0 0x5645deaef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645de9d15e0 con 0x5645deaeae80 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645de9d1280 con 0x5645deaeb7a0 2024-05-08T23:27:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645de8b8720 con 0x5645deae5910 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01858a8b80 10 client.0 _ll_register_callbacks cb 0x5645de9dabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f016a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 0x5645deaef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f016a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 0x5645deaef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44082/0 (socket says 172.21.15.16:44082) 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f0172ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deaeeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f016a7fc700 1 -- 172.21.15.16:0/3774913859 learned_addr learned my addr 172.21.15.16:0/3774913859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.543+0000 7f01737fe700 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeb7a0 0x5645deaef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f016a7fc700 1 -- 172.21.15.16:0/3774913859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 msgr2=0x5645deaeeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deaeeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f016a7fc700 1 -- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeb7a0 msgr2=0x5645deaef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeb7a0 0x5645deaef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f016a7fc700 1 -- 172.21.15.16:0/3774913859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01741a43c0 con 0x5645deaeae80 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 0x5645deaef0c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f015c0ca420 tx=0x7f015c0982e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e380423f64fd1eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f015a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f015a7fc700 1 -- 172.21.15.16:0/3774913859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c0b1030 con 0x5645deaeae80 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f015a7fc700 1 -- 172.21.15.16:0/3774913859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f015c0b2070 con 0x5645deaeae80 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645de98e2f0 con 0x5645deaeae80 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f015a7fc700 1 -- 172.21.15.16:0/3774913859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f015c0a21b0 con 0x5645deaeae80 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f01858a8b80 10 client.17280 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:25.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645de862be0 con 0x5645deaeae80 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f015a7fc700 1 -- 172.21.15.16:0/3774913859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5645de98e2f0 con 0x5645deaeae80 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.544+0000 7f015a7fc700 1 -- 172.21.15.16:0/3774913859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f015c0a1d20 con 0x5645deaeae80 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f01858a8b80 10 client.17280 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f01858a8b80 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f01858a8b80 20 client.17280 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deafedf0 0x5645deb011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f01858a8b80 4 client.17280 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645deab4890 con 0x5645deafedf0 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deafedf0 0x5645deb011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deafedf0 0x5645deb011e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5000b8767e9b4e6a server_cookie=6e67572ecb814f9 in_seq=0 out_seq=0 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.545+0000 7f015a7fc700 10 client.17280 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f015a7fc700 1 -- 172.21.15.16:0/3774913859 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645deab4890 con 0x5645deafedf0 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f015a7fc700 10 client.17280 handle_command_reply: tid=0 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 client.17280 shutdown 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 msgr2=0x5645deaef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 0x5645deaef0c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f015c0ca420 tx=0x7f015c0982e0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 shutdown_connections 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deafedf0 0x5645deb011e0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deaeb7a0 0x5645deaef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f015a7fc700 0 client.17280 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deae5910 0x5645deaeeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3774913859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeae80 0x5645deaef0c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 >> 172.21.15.16:0/3774913859 conn(0x5645dea43290 msgr2=0x5645deadf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 shutdown_connections 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 1 -- 172.21.15.16:0/3774913859 wait complete. 2024-05-08T23:27:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.546+0000 7f01858a8b80 20 client.17280 trim_cache size 0 max 16384 2024-05-08T23:27:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.565+0000 7f01858a8b80 1 Processor -- start 2024-05-08T23:27:25.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.565+0000 7f01858a8b80 1 -- start start 2024-05-08T23:27:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deae53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deaea370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deadfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01858a8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645de98e2f0 con 0x5645deaeb7a0 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01858a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645deae0bb0 con 0x5645deae5910 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01858a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645deaf1ed0 con 0x5645deaeae80 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deadfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deadfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44094/0 (socket says 172.21.15.16:44094) 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 -- 172.21.15.16:0/1333431371 learned_addr learned my addr 172.21.15.16:0/1333431371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f016a7fc700 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deaea370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f0172ffd700 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deae53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 -- 172.21.15.16:0/1333431371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 msgr2=0x5645deaea370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deaea370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 -- 172.21.15.16:0/1333431371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 msgr2=0x5645deae53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deae53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 -- 172.21.15.16:0/1333431371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645deadc560 con 0x5645deaeb7a0 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f0172ffd700 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deae53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f01737fe700 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deadfda0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f016c009f20 tx=0x7f016c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4156838a2886374 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.566+0000 7f0169ffb700 1 -- 172.21.15.16:0/1333431371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c007e30 con 0x5645deaeb7a0 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f0169ffb700 1 -- 172.21.15.16:0/1333431371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f016c00f040 con 0x5645deaeb7a0 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f0169ffb700 1 -- 172.21.15.16:0/1333431371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c017410 con 0x5645deaeb7a0 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 -- 172.21.15.16:0/1333431371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 msgr2=0x5645deadfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deadfda0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f016c009f20 tx=0x7f016c009f50 comp rx=0 tx=0).stop 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 -- 172.21.15.16:0/1333431371 shutdown_connections 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deadfda0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deae53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 --2- 172.21.15.16:0/1333431371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deaea370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 -- 172.21.15.16:0/1333431371 >> 172.21.15.16:0/1333431371 conn(0x5645dea43290 msgr2=0x5645deae9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 -- 172.21.15.16:0/1333431371 shutdown_connections 2024-05-08T23:27:25.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.567+0000 7f01858a8b80 1 -- 172.21.15.16:0/1333431371 wait complete. 2024-05-08T23:27:25.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 -- start start 2024-05-08T23:27:25.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 Processor -- start 2024-05-08T23:27:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deb4f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deb4f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deb49520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645deb49b00 con 0x5645deaeb7a0 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645deb49c70 con 0x5645deae5910 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645deb49e30 con 0x5645deaeae80 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01858a8b80 10 client.0 _ll_register_callbacks cb 0x5645de9dabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deb49520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01737fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deb49520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44096/0 (socket says 172.21.15.16:44096) 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.568+0000 7f01737fe700 1 -- 172.21.15.16:0/3001669635 learned_addr learned my addr 172.21.15.16:0/3001669635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deb4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deb4f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01737fe700 1 -- 172.21.15.16:0/3001669635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 msgr2=0x5645deb4f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01737fe700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deb4f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01737fe700 1 -- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 msgr2=0x5645deb4f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01737fe700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deb4f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01737fe700 1 -- 172.21.15.16:0/3001669635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c009bd0 con 0x5645deaeb7a0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f016a7fc700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deb4f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deb4f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01737fe700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deb49520 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f016c006070 tx=0x7f016c005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ddfdff232632df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f015a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f015a7fc700 1 -- 172.21.15.16:0/3001669635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c023070 con 0x5645deaeb7a0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f015a7fc700 1 -- 172.21.15.16:0/3001669635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f016c005330 con 0x5645deaeb7a0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f015c099d30 con 0x5645deaeb7a0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f015a7fc700 1 -- 172.21.15.16:0/3001669635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f016c004ad0 con 0x5645deaeb7a0 2024-05-08T23:27:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01858a8b80 10 client.17286 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645deae5fc0 con 0x5645deaeb7a0 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.569+0000 7f015a7fc700 1 -- 172.21.15.16:0/3001669635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f015c099d30 con 0x5645deaeb7a0 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f015a7fc700 1 -- 172.21.15.16:0/3001669635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f016c01f420 con 0x5645deaeb7a0 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f01858a8b80 10 client.17286 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f01858a8b80 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f01858a8b80 20 client.17286 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deb56e20 0x5645deb59210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f01858a8b80 4 client.17286 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5645dea2df40 con 0x5645deb56e20 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deb56e20 0x5645deb59210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f015a7fc700 10 client.17286 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.570+0000 7f0172ffd700 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deb56e20 0x5645deb59210 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17242b8eb2bf1dba server_cookie=bb0a67c5e41f1e48 in_seq=0 out_seq=0 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.571+0000 7f015a7fc700 1 -- 172.21.15.16:0/3001669635 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5645dea2df40 con 0x5645deb56e20 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.571+0000 7f015a7fc700 10 client.17286 handle_command_reply: tid=0 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.571+0000 7f01858a8b80 1 client.17286 shutdown 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 msgr2=0x5645deb49520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deb49520 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f016c006070 tx=0x7f016c005ac0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 shutdown_connections 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645deaeb7a0 0x5645deb49520 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5645deb56e20 0x5645deb59210 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5645deae5910 0x5645deb4f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 --2- 172.21.15.16:0/3001669635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645deaeae80 0x5645deb4f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f015a7fc700 0 client.17286 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 >> 172.21.15.16:0/3001669635 conn(0x5645dea43290 msgr2=0x5645deaf0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 shutdown_connections 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 1 -- 172.21.15.16:0/3001669635 wait complete. 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01858a8b80 20 client.17286 trim_cache size 0 max 16384 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "005add7e-f9b5-4ce5-8776-4321a2e4acb4", 2024-05-08T23:27:25.580 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:27:25.581 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:27:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.572+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01600a1290 msgr2=0x7f01600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01600a1290 0x7f01600a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f016400b5c0 tx=0x7f0164005fb0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 msgr2=0x7f0174197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 0x7f0174197890 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c9e0 tx=0x7f015c00ccf0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 shutdown_connections 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01600a1290 0x7f01600a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017419bc30 0x7f017419c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0174100050 0x7f0174197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 --2- 172.21.15.16:0/3019584521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0174105170 0x7f0174197890 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 >> 172.21.15.16:0/3019584521 conn(0x7f0174076fa0 msgr2=0x7f01740698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 shutdown_connections 2024-05-08T23:27:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.573+0000 7f01796ee700 1 -- 172.21.15.16:0/3019584521 wait complete. 2024-05-08T23:27:25.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:27:25.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 -- 172.21.15.16:0/2315858497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74100a10 msgr2=0x7f0e74104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 --2- 172.21.15.16:0/2315858497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74100a10 0x7f0e74104a30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e68009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 -- 172.21.15.16:0/2315858497 shutdown_connections 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 --2- 172.21.15.16:0/2315858497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e74105160 0x7f0e7410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 --2- 172.21.15.16:0/2315858497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e74100060 0x7f0e74100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 --2- 172.21.15.16:0/2315858497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74100a10 0x7f0e74104a30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 -- 172.21.15.16:0/2315858497 >> 172.21.15.16:0/2315858497 conn(0x7f0e74076f40 msgr2=0x7f0e74077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 -- 172.21.15.16:0/2315858497 shutdown_connections 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.726+0000 7f0e7be59700 1 -- 172.21.15.16:0/2315858497 wait complete. 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 Processor -- start 2024-05-08T23:27:25.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 -- start start 2024-05-08T23:27:25.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e74100060 0x7f0e74101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e74105160 0x7f0e74102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 0x7f0e741a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e741a5b00 con 0x7f0e74102550 2024-05-08T23:27:25.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e741a5c70 con 0x7f0e74100060 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.727+0000 7f0e7be59700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e741a5e30 con 0x7f0e74105160 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 0x7f0e741a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 0x7f0e741a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44120/0 (socket says 172.21.15.16:44120) 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 -- 172.21.15.16:0/184440229 learned_addr learned my addr 172.21.15.16:0/184440229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e793f4700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e74105160 0x7f0e74102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e74100060 0x7f0e74101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 -- 172.21.15.16:0/184440229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e74105160 msgr2=0x7f0e74102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e74105160 0x7f0e74102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 -- 172.21.15.16:0/184440229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e74100060 msgr2=0x7f0e74101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e74100060 0x7f0e74101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 -- 172.21.15.16:0/184440229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680095f0 con 0x7f0e74102550 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 0x7f0e741a5530 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00ef30 tx=0x7f0e6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e66ffd700 1 -- 172.21.15.16:0/184440229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c018070 con 0x7f0e74102550 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e741a60b0 con 0x7f0e74102550 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e66ffd700 1 -- 172.21.15.16:0/184440229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e6c00f040 con 0x7f0e74102550 2024-05-08T23:27:25.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e66ffd700 1 -- 172.21.15.16:0/184440229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00cbc0 con 0x7f0e74102550 2024-05-08T23:27:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.728+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e741a6540 con 0x7f0e74102550 2024-05-08T23:27:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.731+0000 7f0e66ffd700 1 -- 172.21.15.16:0/184440229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0e6c00cd80 con 0x7f0e74102550 2024-05-08T23:27:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.731+0000 7f0e66ffd700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0e600a1290 0x7f0e600a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.731+0000 7f0e66ffd700 1 -- 172.21.15.16:0/184440229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0e6c0ccb30 con 0x7f0e74102550 2024-05-08T23:27:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.731+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0e600a1290 0x7f0e600a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.732+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0e600a1290 0x7f0e600a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009c60 tx=0x7f0e70009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.732+0000 7f0e88013b80 1 Processor -- start 2024-05-08T23:27:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.732+0000 7f0e88013b80 1 -- start start 2024-05-08T23:27:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d748f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 0x55b1d7489350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7489cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e88013b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d73e0150 con 0x55b1d748f720 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e88013b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d73ddb10 con 0x55b1d7489890 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e88013b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d73d9400 con 0x55b1d748ee00 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 0x55b1d7489350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 0x55b1d7489350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44130/0 (socket says 172.21.15.16:44130) 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 -- 172.21.15.16:0/2093334532 learned_addr learned my addr 172.21.15.16:0/2093334532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d748f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7489cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 -- 172.21.15.16:0/2093334532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 msgr2=0x55b1d748f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d748f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 -- 172.21.15.16:0/2093334532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 msgr2=0x55b1d7489cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7489cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 -- 172.21.15.16:0/2093334532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1d725a3b0 con 0x55b1d748f720 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.733+0000 7f0e793f4700 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 0x55b1d7489350 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e68005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e88f067c69c0d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e5ffff700 1 -- 172.21.15.16:0/2093334532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d070 con 0x55b1d748f720 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e5ffff700 1 -- 172.21.15.16:0/2093334532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e6801e040 con 0x55b1d748f720 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e5ffff700 1 -- 172.21.15.16:0/2093334532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004740 con 0x55b1d748f720 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 -- 172.21.15.16:0/2093334532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 msgr2=0x55b1d7489350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 0x55b1d7489350 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e68005bc0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 -- 172.21.15.16:0/2093334532 shutdown_connections 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7489cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d748f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2093334532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748f720 0x55b1d7489350 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.734+0000 7f0e88013b80 1 -- 172.21.15.16:0/2093334532 >> 172.21.15.16:0/2093334532 conn(0x55b1d73e7210 msgr2=0x55b1d7483df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.735+0000 7f0e88013b80 1 -- 172.21.15.16:0/2093334532 shutdown_connections 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.735+0000 7f0e88013b80 1 -- 172.21.15.16:0/2093334532 wait complete. 2024-05-08T23:27:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.735+0000 7f0e88013b80 1 -- start start 2024-05-08T23:27:25.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.735+0000 7f0e88013b80 1 Processor -- start 2024-05-08T23:27:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.735+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7495750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.735+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d7492ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748f720 0x55b1d74931f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e88013b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d7375560 con 0x55b1d748ee00 2024-05-08T23:27:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e88013b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d7375200 con 0x55b1d7489890 2024-05-08T23:27:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e88013b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d725c6a0 con 0x55b1d748f720 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e88013b80 10 client.0 _ll_register_callbacks cb 0x55b1d737eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d7492ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e79bf5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7495750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d7492ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44142/0 (socket says 172.21.15.16:44142) 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 -- 172.21.15.16:0/482350251 learned_addr learned my addr 172.21.15.16:0/482350251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748f720 0x55b1d74931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 -- 172.21.15.16:0/482350251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748f720 msgr2=0x55b1d74931f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748f720 0x55b1d74931f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 -- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 msgr2=0x55b1d7495750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7495750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 -- 172.21.15.16:0/482350251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7410d550 con 0x55b1d748ee00 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e793f4700 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d7492ab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009350 tx=0x7f0e6800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8364400a47c1f335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/482350251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d030 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/482350251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0e68004500 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.736+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1d7332270 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/482350251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e680046c0 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 10 client.17298 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b1d7206b60 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/482350251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b1d7332270 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/482350251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e6800f870 con 0x55b1d748ee00 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 10 client.17298 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 20 client.17298 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74a2c80 0x55b1d74a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 4 client.17298 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1d7458810 con 0x55b1d74a2c80 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.737+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74a2c80 0x55b1d74a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.738+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74a2c80 0x55b1d74a50a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb5c92e088015f7 server_cookie=c847e57faf3bf348 in_seq=0 out_seq=0 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.738+0000 7f0e5d7fa700 10 client.17298 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.738+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/482350251 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1d7458810 con 0x55b1d74a2c80 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.738+0000 7f0e5d7fa700 10 client.17298 handle_command_reply: tid=0 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 client.17298 shutdown 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 msgr2=0x55b1d7492ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d7492ab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009350 tx=0x7f0e6800bfa0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 shutdown_connections 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748f720 0x55b1d74931f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74a2c80 0x55b1d74a50a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d7489890 0x55b1d7495750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 --2- 172.21.15.16:0/482350251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d7492ab0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e5d7fa700 0 client.17298 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 >> 172.21.15.16:0/482350251 conn(0x55b1d73e7210 msgr2=0x55b1d74835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 shutdown_connections 2024-05-08T23:27:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 1 -- 172.21.15.16:0/482350251 wait complete. 2024-05-08T23:27:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.739+0000 7f0e88013b80 20 client.17298 trim_cache size 0 max 16384 2024-05-08T23:27:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 Processor -- start 2024-05-08T23:27:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 -- start start 2024-05-08T23:27:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 0x55b1d7483d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d74913a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d74918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d7493a30 con 0x55b1d7489890 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d7484e80 con 0x55b1d748f720 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e88013b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d7493f40 con 0x55b1d748ee00 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e79bf5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 0x55b1d7483d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e79bf5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 0x55b1d7483d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44150/0 (socket says 172.21.15.16:44150) 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.757+0000 7f0e79bf5700 1 -- 172.21.15.16:0/2805686331 learned_addr learned my addr 172.21.15.16:0/2805686331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e793f4700 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d74913a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d74918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e79bf5700 1 -- 172.21.15.16:0/2805686331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 msgr2=0x55b1d74913a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d74913a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e79bf5700 1 -- 172.21.15.16:0/2805686331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 msgr2=0x55b1d74918e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d74918e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e79bf5700 1 -- 172.21.15.16:0/2805686331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1d739f560 con 0x55b1d7489890 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 0x55b1d7483d20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x55b1d74a59b0 tx=0x7f0e70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a28b2e0c3582b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e64ff9700 1 -- 172.21.15.16:0/2805686331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70022070 con 0x55b1d7489890 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e64ff9700 1 -- 172.21.15.16:0/2805686331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e70023040 con 0x55b1d7489890 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.758+0000 7f0e64ff9700 1 -- 172.21.15.16:0/2805686331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70029650 con 0x55b1d7489890 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 -- 172.21.15.16:0/2805686331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 msgr2=0x55b1d7483d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 0x55b1d7483d20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x55b1d74a59b0 tx=0x7f0e70014000 comp rx=0 tx=0).stop 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 -- 172.21.15.16:0/2805686331 shutdown_connections 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d74918e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d7489890 0x55b1d7483d20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 --2- 172.21.15.16:0/2805686331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d748ee00 0x55b1d74913a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 -- 172.21.15.16:0/2805686331 >> 172.21.15.16:0/2805686331 conn(0x55b1d73e7210 msgr2=0x55b1d748cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 -- 172.21.15.16:0/2805686331 shutdown_connections 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 -- 172.21.15.16:0/2805686331 wait complete. 2024-05-08T23:27:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 -- start start 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.759+0000 7f0e88013b80 1 Processor -- start 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d7489890 0x55b1d749a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d749a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d748ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d748b490 con 0x55b1d748ee00 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d748b600 con 0x55b1d748f720 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1d748b7c0 con 0x55b1d7489890 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e88013b80 10 client.0 _ll_register_callbacks cb 0x55b1d737eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d749a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d749a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44162/0 (socket says 172.21.15.16:44162) 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 -- 172.21.15.16:0/99196096 learned_addr learned my addr 172.21.15.16:0/99196096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d7489890 0x55b1d749a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d748ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 -- 172.21.15.16:0/99196096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d7489890 msgr2=0x55b1d749a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d7489890 0x55b1d749a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 -- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 msgr2=0x55b1d748ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d748ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 -- 172.21.15.16:0/99196096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e70017070 con 0x55b1d748ee00 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e7a3f6700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d748ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e793f4700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d749a650 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e6800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a10b9e3d7f72d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.760+0000 7f0e5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/99196096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d070 con 0x55b1d748ee00 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/99196096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0e6800b2c0 con 0x55b1d748ee00 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e6801e040 con 0x55b1d748ee00 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/99196096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e6800bc00 con 0x55b1d748ee00 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 10 client.17304 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b1d725ecd0 con 0x55b1d748ee00 2024-05-08T23:27:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/99196096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e68046080 con 0x55b1d748ee00 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/99196096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e68004500 con 0x55b1d748ee00 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 10 client.17304 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 20 client.17304 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74f9f40 0x55b1d74fc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 4 client.17304 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b1d73d1ec0 con 0x55b1d74f9f40 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.761+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74f9f40 0x55b1d74fc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e79bf5700 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74f9f40 0x55b1d74fc360 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a84c5e09ba8924 server_cookie=6c3364b01362e21b in_seq=0 out_seq=0 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e5d7fa700 10 client.17304 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e5d7fa700 1 -- 172.21.15.16:0/99196096 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b1d73d1ec0 con 0x55b1d74f9f40 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e5d7fa700 10 client.17304 handle_command_reply: tid=0 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 client.17304 shutdown 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 msgr2=0x55b1d749a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d749a650 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009990 tx=0x7f0e6800ba70 comp rx=0 tx=0).stop 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 shutdown_connections 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1d74f9f40 0x55b1d74fc360 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1d748f720 0x55b1d748ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1d7489890 0x55b1d749a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e5d7fa700 0 client.17304 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 --2- 172.21.15.16:0/99196096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d748ee00 0x55b1d749a650 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 >> 172.21.15.16:0/99196096 conn(0x55b1d73e7210 msgr2=0x55b1d7492010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 shutdown_connections 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.762+0000 7f0e88013b80 1 -- 172.21.15.16:0/99196096 wait complete. 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e88013b80 20 client.17304 trim_cache size 0 max 16384 2024-05-08T23:27:25.770 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "005add7e-f9b5-4ce5-8776-4321a2e4acb4": { 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "005add7e-f9b5-4ce5-8776-4321a2e4acb4", 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:27:25.771 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0e600a1290 msgr2=0x7f0e600a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0e600a1290 0x7f0e600a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009c60 tx=0x7f0e70009260 comp rx=0 tx=0).stop 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 msgr2=0x7f0e741a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 0x7f0e741a5530 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00ef30 tx=0x7f0e6c00c3a0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 shutdown_connections 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0e600a1290 0x7f0e600a3750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74102550 0x7f0e741a5530 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e74100060 0x7f0e74101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 --2- 172.21.15.16:0/184440229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0e74105160 0x7f0e74102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 >> 172.21.15.16:0/184440229 conn(0x7f0e74076f40 msgr2=0x7f0e740684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 shutdown_connections 2024-05-08T23:27:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:25.763+0000 7f0e7be59700 1 -- 172.21.15.16:0/184440229 wait complete. 2024-05-08T23:27:25.777 INFO:tasks.cephfs.filesystem:scrub status for tag:005add7e-f9b5-4ce5-8776-4321a2e4acb4 - {'path': '/', 'tag': '005add7e-f9b5-4ce5-8776-4321a2e4acb4', 'options': 'recursive,force'} 2024-05-08T23:27:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:25 smithi016 ceph-mon[42541]: pgmap v242: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 17 op/s 2024-05-08T23:27:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:25 smithi189 ceph-mon[46919]: pgmap v242: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 17 op/s 2024-05-08T23:27:26.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:25 smithi155 ceph-mon[47105]: pgmap v242: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 17 op/s 2024-05-08T23:27:27.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:26 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:27:27.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:26 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:27:27.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:26 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:27:27.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:26 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:27:27.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:26 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:27:27.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:26 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:27:27.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:26 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:27:27.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:26 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:27:27.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:26 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:27:27.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:26 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:27:27.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:26 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:27:27.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:26 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:27:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:27 smithi016 ceph-mon[42541]: pgmap v243: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:27 smithi189 ceph-mon[46919]: pgmap v243: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:27 smithi155 ceph-mon[47105]: pgmap v243: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:30.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:29 smithi016 ceph-mon[42541]: pgmap v244: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:30.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:29 smithi189 ceph-mon[46919]: pgmap v244: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:29 smithi155 ceph-mon[47105]: pgmap v244: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:31.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:30 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:27:31.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:30 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:27:31.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:30 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:27:32.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:31 smithi016 ceph-mon[42541]: pgmap v245: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-05-08T23:27:32.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:31 smithi189 ceph-mon[46919]: pgmap v245: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-05-08T23:27:32.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:31 smithi155 ceph-mon[47105]: pgmap v245: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-05-08T23:27:34.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:33 smithi016 ceph-mon[42541]: pgmap v246: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:34.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:33 smithi189 ceph-mon[46919]: pgmap v246: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:33 smithi155 ceph-mon[47105]: pgmap v246: 129 pgs: 129 active+clean; 11 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:36.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:35 smithi016 ceph-mon[42541]: pgmap v247: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-05-08T23:27:36.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:35 smithi189 ceph-mon[46919]: pgmap v247: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-05-08T23:27:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:35 smithi155 ceph-mon[47105]: pgmap v247: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-05-08T23:27:38.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:37 smithi016 ceph-mon[42541]: pgmap v248: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:38.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:37 smithi189 ceph-mon[46919]: pgmap v248: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:38.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:37 smithi155 ceph-mon[47105]: pgmap v248: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:40.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:39 smithi016 ceph-mon[42541]: pgmap v249: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:40.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:39 smithi189 ceph-mon[46919]: pgmap v249: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:40.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:39 smithi155 ceph-mon[47105]: pgmap v249: 129 pgs: 129 active+clean; 11 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 10 op/s 2024-05-08T23:27:42.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:41 smithi016 ceph-mon[42541]: pgmap v250: 129 pgs: 129 active+clean; 11 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-05-08T23:27:42.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:41 smithi189 ceph-mon[46919]: pgmap v250: 129 pgs: 129 active+clean; 11 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-05-08T23:27:42.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:41 smithi155 ceph-mon[47105]: pgmap v250: 129 pgs: 129 active+clean; 11 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-05-08T23:27:44.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:43 smithi016 ceph-mon[42541]: pgmap v251: 129 pgs: 129 active+clean; 11 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:44.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:43 smithi189 ceph-mon[46919]: pgmap v251: 129 pgs: 129 active+clean; 11 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:44.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:43 smithi155 ceph-mon[47105]: pgmap v251: 129 pgs: 129 active+clean; 11 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:46.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:45 smithi016 ceph-mon[42541]: pgmap v252: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-08T23:27:46.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:45 smithi189 ceph-mon[46919]: pgmap v252: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-08T23:27:46.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:45 smithi155 ceph-mon[47105]: pgmap v252: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-08T23:27:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:47 smithi016 ceph-mon[42541]: pgmap v253: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:47 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:27:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:47 smithi189 ceph-mon[46919]: pgmap v253: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:47 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:27:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:47 smithi155 ceph-mon[47105]: pgmap v253: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:47 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:27:49.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:48 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:27:49.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:48 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:27:49.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:48 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:27:49.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:48 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:27:49.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:48 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:27:49.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:48 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:27:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:48 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:27:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:48 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:27:49.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:48 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:27:50.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:49 smithi016 ceph-mon[42541]: pgmap v254: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:50.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:49 smithi189 ceph-mon[46919]: pgmap v254: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:50.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:49 smithi155 ceph-mon[47105]: pgmap v254: 129 pgs: 129 active+clean; 11 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 37 KiB/s wr, 10 op/s 2024-05-08T23:27:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:51 smithi016 ceph-mon[42541]: pgmap v255: 129 pgs: 129 active+clean; 12 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-08T23:27:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:51 smithi189 ceph-mon[46919]: pgmap v255: 129 pgs: 129 active+clean; 12 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-08T23:27:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:51 smithi155 ceph-mon[47105]: pgmap v255: 129 pgs: 129 active+clean; 12 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-05-08T23:27:54.059 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:313/600: open fd = 3 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:314/600: open fd = 3 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.060 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:315/600: open fd = 3 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:316/600: open fd = 3 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:317/600: open fd = 3 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.061 INFO:tasks.workunit.client.0.smithi016.stdout:318/600: open fd = 3 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:319/600: open fd = 3 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:320/600: open fd = 3 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:321/600: open fd = 3 2024-05-08T23:27:54.062 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:322/600: open fd = 3 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:323/600: open fd = 3 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:324/600: open fd = 3 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.063 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:325/600: open fd = 3 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:326/600: open fd = 3 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:327/600: open fd = 3 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.064 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:328/600: open fd = 3 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:329/600: open fd = 3 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:330/600: open fd = 3 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.065 INFO:tasks.workunit.client.0.smithi016.stdout:331/600: open fd = 3 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:332/600: open fd = 3 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:333/600: open fd = 3 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:334/600: open fd = 3 2024-05-08T23:27:54.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:335/600: open fd = 3 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:336/600: open fd = 3 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:337/600: open fd = 3 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:338/600: open fd = 3 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:339/600: open fd = 3 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:340/600: open fd = 3 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:341/600: open fd = 3 2024-05-08T23:27:54.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:342/600: open fd = 3 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:343/600: open fd = 3 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:344/600: open fd = 3 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:345/600: open fd = 3 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:346/600: open fd = 3 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:347/600: open fd = 3 2024-05-08T23:27:54.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:348/600: open fd = 3 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:349/600: open fd = 3 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:350/600: open fd = 3 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:351/600: open fd = 3 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:352/600: open fd = 3 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:353/600: open fd = 3 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:354/600: open fd = 3 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:355/600: open fd = 3 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:356/600: open fd = 3 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.076 INFO:tasks.workunit.client.0.smithi016.stdout:357/600: open fd = 3 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:358/600: open fd = 3 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:359/600: open fd = 3 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:360/600: open fd = 3 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:361/600: open fd = 3 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:362/600: open fd = 3 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:363/600: open fd = 3 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:364/600: open fd = 3 2024-05-08T23:27:54.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:365/600: open fd = 3 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:366/600: open fd = 3 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:367/600: open fd = 3 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:368/600: open fd = 3 2024-05-08T23:27:54.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:369/600: open fd = 3 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:370/600: open fd = 3 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:371/600: open fd = 3 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:372/600: open fd = 3 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:373/600: open fd = 3 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:374/600: open fd = 3 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:27:54.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:27:54.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:53 smithi016 ceph-mon[42541]: pgmap v256: 129 pgs: 129 active+clean; 12 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-05-08T23:27:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:53 smithi189 ceph-mon[46919]: pgmap v256: 129 pgs: 129 active+clean; 12 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-05-08T23:27:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:53 smithi155 ceph-mon[47105]: pgmap v256: 129 pgs: 129 active+clean; 12 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-05-08T23:27:55.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 -- 172.21.15.16:0/1097433884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8100060 msgr2=0x7f25c8100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 --2- 172.21.15.16:0/1097433884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8100060 0x7f25c8100440 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009990 tx=0x7f25c4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 -- 172.21.15.16:0/1097433884 shutdown_connections 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 --2- 172.21.15.16:0/1097433884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8105160 0x7f25c810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 --2- 172.21.15.16:0/1097433884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8100060 0x7f25c8100440 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 --2- 172.21.15.16:0/1097433884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c8100a10 0x7f25c8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 -- 172.21.15.16:0/1097433884 >> 172.21.15.16:0/1097433884 conn(0x7f25c8076f40 msgr2=0x7f25c8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 -- 172.21.15.16:0/1097433884 shutdown_connections 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.926+0000 7f25d066a700 1 -- 172.21.15.16:0/1097433884 wait complete. 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 Processor -- start 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 -- start start 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c8100060 0x7f25c8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 0x7f25c81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8198b30 0x7f25c81a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81a37f0 con 0x7f25c8105160 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81a3960 con 0x7f25c8100060 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25d066a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81a3b20 con 0x7f25c8198b30 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25cdc05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 0x7f25c81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25cdc05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 0x7f25c81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49708/0 (socket says 172.21.15.16:49708) 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25cdc05700 1 -- 172.21.15.16:0/2568193258 learned_addr learned my addr 172.21.15.16:0/2568193258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.927+0000 7f25ce406700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c8100060 0x7f25c8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cec07700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8198b30 0x7f25c81a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cdc05700 1 -- 172.21.15.16:0/2568193258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8198b30 msgr2=0x7f25c81a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cdc05700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8198b30 0x7f25c81a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cdc05700 1 -- 172.21.15.16:0/2568193258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c8100060 msgr2=0x7f25c8198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cdc05700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c8100060 0x7f25c8198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cdc05700 1 -- 172.21.15.16:0/2568193258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40095f0 con 0x7f25c8105160 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cec07700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8198b30 0x7f25c81a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25cdc05700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 0x7f25c81985a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f25b800d980 tx=0x7f25b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25bf7fe700 1 -- 172.21.15.16:0/2568193258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8009930 con 0x7f25c8105160 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25bf7fe700 1 -- 172.21.15.16:0/2568193258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25b8010460 con 0x7f25c8105160 2024-05-08T23:27:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c81a3e00 con 0x7f25c8105160 2024-05-08T23:27:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25bf7fe700 1 -- 172.21.15.16:0/2568193258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800f620 con 0x7f25c8105160 2024-05-08T23:27:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.928+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c81a4280 con 0x7f25c8105160 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.930+0000 7f25bf7fe700 1 -- 172.21.15.16:0/2568193258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f25b80105d0 con 0x7f25c8105160 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.931+0000 7f25bf7fe700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25b40a1290 0x7f25b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.931+0000 7f25ce406700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25b40a1290 0x7f25b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.931+0000 7f25bf7fe700 1 -- 172.21.15.16:0/2568193258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f25b80ccb00 con 0x7f25c8105160 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.932+0000 7f25ce406700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25b40a1290 0x7f25b40a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25c400b5c0 tx=0x7f25c4005960 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.932+0000 7f25dc824b80 1 Processor -- start 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.932+0000 7f25dc824b80 1 -- start start 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f0810d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08107350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25dc824b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f0805e150 con 0x556f0810ce00 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25dc824b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f0805bb10 con 0x556f08107890 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25dc824b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f08057400 con 0x556f0810d720 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f0810d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f0810d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49714/0 (socket says 172.21.15.16:49714) 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 -- 172.21.15.16:0/1948819522 learned_addr learned my addr 172.21.15.16:0/1948819522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25cec07700 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 -- 172.21.15.16:0/1948819522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 msgr2=0x556f08107350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08107350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 -- 172.21.15.16:0/1948819522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 msgr2=0x556f08107cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 -- 172.21.15.16:0/1948819522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f07ed83b0 con 0x556f0810ce00 2024-05-08T23:27:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.933+0000 7f25ce406700 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f0810d1e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f25c4015000 tx=0x7f25c4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af643766fb5479df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25bcff9700 1 -- 172.21.15.16:0/1948819522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400bec0 con 0x556f0810ce00 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25bcff9700 1 -- 172.21.15.16:0/1948819522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25c401e040 con 0x556f0810ce00 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25bcff9700 1 -- 172.21.15.16:0/1948819522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400f6c0 con 0x556f0810ce00 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 -- 172.21.15.16:0/1948819522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 msgr2=0x556f0810d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f0810d1e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f25c4015000 tx=0x7f25c4020000 comp rx=0 tx=0).stop 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 -- 172.21.15.16:0/1948819522 shutdown_connections 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f0810d1e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1948819522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 -- 172.21.15.16:0/1948819522 >> 172.21.15.16:0/1948819522 conn(0x556f08065210 msgr2=0x556f08101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.934+0000 7f25dc824b80 1 -- 172.21.15.16:0/1948819522 shutdown_connections 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 -- 172.21.15.16:0/1948819522 wait complete. 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 -- start start 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 Processor -- start 2024-05-08T23:27:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 0x556f081136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f0810ce00 0x556f081109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f081110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07ff3560 con 0x556f08107890 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07ff3200 con 0x556f0810ce00 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f07eda6a0 con 0x556f0810d720 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25dc824b80 10 client.0 _ll_register_callbacks cb 0x556f07ffcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25ce406700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 0x556f081136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25ce406700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 0x556f081136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49726/0 (socket says 172.21.15.16:49726) 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25ce406700 1 -- 172.21.15.16:0/1093331392 learned_addr learned my addr 172.21.15.16:0/1093331392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.935+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f0810ce00 0x556f081109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25cec07700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f081110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25ce406700 1 -- 172.21.15.16:0/1093331392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 msgr2=0x556f081110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25ce406700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f081110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25ce406700 1 -- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f0810ce00 msgr2=0x556f081109b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25ce406700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f0810ce00 0x556f081109b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25ce406700 1 -- 172.21.15.16:0/1093331392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c810d550 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25ce406700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 0x556f081136c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009990 tx=0x7f25c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4465ba30d3aff046 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25b1ffb700 1 -- 172.21.15.16:0/1093331392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4004de0 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25b1ffb700 1 -- 172.21.15.16:0/1093331392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f25c400f460 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f07fb0270 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25b1ffb700 1 -- 172.21.15.16:0/1093331392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25c400f620 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25dc824b80 10 client.17316 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f07e84b60 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.936+0000 7f25b1ffb700 1 -- 172.21.15.16:0/1093331392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556f07fb0270 con 0x556f08107890 2024-05-08T23:27:55.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25b1ffb700 1 -- 172.21.15.16:0/1093331392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f25c4041080 con 0x556f08107890 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25dc824b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25dc824b80 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25dc824b80 20 client.17316 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f08125000 0x556f081273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25dc824b80 4 client.17316 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f080d6810 con 0x556f08125000 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.937+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f08125000 0x556f081273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f08125000 0x556f081273f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c70d3b474fae1a server_cookie=bc93986609a90e1b in_seq=0 out_seq=0 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25b1ffb700 10 client.17316 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25b1ffb700 1 -- 172.21.15.16:0/1093331392 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f080d6810 con 0x556f08125000 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25b1ffb700 10 client.17316 handle_command_reply: tid=0 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 client.17316 shutdown 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 msgr2=0x556f081136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 0x556f081136c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009990 tx=0x7f25c4004730 comp rx=0 tx=0).stop 2024-05-08T23:27:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 shutdown_connections 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f081110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f08107890 0x556f081136c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f08125000 0x556f081273f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1093331392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f0810ce00 0x556f081109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25b1ffb700 0 client.17316 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.938+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 >> 172.21.15.16:0/1093331392 conn(0x556f08065210 msgr2=0x556f081015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.939+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 shutdown_connections 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.939+0000 7f25dc824b80 1 -- 172.21.15.16:0/1093331392 wait complete. 2024-05-08T23:27:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.939+0000 7f25dc824b80 20 client.17316 trim_cache size 0 max 16384 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 Processor -- start 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 -- start start 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08101d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f0810f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f080fedc0 con 0x556f0810ce00 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f080fca90 con 0x556f08107890 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25dc824b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f08111930 con 0x556f0810d720 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25cdc05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25cdc05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49742/0 (socket says 172.21.15.16:49742) 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.957+0000 7f25cdc05700 1 -- 172.21.15.16:0/1207896826 learned_addr learned my addr 172.21.15.16:0/1207896826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25ce406700 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cec07700 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f0810f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cdc05700 1 -- 172.21.15.16:0/1207896826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 msgr2=0x556f0810f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f0810f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cdc05700 1 -- 172.21.15.16:0/1207896826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 msgr2=0x556f08107350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cdc05700 1 -- 172.21.15.16:0/1207896826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f08107f40 con 0x556f0810ce00 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25cdc05700 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08101d20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f25b8095d70 tx=0x7f25b80b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad361a62c132d45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25bd7fa700 1 -- 172.21.15.16:0/1207896826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b2260 con 0x556f0810ce00 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25bd7fa700 1 -- 172.21.15.16:0/1207896826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25b809d6a0 con 0x556f0810ce00 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.958+0000 7f25bd7fa700 1 -- 172.21.15.16:0/1207896826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8099ad0 con 0x556f0810ce00 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- 172.21.15.16:0/1207896826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 msgr2=0x556f08101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08101d20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f25b8095d70 tx=0x7f25b80b9040 comp rx=0 tx=0).stop 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- 172.21.15.16:0/1207896826 shutdown_connections 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f0810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f08107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- 172.21.15.16:0/1207896826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08101d20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- 172.21.15.16:0/1207896826 >> 172.21.15.16:0/1207896826 conn(0x556f08065210 msgr2=0x556f081132d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- 172.21.15.16:0/1207896826 shutdown_connections 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- 172.21.15.16:0/1207896826 wait complete. 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- start start 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 Processor -- start 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f0811c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08108dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08109310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f0811ceb0 con 0x556f0810ce00 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f08109850 con 0x556f08107890 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f08109a10 con 0x556f0810d720 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.959+0000 7f25dc824b80 10 client.0 _ll_register_callbacks cb 0x556f07ffcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08108dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08108dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49746/0 (socket says 172.21.15.16:49746) 2024-05-08T23:27:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 -- 172.21.15.16:0/2407913340 learned_addr learned my addr 172.21.15.16:0/2407913340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25ce406700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f0811c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cec07700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 -- 172.21.15.16:0/2407913340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 msgr2=0x556f08109310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08109310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 -- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 msgr2=0x556f0811c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f0811c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 -- 172.21.15.16:0/2407913340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c81a4280 con 0x556f0810ce00 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25cdc05700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08108dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f25b80997b0 tx=0x7f25b80999a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1b51ac78b142ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25b1ffb700 1 -- 172.21.15.16:0/2407913340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b92b0 con 0x556f0810ce00 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25b1ffb700 1 -- 172.21.15.16:0/2407913340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f25b8098d60 con 0x556f0810ce00 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25c401e040 con 0x556f0810ce00 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25b1ffb700 1 -- 172.21.15.16:0/2407913340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25b8098f40 con 0x556f0810ce00 2024-05-08T23:27:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25dc824b80 10 client.17322 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.960+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f0801d560 con 0x556f0810ce00 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25b1ffb700 1 -- 172.21.15.16:0/2407913340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25c401e040 con 0x556f0810ce00 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25b1ffb700 1 -- 172.21.15.16:0/2407913340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f25b8095660 con 0x556f0810ce00 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25dc824b80 10 client.17322 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25dc824b80 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25dc824b80 20 client.17322 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25dc824b80 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f0817c530 0x556f0817e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25dc824b80 4 client.17322 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f0804fec0 con 0x556f0817c530 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25ce406700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f0817c530 0x556f0817e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25b1ffb700 10 client.17322 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.961+0000 7f25ce406700 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f0817c530 0x556f0817e950 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b734cc2cf9c0d51f server_cookie=45c6b5d25a7c52df in_seq=0 out_seq=0 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25b1ffb700 1 -- 172.21.15.16:0/2407913340 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556f0804fec0 con 0x556f0817c530 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25b1ffb700 10 client.17322 handle_command_reply: tid=0 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 client.17322 shutdown 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 msgr2=0x556f08108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08108dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f25b80997b0 tx=0x7f25b80999a0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 shutdown_connections 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f0810d720 0x556f08109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x556f0817c530 0x556f0817e950 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x556f08107890 0x556f0811c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25b1ffb700 0 client.17322 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 --2- 172.21.15.16:0/2407913340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0810ce00 0x556f08108dd0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 >> 172.21.15.16:0/2407913340 conn(0x556f08065210 msgr2=0x556f0810baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 shutdown_connections 2024-05-08T23:27:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 1 -- 172.21.15.16:0/2407913340 wait complete. 2024-05-08T23:27:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.962+0000 7f25dc824b80 20 client.17322 trim_cache size 0 max 16384 2024-05-08T23:27:55.970 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:27:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:27:55.970 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:27:55.970 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25b40a1290 msgr2=0x7f25b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25b40a1290 0x7f25b40a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f25c400b5c0 tx=0x7f25c4005960 comp rx=0 tx=0).stop 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 msgr2=0x7f25c81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 0x7f25c81985a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f25b800d980 tx=0x7f25b800dc90 comp rx=0 tx=0).stop 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 shutdown_connections 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25b40a1290 0x7f25b40a3750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c8198b30 0x7f25c81a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c8100060 0x7f25c8198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 --2- 172.21.15.16:0/2568193258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8105160 0x7f25c81985a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 >> 172.21.15.16:0/2568193258 conn(0x7f25c8076f40 msgr2=0x7f25c80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 shutdown_connections 2024-05-08T23:27:55.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:55.963+0000 7f25d066a700 1 -- 172.21.15.16:0/2568193258 wait complete. 2024-05-08T23:27:55.975 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:27:55.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:27:56.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:55 smithi016 ceph-mon[42541]: pgmap v257: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-05-08T23:27:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 -- 172.21.15.16:0/1701791716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c681008f0 msgr2=0x7f2c68104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1701791716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c681008f0 0x7f2c68104930 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009990 tx=0x7f2c64009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 -- 172.21.15.16:0/1701791716 shutdown_connections 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1701791716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c68105060 0x7f2c6810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1701791716 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c680fff40 0x7f2c68100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1701791716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c681008f0 0x7f2c68104930 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 -- 172.21.15.16:0/1701791716 >> 172.21.15.16:0/1701791716 conn(0x7f2c680fb7e0 msgr2=0x7f2c680fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 -- 172.21.15.16:0/1701791716 shutdown_connections 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.114+0000 7f2c6f993700 1 -- 172.21.15.16:0/1701791716 wait complete. 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 Processor -- start 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 -- start start 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 0x7f2c681019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c68105060 0x7f2c68101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c68102450 0x7f2c681a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a5970 con 0x7f2c680fff40 2024-05-08T23:27:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a5ae0 con 0x7f2c68105060 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6f993700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a5ca0 con 0x7f2c68102450 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.115+0000 7f2c6d72f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 0x7f2c681019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 0x7f2c681019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49764/0 (socket says 172.21.15.16:49764) 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 -- 172.21.15.16:0/1214101228 learned_addr learned my addr 172.21.15.16:0/1214101228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6df30700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c68102450 0x7f2c681a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6cf2e700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c68105060 0x7f2c68101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 -- 172.21.15.16:0/1214101228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c68102450 msgr2=0x7f2c681a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c68102450 0x7f2c681a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 -- 172.21.15.16:0/1214101228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c68105060 msgr2=0x7f2c68101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c68105060 0x7f2c68101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 -- 172.21.15.16:0/1214101228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640095f0 con 0x7f2c680fff40 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6d72f700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 0x7f2c681019d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e9a0 tx=0x7f2c5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58009790 con 0x7f2c680fff40 2024-05-08T23:27:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c5800cbd0 con 0x7f2c680fff40 2024-05-08T23:27:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681a5f80 con 0x7f2c680fff40 2024-05-08T23:27:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58018680 con 0x7f2c680fff40 2024-05-08T23:27:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.116+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a6400 con 0x7f2c680fff40 2024-05-08T23:27:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.117+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68109340 con 0x7f2c680fff40 2024-05-08T23:27:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.118+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2c5800cd40 con 0x7f2c680fff40 2024-05-08T23:27:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.119+0000 7f2c5e7fc700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2c540a1290 0x7f2c540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.119+0000 7f2c6cf2e700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2c540a1290 0x7f2c540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.119+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f2c58014070 con 0x7f2c680fff40 2024-05-08T23:27:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.120+0000 7f2c6cf2e700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2c540a1290 0x7f2c540a3750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c68103100 tx=0x7f2c64005940 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.123+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2c58096740 con 0x7f2c680fff40 2024-05-08T23:27:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.390+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c68102f50 con 0x7f2c680fff40 2024-05-08T23:27:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.391+0000 7f2c5e7fc700 1 -- 172.21.15.16:0/1214101228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f2c58099d60 con 0x7f2c680fff40 2024-05-08T23:27:56.391 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:27:56.391 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:27:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2c540a1290 msgr2=0x7f2c540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2c540a1290 0x7f2c540a3750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c68103100 tx=0x7f2c64005940 comp rx=0 tx=0).stop 2024-05-08T23:27:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 msgr2=0x7f2c681019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 0x7f2c681019d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e9a0 tx=0x7f2c5800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 shutdown_connections 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f2c540a1290 0x7f2c540a3750 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c68102450 0x7f2c681a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c680fff40 0x7f2c681019d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 --2- 172.21.15.16:0/1214101228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c68105060 0x7f2c68101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 >> 172.21.15.16:0/1214101228 conn(0x7f2c680fb7e0 msgr2=0x7f2c68068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 shutdown_connections 2024-05-08T23:27:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.393+0000 7f2c6f993700 1 -- 172.21.15.16:0/1214101228 wait complete. 2024-05-08T23:27:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:27:56.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:27:56.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:55 smithi189 ceph-mon[46919]: pgmap v257: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-05-08T23:27:56.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:55 smithi155 ceph-mon[47105]: pgmap v257: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-05-08T23:27:56.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 -- 172.21.15.16:0/2649291806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280ffdd0 msgr2=0x7f6a2810dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/2649291806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280ffdd0 0x7f6a2810dda0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b0a0 tx=0x7f6a2000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 -- 172.21.15.16:0/2649291806 shutdown_connections 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/2649291806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280ffdd0 0x7f6a2810dda0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/2649291806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a28105ef0 0x7f6a281062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/2649291806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a280ff410 0x7f6a280ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 -- 172.21.15.16:0/2649291806 >> 172.21.15.16:0/2649291806 conn(0x7f6a280763a0 msgr2=0x7f6a280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 -- 172.21.15.16:0/2649291806 shutdown_connections 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.547+0000 7f6a2e16d700 1 -- 172.21.15.16:0/2649291806 wait complete. 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 Processor -- start 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 -- start start 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a280ff410 0x7f6a28197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a280ffdd0 0x7f6a28197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 0x7f6a2819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2819c340 con 0x7f6a28105ef0 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2819c4b0 con 0x7f6a280ff410 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a2e16d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2819c670 con 0x7f6a280ffdd0 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 0x7f6a2819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 0x7f6a2819bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54178/0 (socket says 172.21.15.16:54178) 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.548+0000 7f6a27fff700 1 -- 172.21.15.16:0/798980328 learned_addr learned my addr 172.21.15.16:0/798980328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a280ffdd0 0x7f6a28197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a277fe700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a280ff410 0x7f6a28197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a27fff700 1 -- 172.21.15.16:0/798980328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a280ffdd0 msgr2=0x7f6a28197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a27fff700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a280ffdd0 0x7f6a28197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a27fff700 1 -- 172.21.15.16:0/798980328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a280ff410 msgr2=0x7f6a28197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a27fff700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a280ff410 0x7f6a28197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a27fff700 1 -- 172.21.15.16:0/798980328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a20009d00 con 0x7f6a28105ef0 2024-05-08T23:27:56.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a27fff700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 0x7f6a2819bd00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b070 tx=0x7f6a20007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a24ff9700 1 -- 172.21.15.16:0/798980328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000e040 con 0x7f6a28105ef0 2024-05-08T23:27:56.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a24ff9700 1 -- 172.21.15.16:0/798980328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a200094c0 con 0x7f6a28105ef0 2024-05-08T23:27:56.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2819c8f0 con 0x7f6a28105ef0 2024-05-08T23:27:56.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a24ff9700 1 -- 172.21.15.16:0/798980328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20005040 con 0x7f6a28105ef0 2024-05-08T23:27:56.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.549+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a281a4190 con 0x7f6a28105ef0 2024-05-08T23:27:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.552+0000 7f6a24ff9700 1 -- 172.21.15.16:0/798980328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6a2001d480 con 0x7f6a28105ef0 2024-05-08T23:27:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.552+0000 7f6a24ff9700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6a100a1290 0x7f6a100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.552+0000 7f6a24ff9700 1 -- 172.21.15.16:0/798980328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6a200cdf50 con 0x7f6a28105ef0 2024-05-08T23:27:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.552+0000 7f6a277fe700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6a100a1290 0x7f6a100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.553+0000 7f6a277fe700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6a100a1290 0x7f6a100a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009c60 tx=0x7f6a18009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.553+0000 7f6a3a327b80 1 Processor -- start 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.553+0000 7f6a3a327b80 1 -- start start 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b8992431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b89923d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 0x55b89923dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899194150 con 0x55b89923d890 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899191b10 con 0x55b899243720 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89918d400 con 0x55b899242e00 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 0x55b89923dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 0x55b89923dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54194/0 (socket says 172.21.15.16:54194) 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 -- 172.21.15.16:0/4047812341 learned_addr learned my addr 172.21.15.16:0/4047812341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a277fe700 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b8992431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b89923d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 -- 172.21.15.16:0/4047812341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 msgr2=0x55b8992431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b8992431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 -- 172.21.15.16:0/4047812341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 msgr2=0x55b89923d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b89923d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 -- 172.21.15.16:0/4047812341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b89900e3b0 con 0x55b89923d890 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a27fff700 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 0x55b89923dcd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6a200977c0 tx=0x7f6a2000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe1612b63e2cb1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.554+0000 7f6a0dffb700 1 -- 172.21.15.16:0/4047812341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20003e10 con 0x55b89923d890 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a0dffb700 1 -- 172.21.15.16:0/4047812341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a2009b450 con 0x55b89923d890 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a0dffb700 1 -- 172.21.15.16:0/4047812341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a200cb580 con 0x55b89923d890 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 -- 172.21.15.16:0/4047812341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 msgr2=0x55b89923dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 0x55b89923dcd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6a200977c0 tx=0x7f6a2000bf20 comp rx=0 tx=0).stop 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 -- 172.21.15.16:0/4047812341 shutdown_connections 2024-05-08T23:27:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b89923d890 0x55b89923dcd0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b8992431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/4047812341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b89923d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 -- 172.21.15.16:0/4047812341 >> 172.21.15.16:0/4047812341 conn(0x55b89919b210 msgr2=0x55b899237df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 -- 172.21.15.16:0/4047812341 shutdown_connections 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.555+0000 7f6a3a327b80 1 -- 172.21.15.16:0/4047812341 wait complete. 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 -- start start 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 Processor -- start 2024-05-08T23:27:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b899246a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 0x55b899247170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899129560 con 0x55b899243720 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899129200 con 0x55b89923d890 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8990106a0 con 0x55b899242e00 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.556+0000 7f6a3a327b80 10 client.0 _ll_register_callbacks cb 0x55b899132b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 0x55b899247170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 0x55b899247170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54210/0 (socket says 172.21.15.16:54210) 2024-05-08T23:27:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 -- 172.21.15.16:0/1706558371 learned_addr learned my addr 172.21.15.16:0/1706558371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a277fe700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b899246a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 -- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 msgr2=0x55b899246a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b899246a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 -- 172.21.15.16:0/1706558371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 msgr2=0x55b8992497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 -- 172.21.15.16:0/1706558371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a281a4190 con 0x55b899243720 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b899246a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a27fff700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 0x55b899247170 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6a2009ace0 tx=0x7f6a200099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b0824981b3fdeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a06ffd700 1 -- 172.21.15.16:0/1706558371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2009ad50 con 0x55b899243720 2024-05-08T23:27:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a06ffd700 1 -- 172.21.15.16:0/1706558371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6a2009eb80 con 0x55b899243720 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8990e6270 con 0x55b899243720 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a06ffd700 1 -- 172.21.15.16:0/1706558371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a200a3140 con 0x55b899243720 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a3a327b80 10 client.17340 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.557+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b898fbab60 con 0x55b899243720 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a06ffd700 1 -- 172.21.15.16:0/1706558371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8990e6270 con 0x55b899243720 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a06ffd700 1 -- 172.21.15.16:0/1706558371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a2009a2b0 con 0x55b899243720 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a3a327b80 10 client.17340 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a3a327b80 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a3a327b80 20 client.17340 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b899256ad0 0x55b899258ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a3a327b80 4 client.17340 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b89920c810 con 0x55b899256ad0 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.558+0000 7f6a277fe700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b899256ad0 0x55b899258ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.559+0000 7f6a06ffd700 10 client.17340 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.559+0000 7f6a277fe700 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b899256ad0 0x55b899258ef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fe7b388c9db702b server_cookie=9257a0242449180f in_seq=0 out_seq=0 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.559+0000 7f6a06ffd700 1 -- 172.21.15.16:0/1706558371 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b89920c810 con 0x55b899256ad0 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.559+0000 7f6a06ffd700 10 client.17340 handle_command_reply: tid=0 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.559+0000 7f6a3a327b80 1 client.17340 shutdown 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 msgr2=0x55b899247170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 0x55b899247170 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6a2009ace0 tx=0x7f6a200099e0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 shutdown_connections 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899243720 0x55b899247170 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b899256ad0 0x55b899258ef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/1706558371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899242e00 0x55b899246a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a06ffd700 0 client.17340 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 >> 172.21.15.16:0/1706558371 conn(0x55b89919b210 msgr2=0x55b8992375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 shutdown_connections 2024-05-08T23:27:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 1 -- 172.21.15.16:0/1706558371 wait complete. 2024-05-08T23:27:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.560+0000 7f6a3a327b80 20 client.17340 trim_cache size 0 max 16384 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 Processor -- start 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 -- start start 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b899234210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899243720 0x55b899237d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899238b30 con 0x55b899242e00 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899239a50 con 0x55b89923d890 2024-05-08T23:27:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b899241530 con 0x55b899243720 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b899234210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b899234210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54214/0 (socket says 172.21.15.16:54214) 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 -- 172.21.15.16:0/398597747 learned_addr learned my addr 172.21.15.16:0/398597747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a277fe700 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a27fff700 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899243720 0x55b899237d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 -- 172.21.15.16:0/398597747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899243720 msgr2=0x55b899237d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899243720 0x55b899237d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 -- 172.21.15.16:0/398597747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 msgr2=0x55b8992469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.574+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a26ffd700 1 -- 172.21.15.16:0/398597747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b899153560 con 0x55b899242e00 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a27fff700 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899243720 0x55b899237d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b899234210 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004630 tx=0x7f6a1c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de8558a68eff00c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a0e7fc700 1 -- 172.21.15.16:0/398597747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c017070 con 0x55b899242e00 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a0e7fc700 1 -- 172.21.15.16:0/398597747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a1c004cc0 con 0x55b899242e00 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a0e7fc700 1 -- 172.21.15.16:0/398597747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0038e0 con 0x55b899242e00 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 -- 172.21.15.16:0/398597747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 msgr2=0x55b899234210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b899234210 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c004630 tx=0x7f6a1c00b9f0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 -- 172.21.15.16:0/398597747 shutdown_connections 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b899243720 0x55b899237d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b89923d890 0x55b8992469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/398597747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b899234210 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.575+0000 7f6a3a327b80 1 -- 172.21.15.16:0/398597747 >> 172.21.15.16:0/398597747 conn(0x55b89919b210 msgr2=0x55b8992462c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 -- 172.21.15.16:0/398597747 shutdown_connections 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 -- 172.21.15.16:0/398597747 wait complete. 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 -- start start 2024-05-08T23:27:56.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 Processor -- start 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b89923d890 0x55b899249f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b89924a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b8992b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b89924acd0 con 0x55b899242e00 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8992b0910 con 0x55b899243720 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8992b0a50 con 0x55b89923d890 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.576+0000 7f6a3a327b80 10 client.0 _ll_register_callbacks cb 0x55b899132b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b89924a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b89924a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54224/0 (socket says 172.21.15.16:54224) 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 -- 172.21.15.16:0/2727567791 learned_addr learned my addr 172.21.15.16:0/2727567791 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a277fe700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b89923d890 0x55b899249f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a27fff700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b8992b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 -- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b89923d890 msgr2=0x55b899249f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b89923d890 0x55b899249f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 -- 172.21.15.16:0/2727567791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 msgr2=0x55b8992b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b8992b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 -- 172.21.15.16:0/2727567791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c004310 con 0x55b899242e00 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a27fff700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b8992b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a26ffd700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b89924a480 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01b000 tx=0x7f6a1c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b6798710efe108 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a06ffd700 1 -- 172.21.15.16:0/2727567791 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c017030 con 0x55b899242e00 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a06ffd700 1 -- 172.21.15.16:0/2727567791 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6a1c018070 con 0x55b899242e00 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a2009b450 con 0x55b899242e00 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a06ffd700 1 -- 172.21.15.16:0/2727567791 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1c021070 con 0x55b899242e00 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a3a327b80 10 client.17346 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.577+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b899012cd0 con 0x55b899242e00 2024-05-08T23:27:56.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a06ffd700 1 -- 172.21.15.16:0/2727567791 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a2009b450 con 0x55b899242e00 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a06ffd700 1 -- 172.21.15.16:0/2727567791 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a1c026470 con 0x55b899242e00 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a3a327b80 10 client.17346 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a3a327b80 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a3a327b80 20 client.17346 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b8992b2090 0x55b8992b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a3a327b80 4 client.17346 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b899185ec0 con 0x55b8992b2090 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a277fe700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b8992b2090 0x55b8992b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a06ffd700 10 client.17346 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.578+0000 7f6a277fe700 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b8992b2090 0x55b8992b44b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9413f49783f09dba server_cookie=26b1827d85169bc4 in_seq=0 out_seq=0 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a06ffd700 1 -- 172.21.15.16:0/2727567791 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b899185ec0 con 0x55b8992b2090 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a06ffd700 10 client.17346 handle_command_reply: tid=0 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 client.17346 shutdown 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 msgr2=0x55b89924a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b89924a480 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01b000 tx=0x7f6a1c0056d0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 shutdown_connections 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55b8992b2090 0x55b8992b44b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b899243720 0x55b8992b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b89923d890 0x55b899249f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a06ffd700 0 client.17346 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 --2- 172.21.15.16:0/2727567791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b899242e00 0x55b89924a480 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 >> 172.21.15.16:0/2727567791 conn(0x55b89919b210 msgr2=0x55b8992461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 shutdown_connections 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 1 -- 172.21.15.16:0/2727567791 wait complete. 2024-05-08T23:27:56.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.579+0000 7f6a3a327b80 20 client.17346 trim_cache size 0 max 16384 2024-05-08T23:27:56.587 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:27:56.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6a100a1290 msgr2=0x7f6a100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6a100a1290 0x7f6a100a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009c60 tx=0x7f6a18009260 comp rx=0 tx=0).stop 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 msgr2=0x7f6a2819bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 0x7f6a2819bd00 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b070 tx=0x7f6a20007cc0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 shutdown_connections 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6a100a1290 0x7f6a100a3750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a28105ef0 0x7f6a2819bd00 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a280ff410 0x7f6a28197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 --2- 172.21.15.16:0/798980328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a280ffdd0 0x7f6a28197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 >> 172.21.15.16:0/798980328 conn(0x7f6a280763a0 msgr2=0x7f6a281040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 shutdown_connections 2024-05-08T23:27:56.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.580+0000 7f6a2e16d700 1 -- 172.21.15.16:0/798980328 wait complete. 2024-05-08T23:27:56.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:27:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 -- 172.21.15.16:0/435302167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c100ab0 msgr2=0x7fe38c10ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 --2- 172.21.15.16:0/435302167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c100ab0 0x7fe38c10ccc0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe38400b0d0 tx=0x7fe38400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 -- 172.21.15.16:0/435302167 shutdown_connections 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 --2- 172.21.15.16:0/435302167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c100ab0 0x7fe38c10ccc0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 --2- 172.21.15.16:0/435302167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c105f90 0x7fe38c106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 --2- 172.21.15.16:0/435302167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe38c1000f0 0x7fe38c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 -- 172.21.15.16:0/435302167 >> 172.21.15.16:0/435302167 conn(0x7fe38c076ba0 msgr2=0x7fe38c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 -- 172.21.15.16:0/435302167 shutdown_connections 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.733+0000 7fe392de7700 1 -- 172.21.15.16:0/435302167 wait complete. 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 Processor -- start 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 -- start start 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe38c1000f0 0x7fe38c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c100ab0 0x7fe38c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 0x7fe38c1a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c198f10 con 0x7fe38c105f90 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c1a3740 con 0x7fe38c100ab0 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.734+0000 7fe392de7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c1a3880 con 0x7fe38c1000f0 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 0x7fe38c1a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 0x7fe38c1a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54250/0 (socket says 172.21.15.16:54250) 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 -- 172.21.15.16:0/1340992647 learned_addr learned my addr 172.21.15.16:0/1340992647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe390b83700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe38c1000f0 0x7fe38c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe38bfff700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c100ab0 0x7fe38c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 -- 172.21.15.16:0/1340992647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe38c1000f0 msgr2=0x7fe38c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe38c1000f0 0x7fe38c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 -- 172.21.15.16:0/1340992647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c100ab0 msgr2=0x7fe38c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c100ab0 0x7fe38c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 -- 172.21.15.16:0/1340992647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe384009d00 con 0x7fe38c105f90 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe391384700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 0x7fe38c1a3200 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe384007d70 tx=0x7fe384007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe389ffb700 1 -- 172.21.15.16:0/1340992647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400e040 con 0x7fe38c105f90 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c1a3a80 con 0x7fe38c105f90 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe389ffb700 1 -- 172.21.15.16:0/1340992647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe38401b8e0 con 0x7fe38c105f90 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.735+0000 7fe389ffb700 1 -- 172.21.15.16:0/1340992647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38401c780 con 0x7fe38c105f90 2024-05-08T23:27:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.736+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c1a3f50 con 0x7fe38c105f90 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.738+0000 7fe389ffb700 1 -- 172.21.15.16:0/1340992647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe38401ba50 con 0x7fe38c105f90 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.739+0000 7fe389ffb700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe3780a1290 0x7fe3780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.739+0000 7fe390b83700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe3780a1290 0x7fe3780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.739+0000 7fe389ffb700 1 -- 172.21.15.16:0/1340992647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fe3840ce500 con 0x7fe38c105f90 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe390b83700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe3780a1290 0x7fe3780a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe38c067bb0 tx=0x7fe37c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 Processor -- start 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 -- start start 2024-05-08T23:27:56.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4be00 0x560805e4c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e46350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e46cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805d9d150 con 0x560805e4c720 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805d9ab10 con 0x560805e46890 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805d96400 con 0x560805e4be00 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e46350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e46350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54254/0 (socket says 172.21.15.16:54254) 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe38bfff700 1 -- 172.21.15.16:0/2270084919 learned_addr learned my addr 172.21.15.16:0/2270084919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe391384700 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e46cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.740+0000 7fe390b83700 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4be00 0x560805e4c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe38bfff700 1 -- 172.21.15.16:0/2270084919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4be00 msgr2=0x560805e4c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4be00 0x560805e4c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe38bfff700 1 -- 172.21.15.16:0/2270084919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 msgr2=0x560805e46cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e46cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe38bfff700 1 -- 172.21.15.16:0/2270084919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560805c173b0 con 0x560805e4c720 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe390b83700 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4be00 0x560805e4c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e46350 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c90 tx=0x7fe374010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de63c8f35a764d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe382ffd700 1 -- 172.21.15.16:0/2270084919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374009790 con 0x560805e4c720 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe382ffd700 1 -- 172.21.15.16:0/2270084919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe374010990 con 0x560805e4c720 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.741+0000 7fe382ffd700 1 -- 172.21.15.16:0/2270084919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374003ce0 con 0x560805e4c720 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2270084919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 msgr2=0x560805e46350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e46350 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe374009c90 tx=0x7fe374010370 comp rx=0 tx=0).stop 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2270084919 shutdown_connections 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e46cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4be00 0x560805e4c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2270084919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e46350 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2270084919 >> 172.21.15.16:0/2270084919 conn(0x560805da4210 msgr2=0x560805e40df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2270084919 shutdown_connections 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2270084919 wait complete. 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 -- start start 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.742+0000 7fe39efa1b80 1 Processor -- start 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e4fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4be00 0x560805e50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805d32560 con 0x560805e4c720 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805d32200 con 0x560805e4be00 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805c196a0 con 0x560805e46890 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe39efa1b80 10 client.0 _ll_register_callbacks cb 0x560805d3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54262/0 (socket says 172.21.15.16:54262) 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 -- 172.21.15.16:0/2356753728 learned_addr learned my addr 172.21.15.16:0/2356753728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4be00 0x560805e50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe390b83700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e4fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 -- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 msgr2=0x560805e4fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e4fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 -- 172.21.15.16:0/2356753728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4be00 msgr2=0x560805e50060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4be00 0x560805e50060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe391384700 1 -- 172.21.15.16:0/2356753728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3740098f0 con 0x560805e4c720 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.743+0000 7fe390b83700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e4fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe391384700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e505a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe38409b230 tx=0x7fe38409ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdf1a1d20de7da0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe36bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe36bfff700 1 -- 172.21.15.16:0/2356753728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38409f0d0 con 0x560805e4c720 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe36bfff700 1 -- 172.21.15.16:0/2356753728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fe3840a3690 con 0x560805e4c720 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560805cef270 con 0x560805e4c720 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe36bfff700 1 -- 172.21.15.16:0/2356753728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3840ce170 con 0x560805e4c720 2024-05-08T23:27:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 10 client.17358 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560805bc3b60 con 0x560805e4c720 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe36bfff700 1 -- 172.21.15.16:0/2356753728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560805cef270 con 0x560805e4c720 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe36bfff700 1 -- 172.21.15.16:0/2356753728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe38409d050 con 0x560805e4c720 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 10 client.17358 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 10 client.17358 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 20 client.17358 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805e60040 0x560805e62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 4 client.17358 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.744+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560805e15810 con 0x560805e60040 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.745+0000 7fe390b83700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805e60040 0x560805e62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.745+0000 7fe390b83700 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805e60040 0x560805e62460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7fad7cacaf2afb6 server_cookie=8f113c8e2cfa8957 in_seq=0 out_seq=0 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.745+0000 7fe36bfff700 10 client.17358 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe36bfff700 1 -- 172.21.15.16:0/2356753728 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560805e15810 con 0x560805e60040 2024-05-08T23:27:56.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe36bfff700 10 client.17358 handle_command_reply: tid=0 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 client.17358 shutdown 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 msgr2=0x560805e505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e505a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe38409b230 tx=0x7fe38409ac80 comp rx=0 tx=0).stop 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 shutdown_connections 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4c720 0x560805e505a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e4fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805e60040 0x560805e62460 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2356753728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4be00 0x560805e50060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe36bfff700 0 client.17358 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 >> 172.21.15.16:0/2356753728 conn(0x560805da4210 msgr2=0x560805e405a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 shutdown_connections 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2356753728 wait complete. 2024-05-08T23:27:56.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.746+0000 7fe39efa1b80 20 client.17358 trim_cache size 0 max 16384 2024-05-08T23:27:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 Processor -- start 2024-05-08T23:27:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 -- start start 2024-05-08T23:27:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e5bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4c720 0x560805e40d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805e42ea0 con 0x560805e4be00 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805e67670 con 0x560805e46890 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805e3b800 con 0x560805e4c720 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e5bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e5bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54278/0 (socket says 172.21.15.16:54278) 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe38bfff700 1 -- 172.21.15.16:0/958107931 learned_addr learned my addr 172.21.15.16:0/958107931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe390b83700 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.761+0000 7fe391384700 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4c720 0x560805e40d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe38bfff700 1 -- 172.21.15.16:0/958107931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4c720 msgr2=0x560805e40d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe38bfff700 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4c720 0x560805e40d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe38bfff700 1 -- 172.21.15.16:0/958107931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 msgr2=0x560805e62460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe38bfff700 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e62460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe38bfff700 1 -- 172.21.15.16:0/958107931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560805d5c560 con 0x560805e4be00 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe391384700 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4c720 0x560805e40d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe38bfff700 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e5bc20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe374019000 tx=0x7fe374009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967558af0a4f9576 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe3837fe700 1 -- 172.21.15.16:0/958107931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374010d00 con 0x560805e4be00 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe3837fe700 1 -- 172.21.15.16:0/958107931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe374022040 con 0x560805e4be00 2024-05-08T23:27:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe3837fe700 1 -- 172.21.15.16:0/958107931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374004a70 con 0x560805e4be00 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.762+0000 7fe39efa1b80 1 -- 172.21.15.16:0/958107931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 msgr2=0x560805e5bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e5bc20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe374019000 tx=0x7fe374009790 comp rx=0 tx=0).stop 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 -- 172.21.15.16:0/958107931 shutdown_connections 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e4c720 0x560805e40d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e46890 0x560805e62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/958107931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e5bc20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 -- 172.21.15.16:0/958107931 >> 172.21.15.16:0/958107931 conn(0x560805da4210 msgr2=0x560805e46c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 -- 172.21.15.16:0/958107931 shutdown_connections 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 -- 172.21.15.16:0/958107931 wait complete. 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 -- start start 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 Processor -- start 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.763+0000 7fe39efa1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4c720 0x560805e49240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805e479a0 con 0x560805e4be00 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805e47b10 con 0x560805e4c720 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe39efa1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560805e47cd0 con 0x560805e46890 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe39efa1b80 10 client.0 _ll_register_callbacks cb 0x560805d3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e48b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54292/0 (socket says 172.21.15.16:54292) 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 -- 172.21.15.16:0/2887975615 learned_addr learned my addr 172.21.15.16:0/2887975615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe390b83700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe391384700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4c720 0x560805e49240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 -- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 msgr2=0x560805e57650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e57650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 -- 172.21.15.16:0/2887975615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4c720 msgr2=0x560805e49240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4c720 0x560805e49240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 -- 172.21.15.16:0/2887975615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560805c173b0 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe391384700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4c720 0x560805e49240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe38bfff700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e48b00 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe374005c50 tx=0x7fe374005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5cd398511cf7e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe36bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe36bfff700 1 -- 172.21.15.16:0/2887975615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37402b040 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe36bfff700 1 -- 172.21.15.16:0/2887975615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fe374016070 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.764+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe374010990 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe36bfff700 1 -- 172.21.15.16:0/2887975615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe374004020 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 10 client.17364 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560805c1bcd0 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe36bfff700 1 -- 172.21.15.16:0/2887975615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe374010990 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe36bfff700 1 -- 172.21.15.16:0/2887975615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe374003680 con 0x560805e4be00 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 10 client.17364 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:27:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 20 client.17364 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805eb6990 0x560805eb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 4 client.17364 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560805d8eec0 con 0x560805eb6990 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.765+0000 7fe390b83700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805eb6990 0x560805eb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe390b83700 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805eb6990 0x560805eb8d80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bc7cab0c47b9580 server_cookie=a26ad2949ca59037 in_seq=0 out_seq=0 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe36bfff700 10 client.17364 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe36bfff700 1 -- 172.21.15.16:0/2887975615 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560805d8eec0 con 0x560805eb6990 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe36bfff700 10 client.17364 handle_command_reply: tid=0 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 client.17364 shutdown 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 msgr2=0x560805e48b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e48b00 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe374005c50 tx=0x7fe374005260 comp rx=0 tx=0).stop 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 shutdown_connections 2024-05-08T23:27:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560805eb6990 0x560805eb8d80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560805e4c720 0x560805e49240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560805e46890 0x560805e57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 --2- 172.21.15.16:0/2887975615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560805e4be00 0x560805e48b00 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe36bfff700 0 client.17364 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 >> 172.21.15.16:0/2887975615 conn(0x560805da4210 msgr2=0x560805e4e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 shutdown_connections 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 1 -- 172.21.15.16:0/2887975615 wait complete. 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.766+0000 7fe39efa1b80 20 client.17364 trim_cache size 0 max 16384 2024-05-08T23:27:56.775 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe3780a1290 msgr2=0x7fe3780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe3780a1290 0x7fe3780a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe38c067bb0 tx=0x7fe37c009260 comp rx=0 tx=0).stop 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 msgr2=0x7fe38c1a3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 0x7fe38c1a3200 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe384007d70 tx=0x7fe384007da0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 shutdown_connections 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe3780a1290 0x7fe3780a3750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe38c105f90 0x7fe38c1a3200 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe38c1000f0 0x7fe38c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 --2- 172.21.15.16:0/1340992647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe38c100ab0 0x7fe38c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 >> 172.21.15.16:0/1340992647 conn(0x7fe38c076ba0 msgr2=0x7fe38c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 shutdown_connections 2024-05-08T23:27:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.767+0000 7fe392de7700 1 -- 172.21.15.16:0/1340992647 wait complete. 2024-05-08T23:27:56.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 -- 172.21.15.16:0/2067173467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d60069dd0 msgr2=0x7f5d60105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 --2- 172.21.15.16:0/2067173467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d60069dd0 0x7f5d60105b70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009990 tx=0x7f5d48009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 -- 172.21.15.16:0/2067173467 shutdown_connections 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 --2- 172.21.15.16:0/2067173467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d601062a0 0x7f5d6010de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 --2- 172.21.15.16:0/2067173467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 0x7f5d60069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 --2- 172.21.15.16:0/2067173467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d60069dd0 0x7f5d60105b70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 -- 172.21.15.16:0/2067173467 >> 172.21.15.16:0/2067173467 conn(0x7f5d60076ba0 msgr2=0x7f5d60076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 -- 172.21.15.16:0/2067173467 shutdown_connections 2024-05-08T23:27:56.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.920+0000 7f5d6600c700 1 -- 172.21.15.16:0/2067173467 wait complete. 2024-05-08T23:27:56.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 Processor -- start 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 -- start start 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 0x7f5d60197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d601062a0 0x7f5d60197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 0x7f5d6019c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6019c660 con 0x7f5d6019bbe0 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6019c7d0 con 0x7f5d60069420 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d6600c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6019c990 con 0x7f5d601062a0 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 0x7f5d6019c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 0x7f5d60197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 0x7f5d6019c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54316/0 (socket says 172.21.15.16:54316) 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d5ffff700 1 -- 172.21.15.16:0/4267231206 learned_addr learned my addr 172.21.15.16:0/4267231206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.921+0000 7f5d5effd700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d601062a0 0x7f5d60197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5ffff700 1 -- 172.21.15.16:0/4267231206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d601062a0 msgr2=0x7f5d60197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d601062a0 0x7f5d60197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5ffff700 1 -- 172.21.15.16:0/4267231206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 msgr2=0x7f5d60197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 0x7f5d60197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5ffff700 1 -- 172.21.15.16:0/4267231206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480095f0 con 0x7f5d6019bbe0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 0x7f5d60197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 0x7f5d6019c060 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5d58007ee0 tx=0x7f5d5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5cff9700 1 -- 172.21.15.16:0/4267231206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5800ddd0 con 0x7f5d6019bbe0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5cff9700 1 -- 172.21.15.16:0/4267231206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d5800f040 con 0x7f5d6019bbe0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d601a3ef0 con 0x7f5d6019bbe0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d5cff9700 1 -- 172.21.15.16:0/4267231206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580047d0 con 0x7f5d6019bbe0 2024-05-08T23:27:56.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.922+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d601a4380 con 0x7f5d6019bbe0 2024-05-08T23:27:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.924+0000 7f5d5cff9700 1 -- 172.21.15.16:0/4267231206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5d58003a00 con 0x7f5d6019bbe0 2024-05-08T23:27:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.925+0000 7f5d5cff9700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5d4c0a1290 0x7f5d4c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.926+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5d4c0a1290 0x7f5d4c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.926+0000 7f5d5cff9700 1 -- 172.21.15.16:0/4267231206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5d580cc950 con 0x7f5d6019bbe0 2024-05-08T23:27:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.926+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5d4c0a1290 0x7f5d4c0a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005950 tx=0x7f5d5000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.926+0000 7f5d721c6b80 1 Processor -- start 2024-05-08T23:27:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.926+0000 7f5d721c6b80 1 -- start start 2024-05-08T23:27:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 0x558cdb7c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7c6890 0x558cdb7c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb71d150 con 0x558cdb7cc720 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb71ab10 con 0x558cdb7c6890 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb716400 con 0x558cdb7cbe00 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 0x558cdb7c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 0x558cdb7c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54322/0 (socket says 172.21.15.16:54322) 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 -- 172.21.15.16:0/1213285818 learned_addr learned my addr 172.21.15.16:0/1213285818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7c6890 0x558cdb7c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 -- 172.21.15.16:0/1213285818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 msgr2=0x558cdb7cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 -- 172.21.15.16:0/1213285818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7c6890 msgr2=0x558cdb7c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7c6890 0x558cdb7c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 -- 172.21.15.16:0/1213285818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cdb5973b0 con 0x558cdb7cc720 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.927+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 0x558cdb7c6350 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5d48006010 tx=0x7f5d48005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50648af2b05ea2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d55ffb700 1 -- 172.21.15.16:0/1213285818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480052c0 con 0x558cdb7cc720 2024-05-08T23:27:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d55ffb700 1 -- 172.21.15.16:0/1213285818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d4801e040 con 0x558cdb7cc720 2024-05-08T23:27:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d55ffb700 1 -- 172.21.15.16:0/1213285818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48004760 con 0x558cdb7cc720 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1213285818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 msgr2=0x558cdb7c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 0x558cdb7c6350 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f5d48006010 tx=0x7f5d48005f00 comp rx=0 tx=0).stop 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1213285818 shutdown_connections 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7c6890 0x558cdb7c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1213285818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7cc720 0x558cdb7c6350 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1213285818 >> 172.21.15.16:0/1213285818 conn(0x558cdb724210 msgr2=0x558cdb7c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1213285818 shutdown_connections 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.928+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1213285818 wait complete. 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.929+0000 7f5d721c6b80 1 -- start start 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.929+0000 7f5d721c6b80 1 Processor -- start 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb6b2560 con 0x558cdb7c6890 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb6b2200 con 0x558cdb7cc720 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb5996a0 con 0x558cdb7cbe00 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d721c6b80 10 client.0 _ll_register_callbacks cb 0x558cdb6bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54326/0 (socket says 172.21.15.16:54326) 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/1340258604 learned_addr learned my addr 172.21.15.16:0/1340258604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/1340258604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 msgr2=0x558cdb7cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 msgr2=0x558cdb7d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/1340258604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6010e690 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d2750 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5d50011aa0 tx=0x7f5d500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e4e6655413f8cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d3effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.930+0000 7f5d3effd700 1 -- 172.21.15.16:0/1340258604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50004a30 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d3effd700 1 -- 172.21.15.16:0/1340258604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5d50017070 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cdb66f270 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d3effd700 1 -- 172.21.15.16:0/1340258604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d50004b90 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 10 client.17376 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558cdb543b60 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d3effd700 1 -- 172.21.15.16:0/1340258604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558cdb66f270 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d3effd700 1 -- 172.21.15.16:0/1340258604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d500051e0 con 0x558cdb7c6890 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 10 client.17376 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 20 client.17376 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb7e0290 0x558cdb7e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 4 client.17376 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cdb795810 con 0x558cdb7e0290 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.931+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb7e0290 0x558cdb7e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.932+0000 7f5d3effd700 10 client.17376 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.932+0000 7f5d5effd700 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb7e0290 0x558cdb7e26b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b800262d43c794 server_cookie=3e0595a5c931f6e7 in_seq=0 out_seq=0 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d3effd700 1 -- 172.21.15.16:0/1340258604 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558cdb795810 con 0x558cdb7e0290 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d3effd700 10 client.17376 handle_command_reply: tid=0 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 client.17376 shutdown 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 msgr2=0x558cdb7d2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d2750 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5d50011aa0 tx=0x7f5d500047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 shutdown_connections 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb7e0290 0x558cdb7e26b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d2750 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/1340258604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 >> 172.21.15.16:0/1340258604 conn(0x558cdb724210 msgr2=0x558cdb7c05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d3effd700 0 client.17376 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 shutdown_connections 2024-05-08T23:27:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 1 -- 172.21.15.16:0/1340258604 wait complete. 2024-05-08T23:27:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.933+0000 7f5d721c6b80 20 client.17376 trim_cache size 0 max 16384 2024-05-08T23:27:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.947+0000 7f5d721c6b80 1 Processor -- start 2024-05-08T23:27:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.947+0000 7f5d721c6b80 1 -- start start 2024-05-08T23:27:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7dbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb7d2bf0 con 0x558cdb7c6890 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb7bddc0 con 0x558cdb7cc720 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb7c2e50 con 0x558cdb7cbe00 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54328/0 (socket says 172.21.15.16:54328) 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/3268596468 learned_addr learned my addr 172.21.15.16:0/3268596468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5effd700 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7dbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/3268596468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 msgr2=0x558cdb7dbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7dbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/3268596468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 msgr2=0x558cdb7c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7c0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/3268596468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cdb6dc560 con 0x558cdb7c6890 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d01c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5d50004790 tx=0x7f5d5000b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b07640a1856b51a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.948+0000 7f5d567fc700 1 -- 172.21.15.16:0/3268596468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50004220 con 0x558cdb7c6890 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d567fc700 1 -- 172.21.15.16:0/3268596468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d50023070 con 0x558cdb7c6890 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d567fc700 1 -- 172.21.15.16:0/3268596468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5001e6a0 con 0x558cdb7c6890 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 -- 172.21.15.16:0/3268596468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 msgr2=0x558cdb7d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d01c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5d50004790 tx=0x7f5d5000b540 comp rx=0 tx=0).stop 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 -- 172.21.15.16:0/3268596468 shutdown_connections 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cc720 0x558cdb7c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d01c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/3268596468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cbe00 0x558cdb7dbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 -- 172.21.15.16:0/3268596468 >> 172.21.15.16:0/3268596468 conn(0x558cdb724210 msgr2=0x558cdb7e7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 -- 172.21.15.16:0/3268596468 shutdown_connections 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.949+0000 7f5d721c6b80 1 -- 172.21.15.16:0/3268596468 wait complete. 2024-05-08T23:27:56.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 -- start start 2024-05-08T23:27:56.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 Processor -- start 2024-05-08T23:27:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cbe00 0x558cdb7d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cc720 0x558cdb7d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb7d5be0 con 0x558cdb7c6890 2024-05-08T23:27:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb7d5d50 con 0x558cdb7cbe00 2024-05-08T23:27:56.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cdb7d5f10 con 0x558cdb7cc720 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d721c6b80 10 client.0 _ll_register_callbacks cb 0x558cdb6bbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54336/0 (socket says 172.21.15.16:54336) 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/657701925 learned_addr learned my addr 172.21.15.16:0/657701925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d5effd700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cbe00 0x558cdb7d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.950+0000 7f5d5ffff700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cc720 0x558cdb7d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/657701925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cc720 msgr2=0x558cdb7d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cc720 0x558cdb7d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cbe00 msgr2=0x558cdb7d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cbe00 0x558cdb7d5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d5f7fe700 1 -- 172.21.15.16:0/657701925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cdb543b60 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d5f7fe700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d7330 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5d50000c00 tx=0x7f5d50003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa8d5e8ba447dc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d3effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d3effd700 1 -- 172.21.15.16:0/657701925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50003a90 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d3effd700 1 -- 172.21.15.16:0/657701925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5d50017050 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d4801e040 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d3effd700 1 -- 172.21.15.16:0/657701925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d50004000 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d721c6b80 10 client.17382 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558cdb59bcd0 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.951+0000 7f5d3effd700 1 -- 172.21.15.16:0/657701925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5d4801e040 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d3effd700 1 -- 172.21.15.16:0/657701925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5d5003b030 con 0x558cdb7c6890 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d721c6b80 10 client.17382 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d721c6b80 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d721c6b80 20 client.17382 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:56.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb8368b0 0x558cdb838cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d721c6b80 4 client.17382 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558cdb70eec0 con 0x558cdb8368b0 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d5effd700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb8368b0 0x558cdb838cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d3effd700 10 client.17382 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.952+0000 7f5d5effd700 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb8368b0 0x558cdb838cd0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ed3aa7fb98adc5 server_cookie=ded26e56174d9d8e in_seq=0 out_seq=0 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d3effd700 1 -- 172.21.15.16:0/657701925 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558cdb70eec0 con 0x558cdb8368b0 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d3effd700 10 client.17382 handle_command_reply: tid=0 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 client.17382 shutdown 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 msgr2=0x558cdb7d7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d7330 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5d50000c00 tx=0x7f5d50003e80 comp rx=0 tx=0).stop 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 shutdown_connections 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558cdb7cc720 0x558cdb7d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cdb7c6890 0x558cdb7d7330 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558cdb8368b0 0x558cdb838cd0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 --2- 172.21.15.16:0/657701925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558cdb7cbe00 0x558cdb7d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 >> 172.21.15.16:0/657701925 conn(0x558cdb724210 msgr2=0x558cdb7cabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d3effd700 0 client.17382 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 shutdown_connections 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 1 -- 172.21.15.16:0/657701925 wait complete. 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.953+0000 7f5d721c6b80 20 client.17382 trim_cache size 0 max 16384 2024-05-08T23:27:56.959 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5d4c0a1290 msgr2=0x7f5d4c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5d4c0a1290 0x7f5d4c0a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d50005950 tx=0x7f5d5000b410 comp rx=0 tx=0).stop 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 msgr2=0x7f5d6019c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 0x7f5d6019c060 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f5d58007ee0 tx=0x7f5d5800d3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 shutdown_connections 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5d4c0a1290 0x7f5d4c0a3750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d6019bbe0 0x7f5d6019c060 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d60069420 0x7f5d60197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 --2- 172.21.15.16:0/4267231206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d601062a0 0x7f5d60197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:56.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 >> 172.21.15.16:0/4267231206 conn(0x7f5d60076ba0 msgr2=0x7f5d60107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 shutdown_connections 2024-05-08T23:27:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:56.954+0000 7f5d6600c700 1 -- 172.21.15.16:0/4267231206 wait complete. 2024-05-08T23:27:57.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:56 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1214101228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:27:57.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:56 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1214101228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:27:57.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:56 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1214101228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:27:57.967 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:27:57.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:27:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 -- 172.21.15.16:0/3959612864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86381062a0 msgr2=0x7f863810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 --2- 172.21.15.16:0/3959612864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86381062a0 0x7f863810de20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f863000b0a0 tx=0x7f863000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 -- 172.21.15.16:0/3959612864 shutdown_connections 2024-05-08T23:27:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 --2- 172.21.15.16:0/3959612864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86381062a0 0x7f863810de20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 --2- 172.21.15.16:0/3959612864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8638069420 0x7f8638069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 --2- 172.21.15.16:0/3959612864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638069dd0 0x7f8638105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 -- 172.21.15.16:0/3959612864 >> 172.21.15.16:0/3959612864 conn(0x7f8638076ba0 msgr2=0x7f8638076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 -- 172.21.15.16:0/3959612864 shutdown_connections 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.119+0000 7f864069b700 1 -- 172.21.15.16:0/3959612864 wait complete. 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 Processor -- start 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 -- start start 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638069420 0x7f8638197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86381062a0 0x7f8638197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 0x7f863819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863819c770 con 0x7f863819bcf0 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863819c8e0 con 0x7f86381062a0 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f864069b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863819caa0 con 0x7f8638069420 2024-05-08T23:27:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f863ec38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 0x7f863819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f863ec38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 0x7f863819c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54348/0 (socket says 172.21.15.16:54348) 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f863ec38700 1 -- 172.21.15.16:0/2774213384 learned_addr learned my addr 172.21.15.16:0/2774213384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f863dc36700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86381062a0 0x7f8638197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.120+0000 7f863e437700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638069420 0x7f8638197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f863ec38700 1 -- 172.21.15.16:0/2774213384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638069420 msgr2=0x7f8638197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f863ec38700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638069420 0x7f8638197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f863ec38700 1 -- 172.21.15.16:0/2774213384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86381062a0 msgr2=0x7f8638197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f863ec38700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86381062a0 0x7f8638197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f863ec38700 1 -- 172.21.15.16:0/2774213384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8630009d00 con 0x7f863819bcf0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f863ec38700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 0x7f863819c170 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8630006130 tx=0x7f86300095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f862f7fe700 1 -- 172.21.15.16:0/2774213384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000e040 con 0x7f863819bcf0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f862f7fe700 1 -- 172.21.15.16:0/2774213384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f863001b450 con 0x7f863819bcf0 2024-05-08T23:27:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86381a3fa0 con 0x7f863819bcf0 2024-05-08T23:27:58.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f862f7fe700 1 -- 172.21.15.16:0/2774213384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300052a0 con 0x7f863819bcf0 2024-05-08T23:27:58.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.121+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86381a4420 con 0x7f863819bcf0 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.123+0000 7f862f7fe700 1 -- 172.21.15.16:0/2774213384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f863001b5c0 con 0x7f863819bcf0 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.124+0000 7f862f7fe700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86240a1290 0x7f86240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.124+0000 7f862f7fe700 1 -- 172.21.15.16:0/2774213384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f86300ce1b0 con 0x7f863819bcf0 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.124+0000 7f863e437700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86240a1290 0x7f86240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f863e437700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86240a1290 0x7f86240a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8634005950 tx=0x7f863400a300 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 Processor -- start 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 -- start start 2024-05-08T23:27:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da33d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 0x5638da3377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da28a1d0 con 0x5638da33dbb0 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da287b90 con 0x5638da337d20 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.125+0000 7f864c855b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da283480 con 0x5638da33d290 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 0x5638da3377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863e437700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da33d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 0x5638da3377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54364/0 (socket says 172.21.15.16:54364) 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863e437700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da33d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:58728/0 (socket says 172.21.15.16:58728) 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 -- 172.21.15.16:0/3151362254 learned_addr learned my addr 172.21.15.16:0/3151362254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863ec38700 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 -- 172.21.15.16:0/3151362254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 msgr2=0x5638da33d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da33d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 -- 172.21.15.16:0/3151362254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 msgr2=0x5638da338160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 -- 172.21.15.16:0/3151362254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638da104430 con 0x5638da33dbb0 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863e437700 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da33d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f863dc36700 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 0x5638da3377e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8628009cb0 tx=0x7f8628011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3b3d7cb99f2fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f862cff9700 1 -- 172.21.15.16:0/3151362254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628011d20 con 0x5638da33dbb0 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f862cff9700 1 -- 172.21.15.16:0/3151362254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8628011e80 con 0x5638da33dbb0 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.126+0000 7f862cff9700 1 -- 172.21.15.16:0/3151362254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628005280 con 0x5638da33dbb0 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 -- 172.21.15.16:0/3151362254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 msgr2=0x5638da3377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 0x5638da3377e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8628009cb0 tx=0x7f8628011440 comp rx=0 tx=0).stop 2024-05-08T23:27:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 -- 172.21.15.16:0/3151362254 shutdown_connections 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da33d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 --2- 172.21.15.16:0/3151362254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33dbb0 0x5638da3377e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 -- 172.21.15.16:0/3151362254 >> 172.21.15.16:0/3151362254 conn(0x5638da291290 msgr2=0x5638da32de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 -- 172.21.15.16:0/3151362254 shutdown_connections 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.127+0000 7f864c855b80 1 -- 172.21.15.16:0/3151362254 wait complete. 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 -- start start 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 Processor -- start 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da337d20 0x5638da343c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da340eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da3415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da21f5e0 con 0x5638da33d290 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da21f280 con 0x5638da33dbb0 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da106720 con 0x5638da337d20 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f864c855b80 10 client.0 _ll_register_callbacks cb 0x5638da228bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f863dc36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da340eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f863dc36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da340eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54374/0 (socket says 172.21.15.16:54374) 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f863dc36700 1 -- 172.21.15.16:0/2094392700 learned_addr learned my addr 172.21.15.16:0/2094392700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f863e437700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da337d20 0x5638da343c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.128+0000 7f863ec38700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da3415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f863dc36700 1 -- 172.21.15.16:0/2094392700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da337d20 msgr2=0x5638da343c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f863dc36700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da337d20 0x5638da343c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f863dc36700 1 -- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 msgr2=0x5638da3415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f863dc36700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da3415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f863dc36700 1 -- 172.21.15.16:0/2094392700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863400f9e0 con 0x5638da33d290 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f863dc36700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da340eb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8628009c80 tx=0x7f8628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b305710b8abb60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f8621ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f8621ffb700 1 -- 172.21.15.16:0/2094392700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628004260 con 0x5638da33d290 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f8621ffb700 1 -- 172.21.15.16:0/2094392700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f86280043c0 con 0x5638da33d290 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638da302890 con 0x5638da33d290 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f8621ffb700 1 -- 172.21.15.16:0/2094392700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8628004580 con 0x5638da33d290 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f864c855b80 10 client.17394 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.129+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f86280098e0 con 0x5638da33d290 2024-05-08T23:27:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f8621ffb700 1 -- 172.21.15.16:0/2094392700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638da302890 con 0x5638da33d290 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f8621ffb700 1 -- 172.21.15.16:0/2094392700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8628005080 con 0x5638da33d290 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f864c855b80 10 client.17394 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f864c855b80 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f864c855b80 20 client.17394 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f864c855b80 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3514f0 0x5638da353910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f864c855b80 4 client.17394 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638da27bf40 con 0x5638da3514f0 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f863e437700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3514f0 0x5638da353910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f8621ffb700 10 client.17394 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.130+0000 7f863e437700 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3514f0 0x5638da353910 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed320ce02d121e30 server_cookie=1e4b1612ae007b38 in_seq=0 out_seq=0 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f8621ffb700 1 -- 172.21.15.16:0/2094392700 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638da27bf40 con 0x5638da3514f0 2024-05-08T23:27:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f8621ffb700 10 client.17394 handle_command_reply: tid=0 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 client.17394 shutdown 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 msgr2=0x5638da340eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da340eb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8628009c80 tx=0x7f8628004750 comp rx=0 tx=0).stop 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 shutdown_connections 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3514f0 0x5638da353910 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da3415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f8621ffb700 0 client.17394 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da337d20 0x5638da343c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 --2- 172.21.15.16:0/2094392700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da340eb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 >> 172.21.15.16:0/2094392700 conn(0x5638da291290 msgr2=0x5638da32d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 shutdown_connections 2024-05-08T23:27:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.131+0000 7f864c855b80 1 -- 172.21.15.16:0/2094392700 wait complete. 2024-05-08T23:27:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.132+0000 7f864c855b80 20 client.17394 trim_cache size 0 max 16384 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.151+0000 7f864c855b80 1 Processor -- start 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.151+0000 7f864c855b80 1 -- start start 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f864c855b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f864c855b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da32dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f864c855b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33dbb0 0x5638da343760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f864c855b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da358b20 con 0x5638da33d290 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f864c855b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da3260c0 con 0x5638da337d20 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f864c855b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da33b8a0 con 0x5638da33dbb0 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da32dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863e437700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da32dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54376/0 (socket says 172.21.15.16:54376) 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863e437700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:36864/0 (socket says 172.21.15.16:36864) 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 -- 172.21.15.16:0/1186153434 learned_addr learned my addr 172.21.15.16:0/1186153434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863ec38700 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33dbb0 0x5638da343760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 -- 172.21.15.16:0/1186153434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33dbb0 msgr2=0x5638da343760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33dbb0 0x5638da343760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 -- 172.21.15.16:0/1186153434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 msgr2=0x5638da338100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 -- 172.21.15.16:0/1186153434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638da0b0be0 con 0x5638da33d290 2024-05-08T23:27:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863e437700 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863ec38700 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33dbb0 0x5638da343760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.152+0000 7f863dc36700 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da32dda0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f86280053e0 tx=0x7f8628022460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabad7cff6748e71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f862d7fa700 1 -- 172.21.15.16:0/1186153434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280047c0 con 0x5638da33d290 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f862d7fa700 1 -- 172.21.15.16:0/1186153434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8628011020 con 0x5638da33d290 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f862d7fa700 1 -- 172.21.15.16:0/1186153434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280051a0 con 0x5638da33d290 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 -- 172.21.15.16:0/1186153434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 msgr2=0x5638da32dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da32dda0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f86280053e0 tx=0x7f8628022460 comp rx=0 tx=0).stop 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 -- 172.21.15.16:0/1186153434 shutdown_connections 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33dbb0 0x5638da343760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da337d20 0x5638da338100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 --2- 172.21.15.16:0/1186153434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da33d290 0x5638da32dda0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 -- 172.21.15.16:0/1186153434 >> 172.21.15.16:0/1186153434 conn(0x5638da291290 msgr2=0x5638da32a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 -- 172.21.15.16:0/1186153434 shutdown_connections 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.153+0000 7f864c855b80 1 -- 172.21.15.16:0/1186153434 wait complete. 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 -- start start 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 Processor -- start 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 0x5638da345070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da3455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da345af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da348f40 con 0x5638da337d20 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da3a79f0 con 0x5638da33dbb0 2024-05-08T23:27:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638da3a7b80 con 0x5638da33d290 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f864c855b80 10 client.0 _ll_register_callbacks cb 0x5638da228bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f863e437700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 0x5638da345070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f863e437700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 0x5638da345070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54390/0 (socket says 172.21.15.16:54390) 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f863e437700 1 -- 172.21.15.16:0/2489157825 learned_addr learned my addr 172.21.15.16:0/2489157825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f863ec38700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da345af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.154+0000 7f863dc36700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da3455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f863e437700 1 -- 172.21.15.16:0/2489157825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 msgr2=0x5638da3455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f863e437700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da3455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f863e437700 1 -- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 msgr2=0x5638da345af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f863e437700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da345af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f863e437700 1 -- 172.21.15.16:0/2489157825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863401d040 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f863e437700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 0x5638da345070 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f863400a9d0 tx=0x7f86340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7047cf6d36bef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f8621ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f8621ffb700 1 -- 172.21.15.16:0/2489157825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634022040 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f8621ffb700 1 -- 172.21.15.16:0/2489157825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f8634004a30 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638da1dc2f0 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f8621ffb700 1 -- 172.21.15.16:0/2489157825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8634004bf0 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f864c855b80 10 client.17400 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f86280098e0 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.155+0000 7f8621ffb700 1 -- 172.21.15.16:0/2489157825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638da1dc2f0 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f8621ffb700 1 -- 172.21.15.16:0/2489157825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f86340050b0 con 0x5638da337d20 2024-05-08T23:27:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f864c855b80 10 client.17400 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f864c855b80 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f864c855b80 20 client.17400 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f864c855b80 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3a94a0 0x5638da3ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f864c855b80 4 client.17400 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5638da0d1d70 con 0x5638da3a94a0 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f863dc36700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3a94a0 0x5638da3ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f863dc36700 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3a94a0 0x5638da3ab8c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f383a0edbf6a1ab8 server_cookie=46c84e4b75f89387 in_seq=0 out_seq=0 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.156+0000 7f8621ffb700 10 client.17400 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.157+0000 7f8621ffb700 1 -- 172.21.15.16:0/2489157825 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5638da0d1d70 con 0x5638da3a94a0 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.157+0000 7f8621ffb700 10 client.17400 handle_command_reply: tid=0 2024-05-08T23:27:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.157+0000 7f864c855b80 1 client.17400 shutdown 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 msgr2=0x5638da345070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 0x5638da345070 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f863400a9d0 tx=0x7f86340047c0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 shutdown_connections 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5638da3a94a0 0x5638da3ab8c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5638da33dbb0 0x5638da345af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638da337d20 0x5638da345070 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 --2- 172.21.15.16:0/2489157825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638da33d290 0x5638da3455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 >> 172.21.15.16:0/2489157825 conn(0x5638da291290 msgr2=0x5638da338100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f8621ffb700 0 client.17400 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 shutdown_connections 2024-05-08T23:27:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 1 -- 172.21.15.16:0/2489157825 wait complete. 2024-05-08T23:27:58.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864c855b80 20 client.17400 trim_cache size 0 max 16384 2024-05-08T23:27:58.166 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:27:58.166 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:27:58.166 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ed045838-cf59-47cf-9e37-7304d5f9e21e", 2024-05-08T23:27:58.166 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:27:58.166 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:27:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86240a1290 msgr2=0x7f86240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86240a1290 0x7f86240a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8634005950 tx=0x7f863400a300 comp rx=0 tx=0).stop 2024-05-08T23:27:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 msgr2=0x7f863819c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 0x7f863819c170 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8630006130 tx=0x7f86300095b0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 shutdown_connections 2024-05-08T23:27:58.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86240a1290 0x7f86240a3750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863819bcf0 0x7f863819c170 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8638069420 0x7f8638197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 --2- 172.21.15.16:0/2774213384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86381062a0 0x7f8638197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.158+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 >> 172.21.15.16:0/2774213384 conn(0x7f8638076ba0 msgr2=0x7f86380ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.159+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 shutdown_connections 2024-05-08T23:27:58.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.159+0000 7f864069b700 1 -- 172.21.15.16:0/2774213384 wait complete. 2024-05-08T23:27:58.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 -- 172.21.15.16:0/1033593884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 msgr2=0x7f2910106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 --2- 172.21.15.16:0/1033593884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f2910106320 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2900009990 tx=0x7f2900009ca0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 -- 172.21.15.16:0/1033593884 shutdown_connections 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 --2- 172.21.15.16:0/1033593884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2910100220 0x7f291010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 --2- 172.21.15.16:0/1033593884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f2910106320 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 --2- 172.21.15.16:0/1033593884 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29100ff860 0x7f29100ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 -- 172.21.15.16:0/1033593884 >> 172.21.15.16:0/1033593884 conn(0x7f29100751c0 msgr2=0x7f29100755d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 -- 172.21.15.16:0/1033593884 shutdown_connections 2024-05-08T23:27:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.312+0000 7f2915816700 1 -- 172.21.15.16:0/1033593884 wait complete. 2024-05-08T23:27:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.313+0000 7f2915816700 1 Processor -- start 2024-05-08T23:27:58.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.313+0000 7f2915816700 1 -- start start 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.313+0000 7f2915816700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29100ff860 0x7f2910100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.313+0000 7f2915816700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910100220 0x7f2910100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.313+0000 7f2915816700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f29101014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2915816700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101a5760 con 0x7f2910105f40 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2915816700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101a58d0 con 0x7f2910100220 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2915816700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101a5a90 con 0x7f29100ff860 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f29101014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f29101014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54408/0 (socket says 172.21.15.16:54408) 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 -- 172.21.15.16:0/4114077474 learned_addr learned my addr 172.21.15.16:0/4114077474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290effd700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29100ff860 0x7f2910100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290e7fc700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910100220 0x7f2910100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 -- 172.21.15.16:0/4114077474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29100ff860 msgr2=0x7f2910100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29100ff860 0x7f2910100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 -- 172.21.15.16:0/4114077474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910100220 msgr2=0x7f2910100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910100220 0x7f2910100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 -- 172.21.15.16:0/4114077474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000095f0 con 0x7f2910105f40 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290e7fc700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910100220 0x7f2910100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f290f7fe700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f29101014c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f290800ef30 tx=0x7f290800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2914814700 1 -- 172.21.15.16:0/4114077474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908018070 con 0x7f2910105f40 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29101a5d10 con 0x7f2910105f40 2024-05-08T23:27:58.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2914814700 1 -- 172.21.15.16:0/4114077474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f290800f040 con 0x7f2910105f40 2024-05-08T23:27:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.314+0000 7f2914814700 1 -- 172.21.15.16:0/4114077474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290800cc70 con 0x7f2910105f40 2024-05-08T23:27:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.315+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29101a6220 con 0x7f2910105f40 2024-05-08T23:27:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.317+0000 7f2914814700 1 -- 172.21.15.16:0/4114077474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2908007500 con 0x7f2910105f40 2024-05-08T23:27:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.318+0000 7f2914814700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28f80a1310 0x7f28f80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.318+0000 7f2914814700 1 -- 172.21.15.16:0/4114077474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f29080ccb70 con 0x7f2910105f40 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.318+0000 7f290effd700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28f80a1310 0x7f28f80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28f80a1310 0x7f28f80a37d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2900006010 tx=0x7f290000b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 Processor -- start 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 -- start start 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 0x5608042f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560804244150 con 0x5608042f2e00 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560804241b10 con 0x5608042ed890 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f29219d0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080423d400 con 0x5608042f3720 2024-05-08T23:27:58.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 0x5608042f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 0x5608042f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54424/0 (socket says 172.21.15.16:54424) 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 -- 172.21.15.16:0/3208824473 learned_addr learned my addr 172.21.15.16:0/3208824473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290e7fc700 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 -- 172.21.15.16:0/3208824473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 msgr2=0x5608042ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 -- 172.21.15.16:0/3208824473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 msgr2=0x5608042edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290effd700 1 -- 172.21.15.16:0/3208824473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608040be3b0 con 0x5608042f2e00 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.319+0000 7f290e7fc700 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:58.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.320+0000 7f290effd700 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 0x5608042f31e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f290000b710 tx=0x7f2900020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2d0ca6c6845d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.320+0000 7f28f5ffb700 1 -- 172.21.15.16:0/3208824473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290000f660 con 0x5608042f2e00 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.320+0000 7f28f5ffb700 1 -- 172.21.15.16:0/3208824473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f290001e040 con 0x5608042f2e00 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.320+0000 7f28f5ffb700 1 -- 172.21.15.16:0/3208824473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290002f970 con 0x5608042f2e00 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 -- 172.21.15.16:0/3208824473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 msgr2=0x5608042f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 0x5608042f31e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f290000b710 tx=0x7f2900020000 comp rx=0 tx=0).stop 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 -- 172.21.15.16:0/3208824473 shutdown_connections 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f2e00 0x5608042f31e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3208824473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 -- 172.21.15.16:0/3208824473 >> 172.21.15.16:0/3208824473 conn(0x56080424b210 msgr2=0x5608042e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 -- 172.21.15.16:0/3208824473 shutdown_connections 2024-05-08T23:27:58.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 -- 172.21.15.16:0/3208824473 wait complete. 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 -- start start 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.321+0000 7f29219d0b80 1 Processor -- start 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 0x5608042f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042f2e00 0x5608042f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608041d9560 con 0x5608042ed890 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608041d9200 con 0x5608042f2e00 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608040c06a0 con 0x5608042f3720 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f29219d0b80 10 client.0 _ll_register_callbacks cb 0x5608041e2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 0x5608042f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 0x5608042f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54426/0 (socket says 172.21.15.16:54426) 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 -- 172.21.15.16:0/2646600682 learned_addr learned my addr 172.21.15.16:0/2646600682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290e7fc700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042f2e00 0x5608042f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290f7fe700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 -- 172.21.15.16:0/2646600682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 msgr2=0x5608042f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 -- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042f2e00 msgr2=0x5608042f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042f2e00 0x5608042f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 -- 172.21.15.16:0/2646600682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291010e7a0 con 0x5608042ed890 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290e7fc700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042f2e00 0x5608042f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f290effd700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 0x5608042f6a10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2900000c00 tx=0x7f290002f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ec21c8938d22c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.322+0000 7f28eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f28eeffd700 1 -- 172.21.15.16:0/2646600682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290000f870 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f28eeffd700 1 -- 172.21.15.16:0/2646600682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f290000f9d0 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560804196270 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f28eeffd700 1 -- 172.21.15.16:0/2646600682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f290002f990 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 10 client.17412 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56080406ab60 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f28eeffd700 1 -- 172.21.15.16:0/2646600682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560804196270 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f28eeffd700 1 -- 172.21.15.16:0/2646600682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2900038bf0 con 0x5608042ed890 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 10 client.17412 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:58.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 20 client.17412 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560804306c40 0x560804309030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 4 client.17412 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.323+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608042bc810 con 0x560804306c40 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.324+0000 7f290e7fc700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560804306c40 0x560804309030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.324+0000 7f290e7fc700 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560804306c40 0x560804309030 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac19ba95e9ed6f15 server_cookie=9348be5c4c72bedc in_seq=0 out_seq=0 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.324+0000 7f28eeffd700 10 client.17412 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f28eeffd700 1 -- 172.21.15.16:0/2646600682 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608042bc810 con 0x560804306c40 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f28eeffd700 10 client.17412 handle_command_reply: tid=0 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 client.17412 shutdown 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 msgr2=0x5608042f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 0x5608042f6a10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2900000c00 tx=0x7f290002f820 comp rx=0 tx=0).stop 2024-05-08T23:27:58.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 shutdown_connections 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f3720 0x5608042f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042ed890 0x5608042f6a10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x560804306c40 0x560804309030 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 --2- 172.21.15.16:0/2646600682 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042f2e00 0x5608042f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f28eeffd700 0 client.17412 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 >> 172.21.15.16:0/2646600682 conn(0x56080424b210 msgr2=0x5608042e75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 shutdown_connections 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 1 -- 172.21.15.16:0/2646600682 wait complete. 2024-05-08T23:27:58.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.325+0000 7f29219d0b80 20 client.17412 trim_cache size 0 max 16384 2024-05-08T23:27:58.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 Processor -- start 2024-05-08T23:27:58.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 -- start start 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x5608042f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080430e240 con 0x5608042f3720 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608042e2800 con 0x5608042ed890 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f29219d0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608042e9640 con 0x5608042f2e00 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x5608042f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x5608042f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54430/0 (socket says 172.21.15.16:54430) 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 -- 172.21.15.16:0/3195780927 learned_addr learned my addr 172.21.15.16:0/3195780927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290effd700 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290e7fc700 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 -- 172.21.15.16:0/3195780927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 msgr2=0x5608042f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 -- 172.21.15.16:0/3195780927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 msgr2=0x5608042e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.344+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f290f7fe700 1 -- 172.21.15.16:0/3195780927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560804203560 con 0x5608042f3720 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x5608042f72b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f29080ca520 tx=0x7f2908009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2292deb3266a9f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f28f67fc700 1 -- 172.21.15.16:0/3195780927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080992d0 con 0x5608042f3720 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f28f67fc700 1 -- 172.21.15.16:0/3195780927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29080b2260 con 0x5608042f3720 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f28f67fc700 1 -- 172.21.15.16:0/3195780927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908012340 con 0x5608042f3720 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 -- 172.21.15.16:0/3195780927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 msgr2=0x5608042f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x5608042f72b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f29080ca520 tx=0x7f2908009ef0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 -- 172.21.15.16:0/3195780927 shutdown_connections 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x5608042f72b0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3195780927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 -- 172.21.15.16:0/3195780927 >> 172.21.15.16:0/3195780927 conn(0x56080424b210 msgr2=0x5608042f1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.345+0000 7f29219d0b80 1 -- 172.21.15.16:0/3195780927 shutdown_connections 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 -- 172.21.15.16:0/3195780927 wait complete. 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 -- start start 2024-05-08T23:27:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 Processor -- start 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x56080435c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080435c640 con 0x5608042f3720 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080435c780 con 0x5608042ed890 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080435c910 con 0x5608042f2e00 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f29219d0b80 10 client.0 _ll_register_callbacks cb 0x5608041e2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:27:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.346+0000 7f290f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x56080435c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x56080435c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54442/0 (socket says 172.21.15.16:54442) 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 -- 172.21.15.16:0/3600250176 learned_addr learned my addr 172.21.15.16:0/3600250176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290effd700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290e7fc700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 -- 172.21.15.16:0/3600250176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 msgr2=0x5608042fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042fa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 -- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 msgr2=0x5608042fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 -- 172.21.15.16:0/3600250176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29101a6220 con 0x5608042f3720 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290e7fc700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f290f7fe700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x56080435c100 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2908095e70 tx=0x7f29080995e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442d3f4fdc7b4802 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f28eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f28eeffd700 1 -- 172.21.15.16:0/3600250176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908099dd0 con 0x5608042f3720 2024-05-08T23:27:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f28eeffd700 1 -- 172.21.15.16:0/3600250176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f290809d7a0 con 0x5608042f3720 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f290001e040 con 0x5608042f3720 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f28eeffd700 1 -- 172.21.15.16:0/3600250176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f290809a070 con 0x5608042f3720 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f29219d0b80 10 client.17418 fetch_fsmap learned FSMap version 25 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.347+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5608040c2cd0 con 0x5608042f3720 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f28eeffd700 1 -- 172.21.15.16:0/3600250176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2908082030 con 0x5608042f3720 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f28eeffd700 1 -- 172.21.15.16:0/3600250176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29080a18d0 con 0x5608042f3720 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f29219d0b80 10 client.17418 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f29219d0b80 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f29219d0b80 20 client.17418 populate_metadata read hostname 'smithi016' 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56080435dce0 0x560804360100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f29219d0b80 4 client.17418 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560804235ec0 con 0x56080435dce0 2024-05-08T23:27:58.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f290effd700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56080435dce0 0x560804360100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f28eeffd700 10 client.17418 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.348+0000 7f290effd700 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56080435dce0 0x560804360100 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b846721b50c93b5 server_cookie=be7bc706e6a2ed9a in_seq=0 out_seq=0 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f28eeffd700 1 -- 172.21.15.16:0/3600250176 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560804235ec0 con 0x56080435dce0 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f28eeffd700 10 client.17418 handle_command_reply: tid=0 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 client.17418 shutdown 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 msgr2=0x56080435c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x56080435c100 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2908095e70 tx=0x7f29080995e0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 shutdown_connections 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5608042f3720 0x56080435c100 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56080435dce0 0x560804360100 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5608042ed890 0x5608042fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f28eeffd700 0 client.17418 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 --2- 172.21.15.16:0/3600250176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608042f2e00 0x5608042fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 >> 172.21.15.16:0/3600250176 conn(0x56080424b210 msgr2=0x5608042f4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 shutdown_connections 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 1 -- 172.21.15.16:0/3600250176 wait complete. 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.349+0000 7f29219d0b80 20 client.17418 trim_cache size 0 max 16384 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: "ed045838-cf59-47cf-9e37-7304d5f9e21e": { 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ed045838-cf59-47cf-9e37-7304d5f9e21e", 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:27:58.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:27:58.357 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:27:58.357 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:27:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28f80a1310 msgr2=0x7f28f80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28f80a1310 0x7f28f80a37d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2900006010 tx=0x7f290000b260 comp rx=0 tx=0).stop 2024-05-08T23:27:58.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 msgr2=0x7f29101014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f29101014c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f290800ef30 tx=0x7f290800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 shutdown_connections 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28f80a1310 0x7f28f80a37d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2910105f40 0x7f29101014c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29100ff860 0x7f2910100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 --2- 172.21.15.16:0/4114077474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2910100220 0x7f2910100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 >> 172.21.15.16:0/4114077474 conn(0x7f29100751c0 msgr2=0x7f29100fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 shutdown_connections 2024-05-08T23:27:58.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:27:58.350+0000 7f2915816700 1 -- 172.21.15.16:0/4114077474 wait complete. 2024-05-08T23:27:58.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:57 smithi016 ceph-mon[42541]: pgmap v258: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-08T23:27:58.362 INFO:tasks.cephfs.filesystem:scrub status for tag:ed045838-cf59-47cf-9e37-7304d5f9e21e - {'path': '/', 'tag': 'ed045838-cf59-47cf-9e37-7304d5f9e21e', 'options': 'recursive,force'} 2024-05-08T23:27:58.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:57 smithi189 ceph-mon[46919]: pgmap v258: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-08T23:27:58.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:57 smithi155 ceph-mon[47105]: pgmap v258: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-08T23:28:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:59 smithi016 ceph-mon[42541]: pgmap v259: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-08T23:28:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:59 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:28:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:59 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:59 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:28:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:27:59 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:59 smithi189 ceph-mon[46919]: pgmap v259: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-08T23:28:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:59 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:28:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:59 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:59 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:28:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:27:59 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:59 smithi155 ceph-mon[47105]: pgmap v259: 129 pgs: 129 active+clean; 12 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 9 op/s 2024-05-08T23:28:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:59 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:28:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:59 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:59 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:28:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:27:59 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:01.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:00 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:28:01.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:00 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:28:01.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:00 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:28:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:01 smithi016 ceph-mon[42541]: pgmap v260: 129 pgs: 129 active+clean; 12 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-05-08T23:28:02.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:01 smithi189 ceph-mon[46919]: pgmap v260: 129 pgs: 129 active+clean; 12 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-05-08T23:28:02.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:01 smithi155 ceph-mon[47105]: pgmap v260: 129 pgs: 129 active+clean; 12 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-05-08T23:28:04.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:03 smithi016 ceph-mon[42541]: pgmap v261: 129 pgs: 129 active+clean; 12 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-05-08T23:28:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:03 smithi189 ceph-mon[46919]: pgmap v261: 129 pgs: 129 active+clean; 12 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-05-08T23:28:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:03 smithi155 ceph-mon[47105]: pgmap v261: 129 pgs: 129 active+clean; 12 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 10 op/s 2024-05-08T23:28:06.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:06 smithi016 ceph-mon[42541]: pgmap v262: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 101 KiB/s wr, 32 op/s 2024-05-08T23:28:06.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:06 smithi189 ceph-mon[46919]: pgmap v262: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 101 KiB/s wr, 32 op/s 2024-05-08T23:28:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:06 smithi155 ceph-mon[47105]: pgmap v262: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 101 KiB/s wr, 32 op/s 2024-05-08T23:28:07.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:07 smithi016 ceph-mon[42541]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:28:07.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:07 smithi189 ceph-mon[46919]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:28:07.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:07 smithi155 ceph-mon[47105]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T23:28:08.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:08 smithi016 ceph-mon[42541]: pgmap v264: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-05-08T23:28:08.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:08 smithi189 ceph-mon[46919]: pgmap v264: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-05-08T23:28:08.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:08 smithi155 ceph-mon[47105]: pgmap v264: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-05-08T23:28:10.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:10 smithi016 ceph-mon[42541]: pgmap v265: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-05-08T23:28:10.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:10 smithi189 ceph-mon[46919]: pgmap v265: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-05-08T23:28:10.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:10 smithi155 ceph-mon[47105]: pgmap v265: 129 pgs: 129 active+clean; 12 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-05-08T23:28:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:12 smithi016 ceph-mon[42541]: pgmap v266: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 32 op/s 2024-05-08T23:28:12.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:12 smithi189 ceph-mon[46919]: pgmap v266: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 32 op/s 2024-05-08T23:28:12.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:12 smithi155 ceph-mon[47105]: pgmap v266: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 32 op/s 2024-05-08T23:28:14.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:14 smithi016 ceph-mon[42541]: pgmap v267: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 32 op/s 2024-05-08T23:28:14.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:14 smithi016 ceph-mon[42541]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:28:14.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:14 smithi189 ceph-mon[46919]: pgmap v267: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 32 op/s 2024-05-08T23:28:14.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:14 smithi189 ceph-mon[46919]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:28:14.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:14 smithi155 ceph-mon[47105]: pgmap v267: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 105 KiB/s wr, 32 op/s 2024-05-08T23:28:14.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:14 smithi155 ceph-mon[47105]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T23:28:16.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:16 smithi016 ceph-mon[42541]: pgmap v269: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-08T23:28:16.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:16 smithi189 ceph-mon[46919]: pgmap v269: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-08T23:28:16.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:16 smithi155 ceph-mon[47105]: pgmap v269: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-08T23:28:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:18 smithi016 ceph-mon[42541]: pgmap v270: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T23:28:18.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:18 smithi189 ceph-mon[46919]: pgmap v270: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T23:28:18.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:18 smithi155 ceph-mon[47105]: pgmap v270: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T23:28:20.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:20 smithi016 ceph-mon[42541]: pgmap v271: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T23:28:20.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:20 smithi189 ceph-mon[46919]: pgmap v271: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T23:28:20.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:20 smithi155 ceph-mon[47105]: pgmap v271: 129 pgs: 129 active+clean; 13 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T23:28:22.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:22 smithi016 ceph-mon[42541]: pgmap v272: 129 pgs: 129 active+clean; 13 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:28:22.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:22 smithi189 ceph-mon[46919]: pgmap v272: 129 pgs: 129 active+clean; 13 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:28:22.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:22 smithi155 ceph-mon[47105]: pgmap v272: 129 pgs: 129 active+clean; 13 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:28:24.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:24 smithi016 ceph-mon[42541]: pgmap v273: 129 pgs: 129 active+clean; 13 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:28:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:24 smithi189 ceph-mon[46919]: pgmap v273: 129 pgs: 129 active+clean; 13 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:28:24.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:24 smithi155 ceph-mon[47105]: pgmap v273: 129 pgs: 129 active+clean; 13 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:28:26.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:26 smithi016 ceph-mon[42541]: pgmap v274: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-05-08T23:28:26.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:26 smithi189 ceph-mon[46919]: pgmap v274: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-05-08T23:28:26.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:26 smithi155 ceph-mon[47105]: pgmap v274: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-05-08T23:28:28.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:28:28.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:28 smithi016 ceph-mon[42541]: pgmap v275: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:28.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:28 smithi189 ceph-mon[46919]: pgmap v275: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:28.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:28 smithi155 ceph-mon[47105]: pgmap v275: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 -- 172.21.15.16:0/2696451064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24100a10 msgr2=0x7f5e24104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 --2- 172.21.15.16:0/2696451064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24100a10 0x7f5e24104a30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009990 tx=0x7f5e18009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 -- 172.21.15.16:0/2696451064 shutdown_connections 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 --2- 172.21.15.16:0/2696451064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e24105160 0x7f5e2410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 --2- 172.21.15.16:0/2696451064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e24100060 0x7f5e24100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 --2- 172.21.15.16:0/2696451064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24100a10 0x7f5e24104a30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 -- 172.21.15.16:0/2696451064 >> 172.21.15.16:0/2696451064 conn(0x7f5e24076f40 msgr2=0x7f5e24077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 -- 172.21.15.16:0/2696451064 shutdown_connections 2024-05-08T23:28:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.512+0000 7f5e2a917700 1 -- 172.21.15.16:0/2696451064 wait complete. 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 Processor -- start 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 -- start start 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e24100060 0x7f5e241962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 0x7f5e24196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e2419aaf0 0x7f5e2419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2419b510 con 0x7f5e24105160 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2419b680 con 0x7f5e24100060 2024-05-08T23:28:28.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e2a917700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2419b840 con 0x7f5e2419aaf0 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 0x7f5e24196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 0x7f5e24196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45928/0 (socket says 172.21.15.16:45928) 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e237fe700 1 -- 172.21.15.16:0/994775050 learned_addr learned my addr 172.21.15.16:0/994775050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.513+0000 7f5e23fff700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e24100060 0x7f5e241962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e2419aaf0 0x7f5e2419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e237fe700 1 -- 172.21.15.16:0/994775050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e2419aaf0 msgr2=0x7f5e2419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e237fe700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e2419aaf0 0x7f5e2419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e237fe700 1 -- 172.21.15.16:0/994775050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e24100060 msgr2=0x7f5e241962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e237fe700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e24100060 0x7f5e241962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e237fe700 1 -- 172.21.15.16:0/994775050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180095f0 con 0x7f5e24105160 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e237fe700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 0x7f5e24196810 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5e18005950 tx=0x7f5e18009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e217fa700 1 -- 172.21.15.16:0/994775050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1801d070 con 0x7f5e24105160 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e217fa700 1 -- 172.21.15.16:0/994775050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e1800fb00 con 0x7f5e24105160 2024-05-08T23:28:28.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e241a2dc0 con 0x7f5e24105160 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e217fa700 1 -- 172.21.15.16:0/994775050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18022ac0 con 0x7f5e24105160 2024-05-08T23:28:28.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.514+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e241a3190 con 0x7f5e24105160 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.516+0000 7f5e217fa700 1 -- 172.21.15.16:0/994775050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5e1800fc70 con 0x7f5e24105160 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.517+0000 7f5e217fa700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5e0c0a1290 0x7f5e0c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.517+0000 7f5e23fff700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5e0c0a1290 0x7f5e0c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.517+0000 7f5e217fa700 1 -- 172.21.15.16:0/994775050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5e180ce4a0 con 0x7f5e24105160 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e23fff700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5e0c0a1290 0x7f5e0c0a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e24104c00 tx=0x7f5e1400b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 Processor -- start 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 -- start start 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942e79150 con 0x555942f22890 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942e76b10 con 0x555942f27e00 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942e72400 con 0x555942f28720 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e28eb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e28eb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45934/0 (socket says 172.21.15.16:45934) 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.518+0000 7f5e28eb4700 1 -- 172.21.15.16:0/2654559010 learned_addr learned my addr 172.21.15.16:0/2654559010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e237fe700 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e28eb4700 1 -- 172.21.15.16:0/2654559010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 msgr2=0x555942f22350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f22350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e28eb4700 1 -- 172.21.15.16:0/2654559010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 msgr2=0x555942f281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e28eb4700 1 -- 172.21.15.16:0/2654559010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555942cf33b0 con 0x555942f22890 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e23fff700 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22cd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c003fa0 tx=0x7f5e1c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb09aef005b6a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e0a7fc700 1 -- 172.21.15.16:0/2654559010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c019070 con 0x555942f22890 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e0a7fc700 1 -- 172.21.15.16:0/2654559010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e1c00f040 con 0x555942f22890 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.519+0000 7f5e0a7fc700 1 -- 172.21.15.16:0/2654559010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0079a0 con 0x555942f22890 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2654559010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 msgr2=0x555942f22cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22cd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c003fa0 tx=0x7f5e1c009ba0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2654559010 shutdown_connections 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22cd0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2654559010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f22350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2654559010 >> 172.21.15.16:0/2654559010 conn(0x555942e80210 msgr2=0x555942f1cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2654559010 shutdown_connections 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2654559010 wait complete. 2024-05-08T23:28:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.520+0000 7f5e36ad1b80 1 -- start start 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 Processor -- start 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f2c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f2c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942e0e560 con 0x555942f27e00 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942e0e200 con 0x555942f28720 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942cf56a0 con 0x555942f22890 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e36ad1b80 10 client.0 _ll_register_callbacks cb 0x555942e17b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f2c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f2c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45936/0 (socket says 172.21.15.16:45936) 2024-05-08T23:28:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 -- 172.21.15.16:0/3752816110 learned_addr learned my addr 172.21.15.16:0/3752816110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f2c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e23fff700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 -- 172.21.15.16:0/3752816110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 msgr2=0x555942f2bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f2bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 -- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 msgr2=0x555942f2c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f2c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 -- 172.21.15.16:0/3752816110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e14010c20 con 0x555942f27e00 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f2c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e237fe700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f2c060 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5e1809b2d0 tx=0x7f5e1809add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3353a960aa010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.521+0000 7f5e037fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e037fe700 1 -- 172.21.15.16:0/3752816110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1809b340 con 0x555942f27e00 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e037fe700 1 -- 172.21.15.16:0/3752816110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5e1809f170 con 0x555942f27e00 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555942dcb270 con 0x555942f27e00 2024-05-08T23:28:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e037fe700 1 -- 172.21.15.16:0/3752816110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e180a3730 con 0x555942f27e00 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 10 client.17430 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f5e1c003bd0 con 0x555942f27e00 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e037fe700 1 -- 172.21.15.16:0/3752816110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555942dcb270 con 0x555942f27e00 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e037fe700 1 -- 172.21.15.16:0/3752816110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e180a32a0 con 0x555942f27e00 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 10 client.17430 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 20 client.17430 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f3bf00 0x555942f3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 4 client.17430 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.522+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555942ef1810 con 0x555942f3bf00 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.523+0000 7f5e23fff700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f3bf00 0x555942f3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.523+0000 7f5e23fff700 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f3bf00 0x555942f3e2f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee8fe548981200f server_cookie=197247cb55c63f14 in_seq=0 out_seq=0 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.523+0000 7f5e037fe700 10 client.17430 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e037fe700 1 -- 172.21.15.16:0/3752816110 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555942ef1810 con 0x555942f3bf00 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e037fe700 10 client.17430 handle_command_reply: tid=0 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 client.17430 shutdown 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 msgr2=0x555942f2c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f2c060 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5e1809b2d0 tx=0x7f5e1809add0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 shutdown_connections 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f3bf00 0x555942f3e2f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f2c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f2bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e037fe700 0 client.17430 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/3752816110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f2c060 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 >> 172.21.15.16:0/3752816110 conn(0x555942e80210 msgr2=0x555942f1c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 shutdown_connections 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/3752816110 wait complete. 2024-05-08T23:28:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.524+0000 7f5e36ad1b80 20 client.17430 trim_cache size 0 max 16384 2024-05-08T23:28:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 Processor -- start 2024-05-08T23:28:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 -- start start 2024-05-08T23:28:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f1e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f37c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942dcb270 con 0x555942f22890 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942f2e6b0 con 0x555942f27e00 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942f43500 con 0x555942f28720 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45952/0 (socket says 172.21.15.16:45952) 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 -- 172.21.15.16:0/2913677361 learned_addr learned my addr 172.21.15.16:0/2913677361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e237fe700 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f1e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f37c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 -- 172.21.15.16:0/2913677361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 msgr2=0x555942f37c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f37c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 -- 172.21.15.16:0/2913677361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 msgr2=0x555942f1e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f1e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.543+0000 7f5e23fff700 1 -- 172.21.15.16:0/2913677361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555942c9fb60 con 0x555942f22890 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e23fff700 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22350 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005950 tx=0x7f5e14013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d9b62d512ab546 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e0affd700 1 -- 172.21.15.16:0/2913677361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14026070 con 0x555942f22890 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e0affd700 1 -- 172.21.15.16:0/2913677361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e14015070 con 0x555942f22890 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e0affd700 1 -- 172.21.15.16:0/2913677361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e140116a0 con 0x555942f22890 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2913677361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 msgr2=0x555942f22350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22350 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005950 tx=0x7f5e14013000 comp rx=0 tx=0).stop 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2913677361 shutdown_connections 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f28720 0x555942f37c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f22890 0x555942f22350 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/2913677361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f27e00 0x555942f1e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.544+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2913677361 >> 172.21.15.16:0/2913677361 conn(0x555942e80210 msgr2=0x555942f2a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.545+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2913677361 shutdown_connections 2024-05-08T23:28:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.545+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/2913677361 wait complete. 2024-05-08T23:28:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.545+0000 7f5e36ad1b80 1 -- start start 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.545+0000 7f5e36ad1b80 1 Processor -- start 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f88370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942f89020 con 0x555942f27e00 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942f89190 con 0x555942f28720 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555942f93930 con 0x555942f22890 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e36ad1b80 10 client.0 _ll_register_callbacks cb 0x555942e17b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45966/0 (socket says 172.21.15.16:45966) 2024-05-08T23:28:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 -- 172.21.15.16:0/1759048092 learned_addr learned my addr 172.21.15.16:0/1759048092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e28eb4700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e23fff700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 -- 172.21.15.16:0/1759048092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 msgr2=0x555942f88370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f88370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 -- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 msgr2=0x555942f933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 -- 172.21.15.16:0/1759048092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e140215f0 con 0x555942f27e00 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e237fe700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f888b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5e18099690 tx=0x7f5e1809b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a065b8e86d798d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.546+0000 7f5e037fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e037fe700 1 -- 172.21.15.16:0/1759048092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1809f170 con 0x555942f27e00 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e037fe700 1 -- 172.21.15.16:0/1759048092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5e1809b340 con 0x555942f27e00 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e1c00f040 con 0x555942f27e00 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e037fe700 1 -- 172.21.15.16:0/1759048092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e180a1150 con 0x555942f27e00 2024-05-08T23:28:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 10 client.17436 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555942e38560 con 0x555942f27e00 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e037fe700 1 -- 172.21.15.16:0/1759048092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e1c00f040 con 0x555942f27e00 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e037fe700 1 -- 172.21.15.16:0/1759048092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e18097130 con 0x555942f27e00 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 10 client.17436 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 20 client.17436 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f94c00 0x555942f96ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 4 client.17436 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555942e6aec0 con 0x555942f94c00 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.547+0000 7f5e23fff700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f94c00 0x555942f96ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e037fe700 10 client.17436 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e23fff700 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f94c00 0x555942f96ff0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3169c4296829bbc server_cookie=802a25801d61a602 in_seq=0 out_seq=0 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e037fe700 1 -- 172.21.15.16:0/1759048092 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555942e6aec0 con 0x555942f94c00 2024-05-08T23:28:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e037fe700 10 client.17436 handle_command_reply: tid=0 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 client.17436 shutdown 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 msgr2=0x555942f888b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f888b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5e18099690 tx=0x7f5e1809b1c0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 shutdown_connections 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555942f94c00 0x555942f96ff0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555942f28720 0x555942f933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555942f22890 0x555942f88370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 --2- 172.21.15.16:0/1759048092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555942f27e00 0x555942f888b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e037fe700 0 client.17436 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 >> 172.21.15.16:0/1759048092 conn(0x555942e80210 msgr2=0x555942f2a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 shutdown_connections 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.548+0000 7f5e36ad1b80 1 -- 172.21.15.16:0/1759048092 wait complete. 2024-05-08T23:28:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e36ad1b80 20 client.17436 trim_cache size 0 max 16384 2024-05-08T23:28:28.556 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:28:28.556 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:28:28.556 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:28:28.556 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5e0c0a1290 msgr2=0x7f5e0c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5e0c0a1290 0x7f5e0c0a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e24104c00 tx=0x7f5e1400b410 comp rx=0 tx=0).stop 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 msgr2=0x7f5e24196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 0x7f5e24196810 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f5e18005950 tx=0x7f5e18009580 comp rx=0 tx=0).stop 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 shutdown_connections 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5e0c0a1290 0x7f5e0c0a3750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e2419aaf0 0x7f5e2419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e24100060 0x7f5e241962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 --2- 172.21.15.16:0/994775050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e24105160 0x7f5e24196810 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 >> 172.21.15.16:0/994775050 conn(0x7f5e24076f40 msgr2=0x7f5e240fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 shutdown_connections 2024-05-08T23:28:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.549+0000 7f5e2a917700 1 -- 172.21.15.16:0/994775050 wait complete. 2024-05-08T23:28:28.563 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:28:28.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 -- 172.21.15.16:0/1115145655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83041006b0 msgr2=0x7f8304104b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 --2- 172.21.15.16:0/1115145655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83041006b0 0x7f8304104b70 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009990 tx=0x7f82f4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 -- 172.21.15.16:0/1115145655 shutdown_connections 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 --2- 172.21.15.16:0/1115145655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8304105190 0x7f8304105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 --2- 172.21.15.16:0/1115145655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83040ffd90 0x7f8304100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 --2- 172.21.15.16:0/1115145655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83041006b0 0x7f8304104b70 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 -- 172.21.15.16:0/1115145655 >> 172.21.15.16:0/1115145655 conn(0x7f83040fb7e0 msgr2=0x7f83040fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 -- 172.21.15.16:0/1115145655 shutdown_connections 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.700+0000 7f830bdb1700 1 -- 172.21.15.16:0/1115145655 wait complete. 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 Processor -- start 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 -- start start 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83040ffd90 0x7f83041a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 0x7f83041a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 0x7f830419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830419dcb0 con 0x7f8304105190 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830419de20 con 0x7f83040ffd90 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830bdb1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830419dfe0 con 0x7f83041006b0 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.701+0000 7f830a34e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 0x7f830419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830934c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 0x7f83041a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830934c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 0x7f83041a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:60522/0 (socket says 172.21.15.16:60522) 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830934c700 1 -- 172.21.15.16:0/3546417021 learned_addr learned my addr 172.21.15.16:0/3546417021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 0x7f830419d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45976/0 (socket says 172.21.15.16:45976) 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f8309b4d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83040ffd90 0x7f83041a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 -- 172.21.15.16:0/3546417021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 msgr2=0x7f83041a3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 0x7f83041a3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 -- 172.21.15.16:0/3546417021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83040ffd90 msgr2=0x7f83041a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83040ffd90 0x7f83041a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 -- 172.21.15.16:0/3546417021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40095f0 con 0x7f8304105190 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830934c700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 0x7f83041a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830a34e700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 0x7f830419d650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00ab40 tx=0x7f82fc00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00f070 con 0x7f8304105190 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830419e2c0 con 0x7f8304105190 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82fc007a70 con 0x7f8304105190 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc005280 con 0x7f8304105190 2024-05-08T23:28:28.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.702+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83041aa510 con 0x7f8304105190 2024-05-08T23:28:28.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.703+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83040688d0 con 0x7f8304105190 2024-05-08T23:28:28.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.705+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f82fc004ad0 con 0x7f8304105190 2024-05-08T23:28:28.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.705+0000 7f82faffd700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f82f00a1290 0x7f82f00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:28.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.705+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f82fc0cc9f0 con 0x7f8304105190 2024-05-08T23:28:28.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.705+0000 7f8309b4d700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f82f00a1290 0x7f82f00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:28.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.706+0000 7f8309b4d700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f82f00a1290 0x7f82f00a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8300007be0 tx=0x7f83000073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:28.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.709+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82fc095b30 con 0x7f8304105190 2024-05-08T23:28:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.982+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8304067c10 con 0x7f8304105190 2024-05-08T23:28:29.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.983+0000 7f82faffd700 1 -- 172.21.15.16:0/3546417021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f82fc099150 con 0x7f8304105190 2024-05-08T23:28:29.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:28:29.023 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:28:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f82f00a1290 msgr2=0x7f82f00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f82f00a1290 0x7f82f00a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8300007be0 tx=0x7f83000073d0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 msgr2=0x7f830419d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 0x7f830419d650 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00ab40 tx=0x7f82fc00ae50 comp rx=0 tx=0).stop 2024-05-08T23:28:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 shutdown_connections 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f82f00a1290 0x7f82f00a3750 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8304105190 0x7f830419d650 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83040ffd90 0x7f83041a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 --2- 172.21.15.16:0/3546417021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83041006b0 0x7f83041a3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 >> 172.21.15.16:0/3546417021 conn(0x7f83040fb7e0 msgr2=0x7f83040fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 shutdown_connections 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:28.986+0000 7f830bdb1700 1 -- 172.21.15.16:0/3546417021 wait complete. 2024-05-08T23:28:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:28:29.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:28:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 -- 172.21.15.16:0/343753664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c06a120 msgr2=0x7f259c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 --2- 172.21.15.16:0/343753664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c06a120 0x7f259c105b50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2584009990 tx=0x7f2584009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 -- 172.21.15.16:0/343753664 shutdown_connections 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 --2- 172.21.15.16:0/343753664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f259c106280 0x7f259c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 --2- 172.21.15.16:0/343753664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c069770 0x7f259c069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 --2- 172.21.15.16:0/343753664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c06a120 0x7f259c105b50 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 -- 172.21.15.16:0/343753664 >> 172.21.15.16:0/343753664 conn(0x7f259c076fa0 msgr2=0x7f259c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 -- 172.21.15.16:0/343753664 shutdown_connections 2024-05-08T23:28:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.167+0000 7f25a19f0700 1 -- 172.21.15.16:0/343753664 wait complete. 2024-05-08T23:28:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.168+0000 7f25a19f0700 1 Processor -- start 2024-05-08T23:28:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.168+0000 7f25a19f0700 1 -- start start 2024-05-08T23:28:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a19f0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f259c069770 0x7f259c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a19f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 0x7f259c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a19f0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c19bc00 0x7f259c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a19f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19c680 con 0x7f259c106280 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a19f0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19c7f0 con 0x7f259c19bc00 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a19f0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19c9b0 con 0x7f259c069770 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 0x7f259c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 0x7f259c197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46000/0 (socket says 172.21.15.16:46000) 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 -- 172.21.15.16:0/2763463293 learned_addr learned my addr 172.21.15.16:0/2763463293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f259affd700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f259c069770 0x7f259c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c19bc00 0x7f259c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 -- 172.21.15.16:0/2763463293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f259c069770 msgr2=0x7f259c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f259c069770 0x7f259c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 -- 172.21.15.16:0/2763463293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c19bc00 msgr2=0x7f259c19c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c19bc00 0x7f259c19c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 -- 172.21.15.16:0/2763463293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840095f0 con 0x7f259c106280 2024-05-08T23:28:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c19bc00 0x7f259c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25927fc700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 0x7f259c197860 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f25840095a0 tx=0x7f2584005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.169+0000 7f25a09ee700 1 -- 172.21.15.16:0/2763463293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258401d070 con 0x7f259c106280 2024-05-08T23:28:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.170+0000 7f25a09ee700 1 -- 172.21.15.16:0/2763463293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2584005030 con 0x7f259c106280 2024-05-08T23:28:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.170+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c1a3eb0 con 0x7f259c106280 2024-05-08T23:28:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.170+0000 7f25a09ee700 1 -- 172.21.15.16:0/2763463293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584004a10 con 0x7f259c106280 2024-05-08T23:28:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.170+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c1a4270 con 0x7f259c106280 2024-05-08T23:28:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.172+0000 7f25a09ee700 1 -- 172.21.15.16:0/2763463293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f25840051a0 con 0x7f259c106280 2024-05-08T23:28:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.173+0000 7f25a09ee700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25880a1290 0x7f25880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.173+0000 7f259affd700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25880a1290 0x7f25880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.173+0000 7f25a09ee700 1 -- 172.21.15.16:0/2763463293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f25840cdff0 con 0x7f259c106280 2024-05-08T23:28:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.173+0000 7f259affd700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25880a1290 0x7f25880a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f258c009c60 tx=0x7f258c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 Processor -- start 2024-05-08T23:28:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 -- start start 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3ce00 0x560137a3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3d720 0x560137a37350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a37cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013798e150 con 0x560137a37890 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013798bb10 con 0x560137a3ce00 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25adbaab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137987400 con 0x560137a3d720 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f259b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a37cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f259b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a37cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46012/0 (socket says 172.21.15.16:46012) 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f259affd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3ce00 0x560137a3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f259b7fe700 1 -- 172.21.15.16:0/2704365112 learned_addr learned my addr 172.21.15.16:0/2704365112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.174+0000 7f25927fc700 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3d720 0x560137a37350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259affd700 1 -- 172.21.15.16:0/2704365112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3d720 msgr2=0x560137a37350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259affd700 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3d720 0x560137a37350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259affd700 1 -- 172.21.15.16:0/2704365112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 msgr2=0x560137a37cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259affd700 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a37cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259affd700 1 -- 172.21.15.16:0/2704365112 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601378083b0 con 0x560137a3ce00 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a37cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f259affd700 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3ce00 0x560137a3d1e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e3b0 tx=0x7f258c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fec49bb0436ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f2591ffb700 1 -- 172.21.15.16:0/2704365112 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00ed50 con 0x560137a3ce00 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f2591ffb700 1 -- 172.21.15.16:0/2704365112 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f258c021070 con 0x560137a3ce00 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.175+0000 7f2591ffb700 1 -- 172.21.15.16:0/2704365112 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c003dd0 con 0x560137a3ce00 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 -- 172.21.15.16:0/2704365112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3ce00 msgr2=0x560137a3d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3ce00 0x560137a3d1e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e3b0 tx=0x7f258c014000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 -- 172.21.15.16:0/2704365112 shutdown_connections 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a37cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3ce00 0x560137a3d1e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2704365112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3d720 0x560137a37350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 -- 172.21.15.16:0/2704365112 >> 172.21.15.16:0/2704365112 conn(0x560137995210 msgr2=0x560137a31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 -- 172.21.15.16:0/2704365112 shutdown_connections 2024-05-08T23:28:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 -- 172.21.15.16:0/2704365112 wait complete. 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.176+0000 7f25adbaab80 1 -- start start 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 Processor -- start 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137a41640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137923560 con 0x560137a37890 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137923200 con 0x560137a3d720 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56013780a6a0 con 0x560137a3ce00 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25adbaab80 10 client.0 _ll_register_callbacks cb 0x56013792cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a40bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46026/0 (socket says 172.21.15.16:46026) 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 -- 172.21.15.16:0/2737089194 learned_addr learned my addr 172.21.15.16:0/2737089194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f25927fc700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137a41640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 -- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 msgr2=0x560137a41100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 -- 172.21.15.16:0/2737089194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 msgr2=0x560137a41640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137a41640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259affd700 1 -- 172.21.15.16:0/2737089194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25940090f0 con 0x560137a37890 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.177+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137a41640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f259affd700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a40bc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f258c009b20 tx=0x7f258c003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddb495057e7c30c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f257affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f257affd700 1 -- 172.21.15.16:0/2737089194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02a040 con 0x560137a37890 2024-05-08T23:28:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f257affd700 1 -- 172.21.15.16:0/2737089194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f258c0052c0 con 0x560137a37890 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601378e0270 con 0x560137a37890 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f257affd700 1 -- 172.21.15.16:0/2737089194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c003e80 con 0x560137a37890 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 10 client.17454 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f258c017070 con 0x560137a37890 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f257affd700 1 -- 172.21.15.16:0/2737089194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5601378e0270 con 0x560137a37890 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f257affd700 1 -- 172.21.15.16:0/2737089194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f258c02f430 con 0x560137a37890 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 10 client.17454 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 10 client.17454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 20 client.17454 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137a51130 0x560137a53550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 4 client.17454 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.178+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560137a06810 con 0x560137a51130 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.179+0000 7f25927fc700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137a51130 0x560137a53550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.179+0000 7f25927fc700 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137a51130 0x560137a53550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d838b9b38790881 server_cookie=e5cec31b3b9ef4aa in_seq=0 out_seq=0 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.179+0000 7f257affd700 10 client.17454 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f257affd700 1 -- 172.21.15.16:0/2737089194 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560137a06810 con 0x560137a51130 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f257affd700 10 client.17454 handle_command_reply: tid=0 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 client.17454 shutdown 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 msgr2=0x560137a40bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a40bc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f258c009b20 tx=0x7f258c003880 comp rx=0 tx=0).stop 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 shutdown_connections 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137a51130 0x560137a53550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137a41640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a37890 0x560137a40bc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f257affd700 0 client.17454 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2737089194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 >> 172.21.15.16:0/2737089194 conn(0x560137995210 msgr2=0x560137a315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 shutdown_connections 2024-05-08T23:28:29.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 1 -- 172.21.15.16:0/2737089194 wait complete. 2024-05-08T23:28:29.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.180+0000 7f25adbaab80 20 client.17454 trim_cache size 0 max 16384 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 Processor -- start 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 -- start start 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a37890 0x560137a31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 0x560137a48160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137a50f50 con 0x560137a3d720 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137a421c0 con 0x560137a37890 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.194+0000 7f25adbaab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137a58760 con 0x560137a3ce00 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 0x560137a48160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 0x560137a48160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46038/0 (socket says 172.21.15.16:46038) 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 -- 172.21.15.16:0/2189738733 learned_addr learned my addr 172.21.15.16:0/2189738733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f25927fc700 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259affd700 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a37890 0x560137a31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 -- 172.21.15.16:0/2189738733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 msgr2=0x560137a41460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 -- 172.21.15.16:0/2189738733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a37890 msgr2=0x560137a31d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a37890 0x560137a31d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 -- 172.21.15.16:0/2189738733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601377b4b60 con 0x560137a3d720 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 0x560137a48160 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2594007770 tx=0x7f259400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310bc29a31fbc41f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f2592ffd700 1 -- 172.21.15.16:0/2189738733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400e030 con 0x560137a3d720 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f2592ffd700 1 -- 172.21.15.16:0/2189738733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f259400f040 con 0x560137a3d720 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.195+0000 7f2592ffd700 1 -- 172.21.15.16:0/2189738733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25940054c0 con 0x560137a3d720 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 -- 172.21.15.16:0/2189738733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 msgr2=0x560137a48160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 0x560137a48160 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2594007770 tx=0x7f259400b040 comp rx=0 tx=0).stop 2024-05-08T23:28:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 -- 172.21.15.16:0/2189738733 shutdown_connections 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3d720 0x560137a48160 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a37890 0x560137a31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2189738733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a3ce00 0x560137a41460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 -- 172.21.15.16:0/2189738733 >> 172.21.15.16:0/2189738733 conn(0x560137995210 msgr2=0x560137a37c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 -- 172.21.15.16:0/2189738733 shutdown_connections 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 -- 172.21.15.16:0/2189738733 wait complete. 2024-05-08T23:28:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.196+0000 7f25adbaab80 1 -- start start 2024-05-08T23:28:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 Processor -- start 2024-05-08T23:28:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a37890 0x560137a46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 0x560137a46880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137aa6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137aa6b70 con 0x560137a3ce00 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137aa6cb0 con 0x560137a3d720 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560137aa6df0 con 0x560137a37890 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25adbaab80 10 client.0 _ll_register_callbacks cb 0x56013792cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 0x560137a46880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 0x560137a46880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46054/0 (socket says 172.21.15.16:46054) 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 -- 172.21.15.16:0/2776555331 learned_addr learned my addr 172.21.15.16:0/2776555331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f259b7fe700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137aa6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f259affd700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a37890 0x560137a46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 -- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a37890 msgr2=0x560137a46340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a37890 0x560137a46340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 -- 172.21.15.16:0/2776555331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 msgr2=0x560137aa6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137aa6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.197+0000 7f25927fc700 1 -- 172.21.15.16:0/2776555331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2594007500 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25927fc700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 0x560137a46880 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2584005450 tx=0x7f2584097220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4733f37c2bf3ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f257affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f257affd700 1 -- 172.21.15.16:0/2776555331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840056a0 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f257affd700 1 -- 172.21.15.16:0/2776555331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f25840666b0 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f258c021070 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f257affd700 1 -- 172.21.15.16:0/2776555331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258409b010 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 10 client.17460 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56013794d560 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f257affd700 1 -- 172.21.15.16:0/2776555331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f258c021070 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f257affd700 1 -- 172.21.15.16:0/2776555331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f25840b8f90 con 0x560137a3ce00 2024-05-08T23:28:29.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 10 client.17460 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 10 client.17460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 20 client.17460 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137aa82d0 0x560137aaa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 4 client.17460 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.198+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56013797fec0 con 0x560137aa82d0 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f259affd700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137aa82d0 0x560137aaa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f259affd700 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137aa82d0 0x560137aaa6c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a2fd2a02a9b70d2 server_cookie=b622ec4385019e07 in_seq=0 out_seq=0 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f257affd700 10 client.17460 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f257affd700 1 -- 172.21.15.16:0/2776555331 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56013797fec0 con 0x560137aa82d0 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f257affd700 10 client.17460 handle_command_reply: tid=0 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 client.17460 shutdown 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 msgr2=0x560137a46880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 0x560137a46880 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2584005450 tx=0x7f2584097220 comp rx=0 tx=0).stop 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 shutdown_connections 2024-05-08T23:28:29.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x560137aa82d0 0x560137aaa6c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560137a3d720 0x560137aa6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560137a37890 0x560137a46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 --2- 172.21.15.16:0/2776555331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560137a3ce00 0x560137a46880 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f257affd700 0 client.17460 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.199+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 >> 172.21.15.16:0/2776555331 conn(0x560137995210 msgr2=0x560137a3bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 shutdown_connections 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25adbaab80 1 -- 172.21.15.16:0/2776555331 wait complete. 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25adbaab80 20 client.17460 trim_cache size 0 max 16384 2024-05-08T23:28:29.206 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25880a1290 msgr2=0x7f25880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25880a1290 0x7f25880a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f258c009c60 tx=0x7f258c009260 comp rx=0 tx=0).stop 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 msgr2=0x7f259c197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 0x7f259c197860 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f25840095a0 tx=0x7f2584005e80 comp rx=0 tx=0).stop 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 shutdown_connections 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f25880a1290 0x7f25880a3750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f259c19bc00 0x7f259c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f259c069770 0x7f259c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 --2- 172.21.15.16:0/2763463293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f259c106280 0x7f259c197860 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 >> 172.21.15.16:0/2763463293 conn(0x7f259c076fa0 msgr2=0x7f259c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 shutdown_connections 2024-05-08T23:28:29.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.200+0000 7f25a19f0700 1 -- 172.21.15.16:0/2763463293 wait complete. 2024-05-08T23:28:29.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:28:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 -- 172.21.15.16:0/3541876515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0105180 msgr2=0x7f08d010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 --2- 172.21.15.16:0/3541876515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0105180 0x7f08d010cd00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b0a0 tx=0x7f08c800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 -- 172.21.15.16:0/3541876515 shutdown_connections 2024-05-08T23:28:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 --2- 172.21.15.16:0/3541876515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0105180 0x7f08d010cd00 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 --2- 172.21.15.16:0/3541876515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0100060 0x7f08d0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 --2- 172.21.15.16:0/3541876515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0100a10 0x7f08d0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 -- 172.21.15.16:0/3541876515 >> 172.21.15.16:0/3541876515 conn(0x7f08d0076f40 msgr2=0x7f08d0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 -- 172.21.15.16:0/3541876515 shutdown_connections 2024-05-08T23:28:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.353+0000 7f08d590c700 1 -- 172.21.15.16:0/3541876515 wait complete. 2024-05-08T23:28:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 Processor -- start 2024-05-08T23:28:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 -- start start 2024-05-08T23:28:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 0x7f08d0101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0105180 0x7f08d0102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0102570 0x7f08d01a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01a5bc0 con 0x7f08d0100060 2024-05-08T23:28:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01a5d30 con 0x7f08d0102570 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08d590c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01a5ef0 con 0x7f08d0105180 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 0x7f08d0101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 0x7f08d0101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46080/0 (socket says 172.21.15.16:46080) 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08ceffd700 1 -- 172.21.15.16:0/1675865738 learned_addr learned my addr 172.21.15.16:0/1675865738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.354+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0105180 0x7f08d0102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0102570 0x7f08d01a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08ceffd700 1 -- 172.21.15.16:0/1675865738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0105180 msgr2=0x7f08d0102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08ceffd700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0105180 0x7f08d0102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08ceffd700 1 -- 172.21.15.16:0/1675865738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0102570 msgr2=0x7f08d01a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08ceffd700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0102570 0x7f08d01a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08ceffd700 1 -- 172.21.15.16:0/1675865738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009d00 con 0x7f08d0100060 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0102570 0x7f08d01a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08ceffd700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 0x7f08d0101af0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e9a0 tx=0x7f08c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08d490a700 1 -- 172.21.15.16:0/1675865738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0009790 con 0x7f08d0100060 2024-05-08T23:28:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d01a61d0 con 0x7f08d0100060 2024-05-08T23:28:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08d490a700 1 -- 172.21.15.16:0/1675865738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c000cbd0 con 0x7f08d0100060 2024-05-08T23:28:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08d490a700 1 -- 172.21.15.16:0/1675865738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0018680 con 0x7f08d0100060 2024-05-08T23:28:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.355+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01a66e0 con 0x7f08d0100060 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.357+0000 7f08d490a700 1 -- 172.21.15.16:0/1675865738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f08c000cd40 con 0x7f08d0100060 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.358+0000 7f08d490a700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f08bc0a1290 0x7f08bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.358+0000 7f08d490a700 1 -- 172.21.15.16:0/1675865738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f08c0014070 con 0x7f08d0100060 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.358+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f08bc0a1290 0x7f08bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.359+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f08bc0a1290 0x7f08bc0a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f08d0103220 tx=0x7f08b8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.359+0000 7f08e1ac6b80 1 Processor -- start 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.359+0000 7f08e1ac6b80 1 -- start start 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.359+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 0x560e7715e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715e720 0x560e77158350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e770af150 con 0x560e7715de00 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e770acb10 con 0x560e77158890 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e770a8400 con 0x560e7715e720 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 0x560e7715e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 0x560e7715e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46086/0 (socket says 172.21.15.16:46086) 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 -- 172.21.15.16:0/2708629490 learned_addr learned my addr 172.21.15.16:0/2708629490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715e720 0x560e77158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 -- 172.21.15.16:0/2708629490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715e720 msgr2=0x560e77158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715e720 0x560e77158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 -- 172.21.15.16:0/2708629490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 msgr2=0x560e77158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 -- 172.21.15.16:0/2708629490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e76f293b0 con 0x560e7715de00 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 0x560e7715e1e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f08c0096ae0 tx=0x7f08c00999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954a972852eddd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08c5ffb700 1 -- 172.21.15.16:0/2708629490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00cd160 con 0x560e7715de00 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.360+0000 7f08c5ffb700 1 -- 172.21.15.16:0/2708629490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c009a070 con 0x560e7715de00 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08c5ffb700 1 -- 172.21.15.16:0/2708629490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0099560 con 0x560e7715de00 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2708629490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 msgr2=0x560e7715e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 0x560e7715e1e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f08c0096ae0 tx=0x7f08c00999d0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2708629490 shutdown_connections 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715de00 0x560e7715e1e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2708629490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715e720 0x560e77158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2708629490 >> 172.21.15.16:0/2708629490 conn(0x560e770b6210 msgr2=0x560e77152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2708629490 shutdown_connections 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.361+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2708629490 wait complete. 2024-05-08T23:28:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 -- start start 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 Processor -- start 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e77161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77162010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e77162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e77044560 con 0x560e77158890 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e77044200 con 0x560e7715e720 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e76f2b6a0 con 0x560e7715de00 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08e1ac6b80 10 client.0 _ll_register_callbacks cb 0x560e7704db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.362+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e77161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e77161ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46088/0 (socket says 172.21.15.16:46088) 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 -- 172.21.15.16:0/2500398327 learned_addr learned my addr 172.21.15.16:0/2500398327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 -- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 msgr2=0x560e77162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77162010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 -- 172.21.15.16:0/2500398327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 msgr2=0x560e77162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e77162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 -- 172.21.15.16:0/2500398327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01a66e0 con 0x560e77158890 2024-05-08T23:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:29 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3546417021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77162010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e77161ad0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cac30 tx=0x7f08c000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312d54d26982f217 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08aeffd700 1 -- 172.21.15.16:0/2500398327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00a24f0 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08aeffd700 1 -- 172.21.15.16:0/2500398327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f08c009a4e0 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.363+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e77001270 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08aeffd700 1 -- 172.21.15.16:0/2500398327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c00b82b0 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 10 client.17472 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560e76ed5b60 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08aeffd700 1 -- 172.21.15.16:0/2500398327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560e77001270 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08aeffd700 1 -- 172.21.15.16:0/2500398327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08c0095e60 con 0x560e77158890 2024-05-08T23:28:29.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 10 client.17472 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 10 client.17472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 20 client.17472 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e77171ef0 0x560e77174310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 4 client.17472 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e77127810 con 0x560e77171ef0 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.364+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e77171ef0 0x560e77174310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.365+0000 7f08aeffd700 10 client.17472 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.365+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e77171ef0 0x560e77174310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2dd586a583d877b server_cookie=27c7a742aa4e0cd3 in_seq=0 out_seq=0 2024-05-08T23:28:29.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.365+0000 7f08aeffd700 1 -- 172.21.15.16:0/2500398327 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e77127810 con 0x560e77171ef0 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.365+0000 7f08aeffd700 10 client.17472 handle_command_reply: tid=0 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.365+0000 7f08e1ac6b80 1 client.17472 shutdown 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 msgr2=0x560e77161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e77161ad0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cac30 tx=0x7f08c000ef70 comp rx=0 tx=0).stop 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 shutdown_connections 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e77171ef0 0x560e77174310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e77162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e77161ad0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08aeffd700 0 client.17472 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2500398327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77162010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 >> 172.21.15.16:0/2500398327 conn(0x560e770b6210 msgr2=0x560e771525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 shutdown_connections 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2500398327 wait complete. 2024-05-08T23:28:29.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.366+0000 7f08e1ac6b80 20 client.17472 trim_cache size 0 max 16384 2024-05-08T23:28:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 Processor -- start 2024-05-08T23:28:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 -- start start 2024-05-08T23:28:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77152d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 0x560e7715d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e77179500 con 0x560e7715e720 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e77154a00 con 0x560e77158890 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e7714d800 con 0x560e7715de00 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 0x560e7715d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 0x560e7715d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46100/0 (socket says 172.21.15.16:46100) 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 -- 172.21.15.16:0/4102300523 learned_addr learned my addr 172.21.15.16:0/4102300523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08ceffd700 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 -- 172.21.15.16:0/4102300523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 msgr2=0x560e77152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 -- 172.21.15.16:0/4102300523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 msgr2=0x560e77158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08cf7fe700 1 -- 172.21.15.16:0/4102300523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e7706e560 con 0x560e7715e720 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.380+0000 7f08ceffd700 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.381+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 0x560e7715d2d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009cc0 tx=0x7f08c80092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994a7ce46d737055 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.381+0000 7f08c67fc700 1 -- 172.21.15.16:0/4102300523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800e040 con 0x560e7715e720 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.381+0000 7f08c67fc700 1 -- 172.21.15.16:0/4102300523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c8009480 con 0x560e7715e720 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.381+0000 7f08c67fc700 1 -- 172.21.15.16:0/4102300523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8004e50 con 0x560e7715e720 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.381+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/4102300523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 msgr2=0x560e7715d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.381+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 0x560e7715d2d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009cc0 tx=0x7f08c80092e0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/4102300523 shutdown_connections 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7715e720 0x560e7715d2d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e77158890 0x560e77158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/4102300523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/4102300523 >> 172.21.15.16:0/4102300523 conn(0x560e770b6210 msgr2=0x560e7715cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/4102300523 shutdown_connections 2024-05-08T23:28:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/4102300523 wait complete. 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 -- start start 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.382+0000 7f08e1ac6b80 1 Processor -- start 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e771650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77165620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e771c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e77165f00 con 0x560e77158890 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e771c7900 con 0x560e7715e720 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e771c7a40 con 0x560e7715de00 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 10 client.0 _ll_register_callbacks cb 0x560e7704db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e771650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e771650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46116/0 (socket says 172.21.15.16:46116) 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 -- 172.21.15.16:0/2385016326 learned_addr learned my addr 172.21.15.16:0/2385016326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08cf7fe700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e771c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 -- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 msgr2=0x560e77165620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77165620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 -- 172.21.15.16:0/2385016326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 msgr2=0x560e771c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e771c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 -- 172.21.15.16:0/2385016326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d010d570 con 0x560e77158890 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08ceffd700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e771650e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f08c00961f0 tx=0x7f08c00ca790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32898d467f0ea104 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08aeffd700 1 -- 172.21.15.16:0/2385016326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0099340 con 0x560e77158890 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08aeffd700 1 -- 172.21.15.16:0/2385016326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f08c0005680 con 0x560e77158890 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.383+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08c009a070 con 0x560e77158890 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08aeffd700 1 -- 172.21.15.16:0/2385016326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c00a24f0 con 0x560e77158890 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 10 client.17478 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560e76f2dcd0 con 0x560e77158890 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08aeffd700 1 -- 172.21.15.16:0/2385016326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08c009a070 con 0x560e77158890 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08aeffd700 1 -- 172.21.15.16:0/2385016326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08c00a2060 con 0x560e77158890 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 10 client.17478 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 10 client.17478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 20 client.17478 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e771c8d50 0x560e771cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 4 client.17478 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e770a0ec0 con 0x560e771c8d50 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.384+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e771c8d50 0x560e771cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08ce7fc700 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e771c8d50 0x560e771cb140 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28047e8dde0fa9bf server_cookie=33113517912516f9 in_seq=0 out_seq=0 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08aeffd700 10 client.17478 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08aeffd700 1 -- 172.21.15.16:0/2385016326 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e770a0ec0 con 0x560e771c8d50 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08aeffd700 10 client.17478 handle_command_reply: tid=0 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 client.17478 shutdown 2024-05-08T23:28:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 msgr2=0x560e771650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e771650e0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f08c00961f0 tx=0x7f08c00ca790 comp rx=0 tx=0).stop 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 shutdown_connections 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x560e771c8d50 0x560e771cb140 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560e7715e720 0x560e771c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e77158890 0x560e771650e0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 --2- 172.21.15.16:0/2385016326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e7715de00 0x560e77165620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08aeffd700 0 client.17478 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 >> 172.21.15.16:0/2385016326 conn(0x560e770b6210 msgr2=0x560e7715f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 shutdown_connections 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 1 -- 172.21.15.16:0/2385016326 wait complete. 2024-05-08T23:28:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.385+0000 7f08e1ac6b80 20 client.17478 trim_cache size 0 max 16384 2024-05-08T23:28:29.393 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:28:29.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f08bc0a1290 msgr2=0x7f08bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f08bc0a1290 0x7f08bc0a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f08d0103220 tx=0x7f08b8009260 comp rx=0 tx=0).stop 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 msgr2=0x7f08d0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 0x7f08d0101af0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e9a0 tx=0x7f08c000ecb0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 shutdown_connections 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f08bc0a1290 0x7f08bc0a3750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08d0102570 0x7f08d01a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08d0100060 0x7f08d0101af0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 --2- 172.21.15.16:0/1675865738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0105180 0x7f08d0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 >> 172.21.15.16:0/1675865738 conn(0x7f08d0076f40 msgr2=0x7f08d0069930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 shutdown_connections 2024-05-08T23:28:29.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.386+0000 7f08d590c700 1 -- 172.21.15.16:0/1675865738 wait complete. 2024-05-08T23:28:29.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:28:29.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:29 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3546417021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:28:29.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:29 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3546417021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:28:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 -- 172.21.15.16:0/708212011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 msgr2=0x7f5054100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 --2- 172.21.15.16:0/708212011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054100440 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5050009990 tx=0x7f5050009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 -- 172.21.15.16:0/708212011 shutdown_connections 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 --2- 172.21.15.16:0/708212011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5054105160 0x7f505410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 --2- 172.21.15.16:0/708212011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054100440 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 --2- 172.21.15.16:0/708212011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5054100a10 0x7f5054104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 -- 172.21.15.16:0/708212011 >> 172.21.15.16:0/708212011 conn(0x7f5054076f40 msgr2=0x7f5054077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.538+0000 7f505bbb8700 1 -- 172.21.15.16:0/708212011 shutdown_connections 2024-05-08T23:28:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.539+0000 7f505bbb8700 1 -- 172.21.15.16:0/708212011 wait complete. 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.539+0000 7f505bbb8700 1 Processor -- start 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.539+0000 7f505bbb8700 1 -- start start 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.539+0000 7f505bbb8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.539+0000 7f505bbb8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5054105160 0x7f50541985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505bbb8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5054198b10 0x7f50541a3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505bbb8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541a37b0 con 0x7f5054100060 2024-05-08T23:28:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505bbb8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541a3920 con 0x7f5054105160 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505bbb8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50541a3ae0 con 0x7f5054198b10 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46138/0 (socket says 172.21.15.16:46138) 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 -- 172.21.15.16:0/2668971658 learned_addr learned my addr 172.21.15.16:0/2668971658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505a155700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5054198b10 0x7f50541a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059153700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5054105160 0x7f50541985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 -- 172.21.15.16:0/2668971658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5054198b10 msgr2=0x7f50541a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5054198b10 0x7f50541a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 -- 172.21.15.16:0/2668971658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5054105160 msgr2=0x7f50541985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5054105160 0x7f50541985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 -- 172.21.15.16:0/2668971658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500095f0 con 0x7f5054100060 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f5059954700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054198090 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f50500059d0 tx=0x7f5050004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f504affd700 1 -- 172.21.15.16:0/2668971658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505001d070 con 0x7f5054100060 2024-05-08T23:28:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f504affd700 1 -- 172.21.15.16:0/2668971658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f505000fb00 con 0x7f5054100060 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50541a3d60 con 0x7f5054100060 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f504affd700 1 -- 172.21.15.16:0/2668971658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050022a40 con 0x7f5054100060 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.540+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50541a4180 con 0x7f5054100060 2024-05-08T23:28:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.543+0000 7f504affd700 1 -- 172.21.15.16:0/2668971658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f505000fc70 con 0x7f5054100060 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.544+0000 7f504affd700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f50400a1290 0x7f50400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.544+0000 7f5059153700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f50400a1290 0x7f50400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.544+0000 7f504affd700 1 -- 172.21.15.16:0/2668971658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f50500ce3f0 con 0x7f5054100060 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 Processor -- start 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5059153700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f50400a1290 0x7f50400a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50540687e0 tx=0x7f504400a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 -- start start 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e53891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e5383350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e52da150 con 0x5618e5389720 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e52d7b10 con 0x5618e5383890 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5067d72b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e52d3400 con 0x5618e5388e00 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5059153700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e5383350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5059153700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e5383350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46144/0 (socket says 172.21.15.16:46144) 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5059153700 1 -- 172.21.15.16:0/2255460333 learned_addr learned my addr 172.21.15.16:0/2255460333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.545+0000 7f5059954700 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e53891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f505a155700 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f5059153700 1 -- 172.21.15.16:0/2255460333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 msgr2=0x5618e53891e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f5059153700 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e53891e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f5059153700 1 -- 172.21.15.16:0/2255460333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 msgr2=0x5618e5383cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f5059153700 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f5059153700 1 -- 172.21.15.16:0/2255460333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618e51543b0 con 0x5618e5389720 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f5059153700 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e5383350 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f504400fe70 tx=0x7f5044012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8e0bdce522dc0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f503ffff700 1 -- 172.21.15.16:0/2255460333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044003a90 con 0x5618e5389720 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f503ffff700 1 -- 172.21.15.16:0/2255460333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5044003bf0 con 0x5618e5389720 2024-05-08T23:28:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.546+0000 7f503ffff700 1 -- 172.21.15.16:0/2255460333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504401d400 con 0x5618e5389720 2024-05-08T23:28:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 -- 172.21.15.16:0/2255460333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 msgr2=0x5618e5383350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e5383350 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f504400fe70 tx=0x7f5044012000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 -- 172.21.15.16:0/2255460333 shutdown_connections 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e53891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2255460333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e5383350 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 -- 172.21.15.16:0/2255460333 >> 172.21.15.16:0/2255460333 conn(0x5618e52e1210 msgr2=0x5618e537ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 -- 172.21.15.16:0/2255460333 shutdown_connections 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 -- 172.21.15.16:0/2255460333 wait complete. 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 -- start start 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.547+0000 7f5067d72b80 1 Processor -- start 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e538cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e538d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e538d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e526f560 con 0x5618e5389720 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e526f200 con 0x5618e5383890 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e51566a0 con 0x5618e5388e00 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5067d72b80 10 client.0 _ll_register_callbacks cb 0x5618e5278b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e538d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5059153700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e538d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e538d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46148/0 (socket says 172.21.15.16:46148) 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5059153700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e538d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:60676/0 (socket says 172.21.15.16:60676) 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 -- 172.21.15.16:0/1373545089 learned_addr learned my addr 172.21.15.16:0/1373545089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f5059954700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e538cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 -- 172.21.15.16:0/1373545089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 msgr2=0x5618e538d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e538d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 -- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 msgr2=0x5618e538cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e538cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.548+0000 7f505a155700 1 -- 172.21.15.16:0/1373545089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504400fb20 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5059153700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e538d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f505a155700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e538d550 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f504c004070 tx=0x7f504c004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a1332ba9ca700c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f503d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f503d7fa700 1 -- 172.21.15.16:0/1373545089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c007910 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f503d7fa700 1 -- 172.21.15.16:0/1373545089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f504c007a70 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618e522c270 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f503d7fa700 1 -- 172.21.15.16:0/1373545089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504c007c30 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 10 client.17490 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5618e5100b60 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f503d7fa700 1 -- 172.21.15.16:0/1373545089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5618e522c270 con 0x5618e5389720 2024-05-08T23:28:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f503d7fa700 1 -- 172.21.15.16:0/1373545089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f504c014040 con 0x5618e5389720 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 10 client.17490 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 20 client.17490 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.549+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e539d300 0x5618e539f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.550+0000 7f5067d72b80 4 client.17490 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.550+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618e5352810 con 0x5618e539d300 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.550+0000 7f5059954700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e539d300 0x5618e539f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.550+0000 7f503d7fa700 10 client.17490 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.550+0000 7f5059954700 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e539d300 0x5618e539f720 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac9ffc01c44d2df server_cookie=76c62beabdc3b502 in_seq=0 out_seq=0 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f503d7fa700 1 -- 172.21.15.16:0/1373545089 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618e5352810 con 0x5618e539d300 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f503d7fa700 10 client.17490 handle_command_reply: tid=0 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 client.17490 shutdown 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 msgr2=0x5618e538d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e538d550 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f504c004070 tx=0x7f504c004380 comp rx=0 tx=0).stop 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 shutdown_connections 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e538d550 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e539d300 0x5618e539f720 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e538cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1373545089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e538d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f503d7fa700 0 client.17490 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 >> 172.21.15.16:0/1373545089 conn(0x5618e52e1210 msgr2=0x5618e537d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 shutdown_connections 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 1 -- 172.21.15.16:0/1373545089 wait complete. 2024-05-08T23:28:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.551+0000 7f5067d72b80 20 client.17490 trim_cache size 0 max 16384 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.565+0000 7f5067d72b80 1 Processor -- start 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.565+0000 7f5067d72b80 1 -- start start 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e539f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e537dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5067d72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e5399d80 con 0x5618e5389720 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5067d72b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e537e910 con 0x5618e5383890 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5067d72b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e5389560 con 0x5618e5388e00 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e537dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e537dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46150/0 (socket says 172.21.15.16:46150) 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 -- 172.21.15.16:0/2173339238 learned_addr learned my addr 172.21.15.16:0/2173339238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5059153700 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e539f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5059954700 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 -- 172.21.15.16:0/2173339238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 msgr2=0x5618e539f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e539f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 -- 172.21.15.16:0/2173339238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 msgr2=0x5618e5383350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 -- 172.21.15.16:0/2173339238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618e5299560 con 0x5618e5389720 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5059153700 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e539f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f505a155700 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e537dd20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f504c018000 tx=0x7f504c004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f32dcd6abbbafe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5048da4700 1 -- 172.21.15.16:0/2173339238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c004c00 con 0x5618e5389720 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5048da4700 1 -- 172.21.15.16:0/2173339238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f504c00f040 con 0x5618e5389720 2024-05-08T23:28:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.566+0000 7f5048da4700 1 -- 172.21.15.16:0/2173339238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c015f10 con 0x5618e5389720 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 -- 172.21.15.16:0/2173339238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 msgr2=0x5618e537dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e537dd20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f504c018000 tx=0x7f504c004000 comp rx=0 tx=0).stop 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 -- 172.21.15.16:0/2173339238 shutdown_connections 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e537dd20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5383890 0x5618e5383350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 --2- 172.21.15.16:0/2173339238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5388e00 0x5618e539f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 -- 172.21.15.16:0/2173339238 >> 172.21.15.16:0/2173339238 conn(0x5618e52e1210 msgr2=0x5618e5383c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 -- 172.21.15.16:0/2173339238 shutdown_connections 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 -- 172.21.15.16:0/2173339238 wait complete. 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.567+0000 7f5067d72b80 1 -- start start 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 Processor -- start 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5383890 0x5618e5394170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5388e00 0x5618e53946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e53849a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e5384f20 con 0x5618e5389720 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e5385090 con 0x5618e5388e00 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618e5385250 con 0x5618e5383890 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5067d72b80 10 client.0 _ll_register_callbacks cb 0x5618e5278b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f505a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e53849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f505a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e53849a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46164/0 (socket says 172.21.15.16:46164) 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f505a155700 1 -- 172.21.15.16:0/1362505519 learned_addr learned my addr 172.21.15.16:0/1362505519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5059153700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5388e00 0x5618e53946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.568+0000 7f5059954700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5383890 0x5618e5394170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f505a155700 1 -- 172.21.15.16:0/1362505519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5383890 msgr2=0x5618e5394170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f505a155700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5383890 0x5618e5394170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f505a155700 1 -- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5388e00 msgr2=0x5618e53946b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f505a155700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5388e00 0x5618e53946b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f505a155700 1 -- 172.21.15.16:0/1362505519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618e5100b60 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f505a155700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e53849a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f504c000f80 tx=0x7f504c009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9406fbb05cd275e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f503d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f503d7fa700 1 -- 172.21.15.16:0/1362505519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504c0092b0 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f503d7fa700 1 -- 172.21.15.16:0/1362505519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f504c015a20 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5044003bf0 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f503d7fa700 1 -- 172.21.15.16:0/1362505519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504c015be0 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f5067d72b80 10 client.17496 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5618e5158cd0 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.569+0000 7f503d7fa700 1 -- 172.21.15.16:0/1362505519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5044003bf0 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f503d7fa700 1 -- 172.21.15.16:0/1362505519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f504c014070 con 0x5618e5389720 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5067d72b80 10 client.17496 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5067d72b80 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5067d72b80 20 client.17496 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e53f37c0 0x5618e53f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5067d72b80 4 client.17496 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:28:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5618e52cbec0 con 0x5618e53f37c0 2024-05-08T23:28:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5059954700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e53f37c0 0x5618e53f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f5059954700 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e53f37c0 0x5618e53f5bb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb7191e28cb002c server_cookie=64045ef7a5ddc8e1 in_seq=0 out_seq=0 2024-05-08T23:28:29.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.570+0000 7f503d7fa700 10 client.17496 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f503d7fa700 1 -- 172.21.15.16:0/1362505519 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5618e52cbec0 con 0x5618e53f37c0 2024-05-08T23:28:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f503d7fa700 10 client.17496 handle_command_reply: tid=0 2024-05-08T23:28:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 client.17496 shutdown 2024-05-08T23:28:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 msgr2=0x5618e53849a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e53849a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f504c000f80 tx=0x7f504c009a00 comp rx=0 tx=0).stop 2024-05-08T23:28:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 shutdown_connections 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618e5389720 0x5618e53849a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618e5383890 0x5618e5394170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5618e53f37c0 0x5618e53f5bb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 --2- 172.21.15.16:0/1362505519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5618e5388e00 0x5618e53946b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f503d7fa700 0 client.17496 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 >> 172.21.15.16:0/1362505519 conn(0x5618e52e1210 msgr2=0x5618e5387b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 shutdown_connections 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 1 -- 172.21.15.16:0/1362505519 wait complete. 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f5067d72b80 20 client.17496 trim_cache size 0 max 16384 2024-05-08T23:28:29.578 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:28:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f50400a1290 msgr2=0x7f50400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f505bbb8700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f50400a1290 0x7f50400a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f50540687e0 tx=0x7f504400a380 comp rx=0 tx=0).stop 2024-05-08T23:28:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 msgr2=0x7f5054198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.571+0000 7f505bbb8700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054198090 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f50500059d0 tx=0x7f5050004730 comp rx=0 tx=0).stop 2024-05-08T23:28:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 shutdown_connections 2024-05-08T23:28:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f50400a1290 0x7f50400a3750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5054198b10 0x7f50541a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5054100060 0x7f5054198090 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 --2- 172.21.15.16:0/2668971658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5054105160 0x7f50541985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 >> 172.21.15.16:0/2668971658 conn(0x7f5054076f40 msgr2=0x7f50540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 shutdown_connections 2024-05-08T23:28:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:29.572+0000 7f505bbb8700 1 -- 172.21.15.16:0/2668971658 wait complete. 2024-05-08T23:28:30.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:30 smithi016 ceph-mon[42541]: pgmap v276: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:30.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:30 smithi189 ceph-mon[46919]: pgmap v276: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:30.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:30 smithi155 ceph-mon[47105]: pgmap v276: 129 pgs: 129 active+clean; 13 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:30.586 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:28:30.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:28:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 -- 172.21.15.16:0/2823920207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1850100a10 msgr2=0x7f1850104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 --2- 172.21.15.16:0/2823920207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1850100a10 0x7f1850104a30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1844009990 tx=0x7f1844009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 -- 172.21.15.16:0/2823920207 shutdown_connections 2024-05-08T23:28:30.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 --2- 172.21.15.16:0/2823920207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1850105160 0x7f185010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 --2- 172.21.15.16:0/2823920207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1850100060 0x7f1850100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 --2- 172.21.15.16:0/2823920207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1850100a10 0x7f1850104a30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 -- 172.21.15.16:0/2823920207 >> 172.21.15.16:0/2823920207 conn(0x7f1850076f40 msgr2=0x7f1850077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 -- 172.21.15.16:0/2823920207 shutdown_connections 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 -- 172.21.15.16:0/2823920207 wait complete. 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.735+0000 7f18561c3700 1 Processor -- start 2024-05-08T23:28:30.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 -- start start 2024-05-08T23:28:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1850100060 0x7f18501972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1850105160 0x7f1850197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 0x7f185019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019c650 con 0x7f185019bbd0 2024-05-08T23:28:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019c7c0 con 0x7f1850100060 2024-05-08T23:28:30.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f18561c3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019c980 con 0x7f1850105160 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f184ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 0x7f185019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f184ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 0x7f185019c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46180/0 (socket says 172.21.15.16:46180) 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f184ffff700 1 -- 172.21.15.16:0/1709474426 learned_addr learned my addr 172.21.15.16:0/1709474426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f184effd700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1850105160 0x7f1850197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.736+0000 7f184f7fe700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1850100060 0x7f18501972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184ffff700 1 -- 172.21.15.16:0/1709474426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1850105160 msgr2=0x7f1850197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184ffff700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1850105160 0x7f1850197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184ffff700 1 -- 172.21.15.16:0/1709474426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1850100060 msgr2=0x7f18501972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184ffff700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1850100060 0x7f18501972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184ffff700 1 -- 172.21.15.16:0/1709474426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440095f0 con 0x7f185019bbd0 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184ffff700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 0x7f185019c050 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1848007d70 tx=0x7f184800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184cff9700 1 -- 172.21.15.16:0/1709474426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184800ac90 con 0x7f185019bbd0 2024-05-08T23:28:30.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184cff9700 1 -- 172.21.15.16:0/1709474426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f184800adf0 con 0x7f185019bbd0 2024-05-08T23:28:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18501a3ee0 con 0x7f185019bbd0 2024-05-08T23:28:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f184cff9700 1 -- 172.21.15.16:0/1709474426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480115e0 con 0x7f185019bbd0 2024-05-08T23:28:30.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.737+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18501a4310 con 0x7f185019bbd0 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.739+0000 7f184cff9700 1 -- 172.21.15.16:0/1709474426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1848012420 con 0x7f185019bbd0 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.740+0000 7f184cff9700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18380a1290 0x7f18380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.740+0000 7f184cff9700 1 -- 172.21.15.16:0/1709474426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f18480cd080 con 0x7f185019bbd0 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.740+0000 7f184f7fe700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18380a1290 0x7f18380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 Processor -- start 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f184f7fe700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18380a1290 0x7f18380a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1840009c60 tx=0x7f1840009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 -- start start 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924ba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c924111d0 con 0x555c924bfe80 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9240eb90 con 0x555c924c07a0 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f186237db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9240a480 con 0x555c924ba910 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f184f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46184/0 (socket says 172.21.15.16:46184) 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f184f7fe700 1 -- 172.21.15.16:0/591005403 learned_addr learned my addr 172.21.15.16:0/591005403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f184ffff700 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.741+0000 7f184effd700 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924ba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184f7fe700 1 -- 172.21.15.16:0/591005403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 msgr2=0x555c924bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184f7fe700 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184f7fe700 1 -- 172.21.15.16:0/591005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 msgr2=0x555c924ba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184f7fe700 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924ba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184f7fe700 1 -- 172.21.15.16:0/591005403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9228b430 con 0x555c924bfe80 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184effd700 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f184f7fe700 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c0260 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f184000e4e0 tx=0x7f1840014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea07561d67678c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f1835ffb700 1 -- 172.21.15.16:0/591005403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184000ee00 con 0x555c924bfe80 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f1835ffb700 1 -- 172.21.15.16:0/591005403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1840021070 con 0x555c924bfe80 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.742+0000 7f1835ffb700 1 -- 172.21.15.16:0/591005403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184001d720 con 0x555c924bfe80 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 -- 172.21.15.16:0/591005403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 msgr2=0x555c924c0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c0260 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f184000e4e0 tx=0x7f1840014000 comp rx=0 tx=0).stop 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 -- 172.21.15.16:0/591005403 shutdown_connections 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c0260 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 --2- 172.21.15.16:0/591005403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924ba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 -- 172.21.15.16:0/591005403 >> 172.21.15.16:0/591005403 conn(0x555c92418290 msgr2=0x555c924b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 -- 172.21.15.16:0/591005403 shutdown_connections 2024-05-08T23:28:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 -- 172.21.15.16:0/591005403 wait complete. 2024-05-08T23:28:30.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.743+0000 7f186237db80 1 -- start start 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 Processor -- start 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924ba910 0x555c924c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924c07a0 0x555c924c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c923a65e0 con 0x555c924bfe80 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c923a6280 con 0x555c924ba910 2024-05-08T23:28:30.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9228d720 con 0x555c924c07a0 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f186237db80 10 client.0 _ll_register_callbacks cb 0x555c923afbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f184effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f184effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46200/0 (socket says 172.21.15.16:46200) 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f184effd700 1 -- 172.21.15.16:0/327217584 learned_addr learned my addr 172.21.15.16:0/327217584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f184f7fe700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924ba910 0x555c924c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.744+0000 7f184ffff700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924c07a0 0x555c924c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184effd700 1 -- 172.21.15.16:0/327217584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924c07a0 msgr2=0x555c924c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184effd700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924c07a0 0x555c924c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184effd700 1 -- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924ba910 msgr2=0x555c924c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184effd700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924ba910 0x555c924c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184effd700 1 -- 172.21.15.16:0/327217584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1840017070 con 0x555c924bfe80 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184f7fe700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924ba910 0x555c924c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f184effd700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c4130 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1844015000 tx=0x7f1844005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849c10c031b16b11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f182effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f182effd700 1 -- 172.21.15.16:0/327217584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184401d030 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f182effd700 1 -- 172.21.15.16:0/327217584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1844004500 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c923632f0 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f182effd700 1 -- 172.21.15.16:0/327217584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1844005a00 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f186237db80 10 client.17508 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.745+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c92237be0 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f182effd700 1 -- 172.21.15.16:0/327217584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555c923632f0 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f182effd700 1 -- 172.21.15.16:0/327217584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f184400f620 con 0x555c924bfe80 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f186237db80 10 client.17508 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f186237db80 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f186237db80 20 client.17508 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:30.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f186237db80 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c924d4240 0x555c924d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f186237db80 4 client.17508 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c92489890 con 0x555c924d4240 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.746+0000 7f184f7fe700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c924d4240 0x555c924d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f184f7fe700 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c924d4240 0x555c924d6660 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82fd98d42284fb30 server_cookie=75401c75b31a5ca0 in_seq=0 out_seq=0 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f182effd700 10 client.17508 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f182effd700 1 -- 172.21.15.16:0/327217584 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c92489890 con 0x555c924d4240 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f182effd700 10 client.17508 handle_command_reply: tid=0 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f186237db80 1 client.17508 shutdown 2024-05-08T23:28:30.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 msgr2=0x555c924c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.747+0000 7f186237db80 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c4130 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f1844015000 tx=0x7f1844005800 comp rx=0 tx=0).stop 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 shutdown_connections 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924c07a0 0x555c924c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c924d4240 0x555c924d6660 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924ba910 0x555c924c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f182effd700 0 client.17508 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 --2- 172.21.15.16:0/327217584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c4130 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 >> 172.21.15.16:0/327217584 conn(0x555c92418290 msgr2=0x555c924b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 shutdown_connections 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 1 -- 172.21.15.16:0/327217584 wait complete. 2024-05-08T23:28:30.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.748+0000 7f186237db80 20 client.17508 trim_cache size 0 max 16384 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 Processor -- start 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 -- start start 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c924db8a0 con 0x555c924bfe80 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c924af880 con 0x555c924c07a0 2024-05-08T23:28:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f186237db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c924b5f00 con 0x555c924ba910 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.766+0000 7f184effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46216/0 (socket says 172.21.15.16:46216) 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 -- 172.21.15.16:0/2546384009 learned_addr learned my addr 172.21.15.16:0/2546384009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184ffff700 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184f7fe700 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 -- 172.21.15.16:0/2546384009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 msgr2=0x555c924b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 -- 172.21.15.16:0/2546384009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 msgr2=0x555c924c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 -- 172.21.15.16:0/2546384009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c923d05e0 con 0x555c924bfe80 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184f7fe700 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f184effd700 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c3d80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f184400ff00 tx=0x7f184400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce51ccd37f7eb9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f18367fc700 1 -- 172.21.15.16:0/2546384009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184401d050 con 0x555c924bfe80 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f18367fc700 1 -- 172.21.15.16:0/2546384009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f184401e040 con 0x555c924bfe80 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.767+0000 7f18367fc700 1 -- 172.21.15.16:0/2546384009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844004870 con 0x555c924bfe80 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 -- 172.21.15.16:0/2546384009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 msgr2=0x555c924c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c3d80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f184400ff00 tx=0x7f184400b890 comp rx=0 tx=0).stop 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 -- 172.21.15.16:0/2546384009 shutdown_connections 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924c07a0 0x555c924c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 --2- 172.21.15.16:0/2546384009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924bfe80 0x555c924c3d80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 -- 172.21.15.16:0/2546384009 >> 172.21.15.16:0/2546384009 conn(0x555c92418290 msgr2=0x555c924bf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 -- 172.21.15.16:0/2546384009 shutdown_connections 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 -- 172.21.15.16:0/2546384009 wait complete. 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.768+0000 7f186237db80 1 -- start start 2024-05-08T23:28:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 Processor -- start 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924bfe80 0x555c924ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 0x555c9252a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c924cafb0 con 0x555c924c07a0 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9252ac90 con 0x555c924bfe80 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9252add0 con 0x555c924ba910 2024-05-08T23:28:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f186237db80 10 client.0 _ll_register_callbacks cb 0x555c923afbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 0x555c9252a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 0x555c9252a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 -- 172.21.15.16:0/4239671492 learned_addr learned my addr 172.21.15.16:0/4239671492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184f7fe700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184effd700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924bfe80 0x555c924ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 -- 172.21.15.16:0/4239671492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 msgr2=0x555c924ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 -- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924bfe80 msgr2=0x555c924ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924bfe80 0x555c924ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 -- 172.21.15.16:0/4239671492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c92237be0 con 0x555c924c07a0 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184effd700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924bfe80 0x555c924ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f184ffff700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 0x555c9252a750 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1848099c00 tx=0x7f184809a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c78072ee7efee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.769+0000 7f182effd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f182effd700 1 -- 172.21.15.16:0/4239671492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184809a360 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f182effd700 1 -- 172.21.15.16:0/4239671492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f18480993f0 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1840021070 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f182effd700 1 -- 172.21.15.16:0/4239671492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18480995b0 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 10 client.17514 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555c9228fd50 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f182effd700 1 -- 172.21.15.16:0/4239671492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18480b8030 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f182effd700 1 -- 172.21.15.16:0/4239671492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f18480cab80 con 0x555c924c07a0 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 10 client.17514 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 20 client.17514 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c9252c600 0x555c9252e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 4 client.17514 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555c92402f40 con 0x555c9252c600 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.770+0000 7f184f7fe700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c9252c600 0x555c9252e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.771+0000 7f182effd700 10 client.17514 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.771+0000 7f184f7fe700 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c9252c600 0x555c9252e9f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e026890483581e3 server_cookie=1934ed7c5ed3b6ff in_seq=0 out_seq=0 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f182effd700 1 -- 172.21.15.16:0/4239671492 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c92402f40 con 0x555c9252c600 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f182effd700 10 client.17514 handle_command_reply: tid=0 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 client.17514 shutdown 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 msgr2=0x555c9252a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 0x555c9252a750 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1848099c00 tx=0x7f184809a040 comp rx=0 tx=0).stop 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 shutdown_connections 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c924c07a0 0x555c9252a750 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c924ba910 0x555c924ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555c9252c600 0x555c9252e9f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 --2- 172.21.15.16:0/4239671492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555c924bfe80 0x555c924ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f182effd700 0 client.17514 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 >> 172.21.15.16:0/4239671492 conn(0x555c92418290 msgr2=0x555c924c1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 shutdown_connections 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.772+0000 7f186237db80 1 -- 172.21.15.16:0/4239671492 wait complete. 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f186237db80 20 client.17514 trim_cache size 0 max 16384 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:28:30.779 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:28:30.780 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ded81e2c-fe46-4d06-9938-e284e3abf189", 2024-05-08T23:28:30.780 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:28:30.780 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:28:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18380a1290 msgr2=0x7f18380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18380a1290 0x7f18380a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1840009c60 tx=0x7f1840009260 comp rx=0 tx=0).stop 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 msgr2=0x7f185019c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 0x7f185019c050 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1848007d70 tx=0x7f184800a340 comp rx=0 tx=0).stop 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 shutdown_connections 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18380a1290 0x7f18380a3750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f185019bbd0 0x7f185019c050 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1850100060 0x7f18501972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 --2- 172.21.15.16:0/1709474426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1850105160 0x7f1850197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 >> 172.21.15.16:0/1709474426 conn(0x7f1850076f40 msgr2=0x7f18500fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 shutdown_connections 2024-05-08T23:28:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.773+0000 7f18561c3700 1 -- 172.21.15.16:0/1709474426 wait complete. 2024-05-08T23:28:30.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:28:30.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2380451303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 msgr2=0x7f0dd8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2380451303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd8069800 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009990 tx=0x7f0dc8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2380451303 shutdown_connections 2024-05-08T23:28:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2380451303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd81062a0 0x7f0dd810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2380451303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd8069800 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2380451303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dd8069dd0 0x7f0dd8105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2380451303 >> 172.21.15.16:0/2380451303 conn(0x7f0dd8076ba0 msgr2=0x7f0dd8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2380451303 shutdown_connections 2024-05-08T23:28:30.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.926+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2380451303 wait complete. 2024-05-08T23:28:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 Processor -- start 2024-05-08T23:28:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 -- start start 2024-05-08T23:28:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd81062a0 0x7f0dd8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dd819bba0 0x7f0dd819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd819c620 con 0x7f0dd8069420 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd819c790 con 0x7f0dd819bba0 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.927+0000 7f0ddc8b6700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd819c950 con 0x7f0dd81062a0 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd81972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46252/0 (socket says 172.21.15.16:46252) 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 -- 172.21.15.16:0/2613162148 learned_addr learned my addr 172.21.15.16:0/2613162148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dd819bba0 0x7f0dd819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd81062a0 0x7f0dd8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 -- 172.21.15.16:0/2613162148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd81062a0 msgr2=0x7f0dd8197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd81062a0 0x7f0dd8197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 -- 172.21.15.16:0/2613162148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dd819bba0 msgr2=0x7f0dd819c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dd819bba0 0x7f0dd819c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 -- 172.21.15.16:0/2613162148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80095f0 con 0x7f0dd8069420 2024-05-08T23:28:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dd659c700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd81972c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80036c0 tx=0x7f0dc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dc77fe700 1 -- 172.21.15.16:0/2613162148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc801d070 con 0x7f0dd8069420 2024-05-08T23:28:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dc77fe700 1 -- 172.21.15.16:0/2613162148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dc800fb00 con 0x7f0dd8069420 2024-05-08T23:28:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd81a3e50 con 0x7f0dd8069420 2024-05-08T23:28:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0dc77fe700 1 -- 172.21.15.16:0/2613162148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8022b60 con 0x7f0dd8069420 2024-05-08T23:28:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.928+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd81a4210 con 0x7f0dd8069420 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.931+0000 7f0dc77fe700 1 -- 172.21.15.16:0/2613162148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0dc800fc70 con 0x7f0dd8069420 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.931+0000 7f0dc77fe700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0dc00a1290 0x7f0dc00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.931+0000 7f0dc77fe700 1 -- 172.21.15.16:0/2613162148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0dc80ce600 con 0x7f0dd8069420 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.931+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0dc00a1290 0x7f0dc00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.932+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0dc00a1290 0x7f0dc00a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc005ea0 tx=0x7f0dcc005e30 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.932+0000 7f0de8a70b80 1 Processor -- start 2024-05-08T23:28:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 -- start start 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861de210 0x561e861de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861deb30 0x561e861d8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e8612b150 con 0x561e861d8ca0 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e86128b10 con 0x561e861deb30 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e86124400 con 0x561e861de210 2024-05-08T23:28:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861d90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46266/0 (socket says 172.21.15.16:46266) 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 -- 172.21.15.16:0/1602142904 learned_addr learned my addr 172.21.15.16:0/1602142904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861de210 0x561e861de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861deb30 0x561e861d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 -- 172.21.15.16:0/1602142904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861de210 msgr2=0x561e861de5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861de210 0x561e861de5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 -- 172.21.15.16:0/1602142904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861deb30 msgr2=0x561e861d8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861deb30 0x561e861d8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd6d9d700 1 -- 172.21.15.16:0/1602142904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e85fa53b0 con 0x561e861d8ca0 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.933+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861deb30 0x561e861d8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.934+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861d90e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0003fa0 tx=0x7f0dd0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a4a7ddab828fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.934+0000 7f0dc4ff9700 1 -- 172.21.15.16:0/1602142904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0019070 con 0x561e861d8ca0 2024-05-08T23:28:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.934+0000 7f0dc4ff9700 1 -- 172.21.15.16:0/1602142904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dd000f040 con 0x561e861d8ca0 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.934+0000 7f0dc4ff9700 1 -- 172.21.15.16:0/1602142904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0007900 con 0x561e861d8ca0 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.934+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1602142904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 msgr2=0x561e861d90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.934+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861d90e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0003fa0 tx=0x7f0dd0009ba0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1602142904 shutdown_connections 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861d90e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861de210 0x561e861de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1602142904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861deb30 0x561e861d8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1602142904 >> 172.21.15.16:0/1602142904 conn(0x561e86132210 msgr2=0x561e861cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1602142904 shutdown_connections 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1602142904 wait complete. 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- start start 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 Processor -- start 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861d8ca0 0x561e861e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 0x561e861e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e860c0560 con 0x561e861de210 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e860c0200 con 0x561e861d8ca0 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.935+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e85fa76a0 con 0x561e861deb30 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0de8a70b80 10 client.0 _ll_register_callbacks cb 0x561e860c9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 0x561e861e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 0x561e861e1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46268/0 (socket says 172.21.15.16:46268) 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 -- 172.21.15.16:0/3437974611 learned_addr learned my addr 172.21.15.16:0/3437974611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd659c700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861d8ca0 0x561e861e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 -- 172.21.15.16:0/3437974611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 msgr2=0x561e861e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 -- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861d8ca0 msgr2=0x561e861e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861d8ca0 0x561e861e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 -- 172.21.15.16:0/3437974611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0003bd0 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 0x561e861e1d90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc010a30 tx=0x7f0dcc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64ebd35fd981bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3437974611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00fa50 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e8607d270 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3437974611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0dcc00fbb0 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3437974611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dcc00ee90 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.936+0000 7f0de8a70b80 10 client.17526 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e85f51b60 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3437974611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561e8607d270 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3437974611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0dcc024020 con 0x561e861de210 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 10 client.17526 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 20 client.17526 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e861f2580 0x561e861f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 4 client.17526 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:28:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e861a3810 con 0x561e861f2580 2024-05-08T23:28:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.937+0000 7f0dd659c700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e861f2580 0x561e861f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.938+0000 7f0dbdffb700 10 client.17526 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.938+0000 7f0dd659c700 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e861f2580 0x561e861f49a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be6c369a6b418ea server_cookie=64d428c53632479d in_seq=0 out_seq=0 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.938+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3437974611 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e861a3810 con 0x561e861f2580 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.938+0000 7f0dbdffb700 10 client.17526 handle_command_reply: tid=0 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.938+0000 7f0de8a70b80 1 client.17526 shutdown 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 msgr2=0x561e861e1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 0x561e861e1d90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc010a30 tx=0x7f0dcc0047c0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 shutdown_connections 2024-05-08T23:28:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e861f2580 0x561e861f49a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861d8ca0 0x561e861e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/3437974611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861de210 0x561e861e1d90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0dbdffb700 0 client.17526 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 >> 172.21.15.16:0/3437974611 conn(0x561e86132210 msgr2=0x561e861ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 shutdown_connections 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 1 -- 172.21.15.16:0/3437974611 wait complete. 2024-05-08T23:28:30.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.939+0000 7f0de8a70b80 20 client.17526 trim_cache size 0 max 16384 2024-05-08T23:28:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 Processor -- start 2024-05-08T23:28:30.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 -- start start 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e861e3450 con 0x561e861d8ca0 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e861d8790 con 0x561e861de210 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e861f9bc0 con 0x561e861deb30 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0dd659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0dd659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46282/0 (socket says 172.21.15.16:46282) 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.957+0000 7f0dd659c700 1 -- 172.21.15.16:0/1236381849 learned_addr learned my addr 172.21.15.16:0/1236381849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd659c700 1 -- 172.21.15.16:0/1236381849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 msgr2=0x561e861dbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861dbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd659c700 1 -- 172.21.15.16:0/1236381849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 msgr2=0x561e861e4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd659c700 1 -- 172.21.15.16:0/1236381849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e860ea560 con 0x561e861d8ca0 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861ced20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80cbfb0 tx=0x7f0dc8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4c55707bcba28d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dc57fa700 1 -- 172.21.15.16:0/1236381849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc809b400 con 0x561e861d8ca0 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dc57fa700 1 -- 172.21.15.16:0/1236381849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dc809f230 con 0x561e861d8ca0 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.958+0000 7f0dc57fa700 1 -- 172.21.15.16:0/1236381849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc809b660 con 0x561e861d8ca0 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1236381849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 msgr2=0x561e861ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861ced20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80cbfb0 tx=0x7f0dc8086040 comp rx=0 tx=0).stop 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1236381849 shutdown_connections 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861dbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861ced20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1236381849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1236381849 >> 172.21.15.16:0/1236381849 conn(0x561e86132210 msgr2=0x561e861ee050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1236381849 shutdown_connections 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1236381849 wait complete. 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 -- start start 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 Processor -- start 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.959+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e861da2f0 con 0x561e861d8ca0 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e861da460 con 0x561e861de210 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e861da620 con 0x561e861deb30 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 10 client.0 _ll_register_callbacks cb 0x561e860c9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861e95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46294/0 (socket says 172.21.15.16:46294) 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 -- 172.21.15.16:0/1747370294 learned_addr learned my addr 172.21.15.16:0/1747370294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd6d9d700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 -- 172.21.15.16:0/1747370294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 msgr2=0x561e861d9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861d9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 -- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 msgr2=0x561e861e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 -- 172.21.15.16:0/1747370294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd81a4210 con 0x561e861d8ca0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dd659c700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861e95a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0dc809b340 tx=0x7f0dc8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c9903c5b162435 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dbdffb700 1 -- 172.21.15.16:0/1747370294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80ce3e0 con 0x561e861d8ca0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0dbdffb700 1 -- 172.21.15.16:0/1747370294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0dc809b400 con 0x561e861d8ca0 2024-05-08T23:28:30.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.960+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0dd000f040 con 0x561e861d8ca0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0dbdffb700 1 -- 172.21.15.16:0/1747370294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dc809a8f0 con 0x561e861d8ca0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 10 client.17532 fetch_fsmap learned FSMap version 25 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e85fa9cd0 con 0x561e861d8ca0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0dbdffb700 1 -- 172.21.15.16:0/1747370294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0dd000f040 con 0x561e861d8ca0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0dbdffb700 1 -- 172.21.15.16:0/1747370294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0dc80b1070 con 0x561e861d8ca0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 10 client.17532 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 20 client.17532 populate_metadata read hostname 'smithi016' 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e86249520 0x561e8624b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 4 client.17532 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561e8611cec0 con 0x561e86249520 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.961+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e86249520 0x561e8624b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0dd5d9b700 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e86249520 0x561e8624b940 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ee35fb847e08b2 server_cookie=4608e929cde4f0ee in_seq=0 out_seq=0 2024-05-08T23:28:30.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0dbdffb700 10 client.17532 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0dbdffb700 1 -- 172.21.15.16:0/1747370294 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561e8611cec0 con 0x561e86249520 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0dbdffb700 10 client.17532 handle_command_reply: tid=0 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 client.17532 shutdown 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 msgr2=0x561e861e95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861e95a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0dc809b340 tx=0x7f0dc8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 shutdown_connections 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e861deb30 0x561e861d9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e861d8ca0 0x561e861e95a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561e86249520 0x561e8624b940 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 --2- 172.21.15.16:0/1747370294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561e861de210 0x561e861e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0dbdffb700 0 client.17532 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 >> 172.21.15.16:0/1747370294 conn(0x561e86132210 msgr2=0x561e861ee050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 shutdown_connections 2024-05-08T23:28:30.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 1 -- 172.21.15.16:0/1747370294 wait complete. 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.962+0000 7f0de8a70b80 20 client.17532 trim_cache size 0 max 16384 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: "ded81e2c-fe46-4d06-9938-e284e3abf189": { 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "ded81e2c-fe46-4d06-9938-e284e3abf189", 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:28:30.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:28:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:28:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0dc00a1290 msgr2=0x7f0dc00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0dc00a1290 0x7f0dc00a3750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc005ea0 tx=0x7f0dcc005e30 comp rx=0 tx=0).stop 2024-05-08T23:28:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 msgr2=0x7f0dd81972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:28:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd81972c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80036c0 tx=0x7f0dc8004730 comp rx=0 tx=0).stop 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 shutdown_connections 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0dc00a1290 0x7f0dc00a3750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0dd819bba0 0x7f0dd819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd8069420 0x7f0dd81972c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 --2- 172.21.15.16:0/2613162148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd81062a0 0x7f0dd8197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 >> 172.21.15.16:0/2613162148 conn(0x7f0dd8076ba0 msgr2=0x7f0dd8107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 shutdown_connections 2024-05-08T23:28:30.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:28:30.963+0000 7f0ddc8b6700 1 -- 172.21.15.16:0/2613162148 wait complete. 2024-05-08T23:28:30.977 INFO:tasks.cephfs.filesystem:scrub status for tag:ded81e2c-fe46-4d06-9938-e284e3abf189 - {'path': '/', 'tag': 'ded81e2c-fe46-4d06-9938-e284e3abf189', 'options': 'recursive,force'} 2024-05-08T23:28:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:32 smithi016 ceph-mon[42541]: pgmap v277: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-05-08T23:28:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:32 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:28:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:32 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:32 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:28:32.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:32 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:32.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:32 smithi189 ceph-mon[46919]: pgmap v277: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-05-08T23:28:32.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:32 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:28:32.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:32 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:32.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:32 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:28:32.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:32 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:32 smithi155 ceph-mon[47105]: pgmap v277: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-05-08T23:28:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:32 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:28:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:32 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:32 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:28:32.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:32 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:28:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:34 smithi016 ceph-mon[42541]: pgmap v278: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:34.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:34 smithi189 ceph-mon[46919]: pgmap v278: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:34.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:34 smithi155 ceph-mon[47105]: pgmap v278: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:36.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:36 smithi016 ceph-mon[42541]: pgmap v279: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-05-08T23:28:36.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:36 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:28:36.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:36 smithi189 ceph-mon[46919]: pgmap v279: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-05-08T23:28:36.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:36 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:28:36.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:36 smithi155 ceph-mon[47105]: pgmap v279: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-05-08T23:28:36.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:36 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:28:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:38 smithi016 ceph-mon[42541]: pgmap v280: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:38.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:38 smithi189 ceph-mon[46919]: pgmap v280: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:38.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:38 smithi155 ceph-mon[47105]: pgmap v280: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:40.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:40 smithi016 ceph-mon[42541]: pgmap v281: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:40.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:40 smithi189 ceph-mon[46919]: pgmap v281: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:40.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:40 smithi155 ceph-mon[47105]: pgmap v281: 129 pgs: 129 active+clean; 14 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:28:42.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:42 smithi016 ceph-mon[42541]: pgmap v282: 129 pgs: 129 active+clean; 14 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-05-08T23:28:42.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:42 smithi189 ceph-mon[46919]: pgmap v282: 129 pgs: 129 active+clean; 14 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-05-08T23:28:42.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:42 smithi155 ceph-mon[47105]: pgmap v282: 129 pgs: 129 active+clean; 14 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-05-08T23:28:44.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:44 smithi016 ceph-mon[42541]: pgmap v283: 129 pgs: 129 active+clean; 14 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:44.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:44 smithi189 ceph-mon[46919]: pgmap v283: 129 pgs: 129 active+clean; 14 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:44.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:44 smithi155 ceph-mon[47105]: pgmap v283: 129 pgs: 129 active+clean; 14 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:46.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:46 smithi016 ceph-mon[42541]: pgmap v284: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-05-08T23:28:46.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:46 smithi189 ceph-mon[46919]: pgmap v284: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-05-08T23:28:46.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:46 smithi155 ceph-mon[47105]: pgmap v284: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-05-08T23:28:47.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:47 smithi189 ceph-mon[46919]: pgmap v285: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:47.466 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:47 smithi155 ceph-mon[47105]: pgmap v285: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:47.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:47 smithi016 ceph-mon[42541]: pgmap v285: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:49.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:49 smithi189 ceph-mon[46919]: pgmap v286: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:49 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:28:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:49 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:28:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:49 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:28:49.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:49 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:28:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:49 smithi155 ceph-mon[47105]: pgmap v286: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:49 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:28:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:49 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:28:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:49 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:28:49.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:49 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:28:49.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:49 smithi016 ceph-mon[42541]: pgmap v286: 129 pgs: 129 active+clean; 14 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:49 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:28:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:49 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:28:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:49 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:28:49.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:49 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:28:51.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:51 smithi189 ceph-mon[46919]: pgmap v287: 129 pgs: 129 active+clean; 14 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-08T23:28:51.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:51 smithi155 ceph-mon[47105]: pgmap v287: 129 pgs: 129 active+clean; 14 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-08T23:28:51.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:51 smithi016 ceph-mon[42541]: pgmap v287: 129 pgs: 129 active+clean; 14 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-05-08T23:28:53.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:53 smithi189 ceph-mon[46919]: pgmap v288: 129 pgs: 129 active+clean; 14 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:53.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:53 smithi155 ceph-mon[47105]: pgmap v288: 129 pgs: 129 active+clean; 14 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:53.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:53 smithi016 ceph-mon[42541]: pgmap v288: 129 pgs: 129 active+clean; 14 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:55.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:55 smithi189 ceph-mon[46919]: pgmap v289: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-05-08T23:28:55.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:55 smithi155 ceph-mon[47105]: pgmap v289: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-05-08T23:28:55.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:55 smithi016 ceph-mon[42541]: pgmap v289: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-05-08T23:28:56.649 INFO:tasks.workunit.client.0.smithi016.stdout:375/600: open fd = 3 2024-05-08T23:28:56.649 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:376/600: open fd = 3 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:377/600: open fd = 3 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:378/600: open fd = 3 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.650 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:379/600: open fd = 3 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:380/600: open fd = 3 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:381/600: open fd = 3 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.651 INFO:tasks.workunit.client.0.smithi016.stdout:382/600: open fd = 3 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:383/600: open fd = 3 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:384/600: open fd = 3 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:385/600: open fd = 3 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.652 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:386/600: open fd = 3 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:387/600: open fd = 3 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:388/600: open fd = 3 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.653 INFO:tasks.workunit.client.0.smithi016.stdout:389/600: open fd = 3 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:390/600: open fd = 3 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:391/600: open fd = 3 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:392/600: open fd = 3 2024-05-08T23:28:56.654 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:393/600: open fd = 3 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:394/600: open fd = 3 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:395/600: open fd = 3 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.655 INFO:tasks.workunit.client.0.smithi016.stdout:396/600: open fd = 3 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:397/600: open fd = 3 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:398/600: open fd = 3 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:399/600: open fd = 3 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.656 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:400/600: open fd = 3 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:401/600: open fd = 3 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:402/600: open fd = 3 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.657 INFO:tasks.workunit.client.0.smithi016.stdout:403/600: open fd = 3 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:404/600: open fd = 3 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:405/600: open fd = 3 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:406/600: open fd = 3 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.658 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:407/600: open fd = 3 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:408/600: open fd = 3 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:409/600: open fd = 3 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.659 INFO:tasks.workunit.client.0.smithi016.stdout:410/600: open fd = 3 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:411/600: open fd = 3 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:412/600: open fd = 3 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:413/600: open fd = 3 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.660 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:414/600: open fd = 3 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:415/600: open fd = 3 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:416/600: open fd = 3 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:417/600: open fd = 3 2024-05-08T23:28:56.661 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:418/600: open fd = 3 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:419/600: open fd = 3 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:420/600: open fd = 3 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.662 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:421/600: open fd = 3 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:422/600: open fd = 3 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:423/600: open fd = 3 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:424/600: open fd = 3 2024-05-08T23:28:56.663 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:425/600: open fd = 3 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:426/600: open fd = 3 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:427/600: open fd = 3 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.664 INFO:tasks.workunit.client.0.smithi016.stdout:428/600: open fd = 3 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:429/600: open fd = 3 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:430/600: open fd = 3 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:431/600: open fd = 3 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.665 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:432/600: open fd = 3 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:433/600: open fd = 3 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:434/600: open fd = 3 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:435/600: open fd = 3 2024-05-08T23:28:56.666 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.667 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.667 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:56.667 INFO:tasks.workunit.client.0.smithi016.stdout:436/600: open fd = 3 2024-05-08T23:28:56.667 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.667 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:28:56.667 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:28:57.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:57 smithi189 ceph-mon[46919]: pgmap v290: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:57.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:57 smithi155 ceph-mon[47105]: pgmap v290: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:57.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:57 smithi016 ceph-mon[42541]: pgmap v290: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:59.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:28:59 smithi189 ceph-mon[46919]: pgmap v291: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:59.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:28:59 smithi155 ceph-mon[47105]: pgmap v291: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:28:59.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:28:59 smithi016 ceph-mon[42541]: pgmap v291: 129 pgs: 129 active+clean; 15 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:00.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:29:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 -- 172.21.15.16:0/584927127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad041000f0 msgr2=0x7fad04100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 --2- 172.21.15.16:0/584927127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad041000f0 0x7fad04100570 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fad000099c0 tx=0x7fad00009cd0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 -- 172.21.15.16:0/584927127 shutdown_connections 2024-05-08T23:29:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 --2- 172.21.15.16:0/584927127 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad04100ab0 0x7fad0410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 --2- 172.21.15.16:0/584927127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad04105f90 0x7fad04106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 --2- 172.21.15.16:0/584927127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad041000f0 0x7fad04100570 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 -- 172.21.15.16:0/584927127 >> 172.21.15.16:0/584927127 conn(0x7fad04076ba0 msgr2=0x7fad04076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 -- 172.21.15.16:0/584927127 shutdown_connections 2024-05-08T23:29:01.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.127+0000 7fad0c04c700 1 -- 172.21.15.16:0/584927127 wait complete. 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 Processor -- start 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 -- start start 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad041000f0 0x7fad04197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 0x7fad041978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad04105f90 0x7fad0419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0419c290 con 0x7fad04100ab0 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0419c400 con 0x7fad041000f0 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0c04c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0419c5c0 con 0x7fad04105f90 2024-05-08T23:29:01.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad095e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 0x7fad041978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad095e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 0x7fad041978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35378/0 (socket says 172.21.15.16:35378) 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad095e7700 1 -- 172.21.15.16:0/1554387733 learned_addr learned my addr 172.21.15.16:0/1554387733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad09de8700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad041000f0 0x7fad04197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.128+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad04105f90 0x7fad0419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad095e7700 1 -- 172.21.15.16:0/1554387733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad04105f90 msgr2=0x7fad0419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad095e7700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad04105f90 0x7fad0419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad095e7700 1 -- 172.21.15.16:0/1554387733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad041000f0 msgr2=0x7fad04197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad095e7700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad041000f0 0x7fad04197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad095e7700 1 -- 172.21.15.16:0/1554387733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000095f0 con 0x7fad04100ab0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad095e7700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 0x7fad041978b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fad00009f50 tx=0x7fad00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7facfaffd700 1 -- 172.21.15.16:0/1554387733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0001d070 con 0x7fad04100ab0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7facfaffd700 1 -- 172.21.15.16:0/1554387733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad0000fcf0 con 0x7fad04100ab0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0419c840 con 0x7fad04100ab0 2024-05-08T23:29:01.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7facfaffd700 1 -- 172.21.15.16:0/1554387733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00022d40 con 0x7fad04100ab0 2024-05-08T23:29:01.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.129+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad041a40e0 con 0x7fad04100ab0 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.132+0000 7facfaffd700 1 -- 172.21.15.16:0/1554387733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fad00022470 con 0x7fad04100ab0 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.132+0000 7facfaffd700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7facf00a1290 0x7facf00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.132+0000 7facfaffd700 1 -- 172.21.15.16:0/1554387733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fad000cdef0 con 0x7fad04100ab0 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.132+0000 7fad09de8700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7facf00a1290 0x7facf00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.133+0000 7fad09de8700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7facf00a1290 0x7facf00a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad04067bb0 tx=0x7facf400b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.133+0000 7fad18206b80 1 Processor -- start 2024-05-08T23:29:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.133+0000 7fad18206b80 1 -- start start 2024-05-08T23:29:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad18206b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad18206b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 0x5616e90a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad18206b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad18206b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8ffe150 con 0x5616e90ad720 2024-05-08T23:29:01.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad18206b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8ffbb10 con 0x5616e90a7890 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad18206b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8ff7400 con 0x5616e90ace00 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 0x5616e90a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 0x5616e90a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35382/0 (socket says 172.21.15.16:35382) 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 -- 172.21.15.16:0/4071799392 learned_addr learned my addr 172.21.15.16:0/4071799392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad09de8700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 -- 172.21.15.16:0/4071799392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 msgr2=0x5616e90ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 -- 172.21.15.16:0/4071799392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 msgr2=0x5616e90a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 -- 172.21.15.16:0/4071799392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e8e783b0 con 0x5616e90ad720 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad09de8700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7fad095e7700 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 0x5616e90a7350 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fad0009a9f0 tx=0x7fad0009a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b5fd217526b6c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7faceffff700 1 -- 172.21.15.16:0/4071799392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00022ea0 con 0x5616e90ad720 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7faceffff700 1 -- 172.21.15.16:0/4071799392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad0009acf0 con 0x5616e90ad720 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.134+0000 7faceffff700 1 -- 172.21.15.16:0/4071799392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0009a440 con 0x5616e90ad720 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 -- 172.21.15.16:0/4071799392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 msgr2=0x5616e90a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 0x5616e90a7350 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fad0009a9f0 tx=0x7fad0009a830 comp rx=0 tx=0).stop 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 -- 172.21.15.16:0/4071799392 shutdown_connections 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 --2- 172.21.15.16:0/4071799392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ad720 0x5616e90a7350 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 -- 172.21.15.16:0/4071799392 >> 172.21.15.16:0/4071799392 conn(0x5616e9005210 msgr2=0x5616e90a1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 -- 172.21.15.16:0/4071799392 shutdown_connections 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.135+0000 7fad18206b80 1 -- 172.21.15.16:0/4071799392 wait complete. 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 -- start start 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 Processor -- start 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 0x5616e90b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ad720 0x5616e90b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8f93560 con 0x5616e90ace00 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8f93200 con 0x5616e90a7890 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8e7a6a0 con 0x5616e90ad720 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.136+0000 7fad18206b80 10 client.0 _ll_register_callbacks cb 0x5616e8f9cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 0x5616e90b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 0x5616e90b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35396/0 (socket says 172.21.15.16:35396) 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 -- 172.21.15.16:0/3137168291 learned_addr learned my addr 172.21.15.16:0/3137168291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad09de8700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ad720 0x5616e90b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 -- 172.21.15.16:0/3137168291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ad720 msgr2=0x5616e90b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ad720 0x5616e90b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 -- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 msgr2=0x5616e90b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 -- 172.21.15.16:0/3137168291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad041a40e0 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad095e7700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 0x5616e90b0f50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fad000971f0 tx=0x7fad0009b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5a8bb1f3abe7eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7faced7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7faced7fa700 1 -- 172.21.15.16:0/3137168291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00022ea0 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7faced7fa700 1 -- 172.21.15.16:0/3137168291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fad000a30e0 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616e8f50270 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7faced7fa700 1 -- 172.21.15.16:0/3137168291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad000a2e60 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad18206b80 10 client.17544 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.137+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5616e8e24b60 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7faced7fa700 1 -- 172.21.15.16:0/3137168291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad00064080 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7faced7fa700 1 -- 172.21.15.16:0/3137168291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad00096ca0 con 0x5616e90ace00 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad18206b80 10 client.17544 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad18206b80 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:01.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad18206b80 20 client.17544 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad18206b80 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e90c0900 0x5616e90c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad18206b80 4 client.17544 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616e9076810 con 0x5616e90c0900 2024-05-08T23:29:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.138+0000 7fad09de8700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e90c0900 0x5616e90c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7faced7fa700 10 client.17544 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:01.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7fad09de8700 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e90c0900 0x5616e90c2cf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498a7069998bcd07 server_cookie=8d3a8e0c499f8fe0 in_seq=0 out_seq=0 2024-05-08T23:29:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7faced7fa700 1 -- 172.21.15.16:0/3137168291 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616e9076810 con 0x5616e90c0900 2024-05-08T23:29:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7faced7fa700 10 client.17544 handle_command_reply: tid=0 2024-05-08T23:29:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7fad18206b80 1 client.17544 shutdown 2024-05-08T23:29:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 msgr2=0x5616e90b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.139+0000 7fad18206b80 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 0x5616e90b0f50 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fad000971f0 tx=0x7fad0009b180 comp rx=0 tx=0).stop 2024-05-08T23:29:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 shutdown_connections 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ad720 0x5616e90b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e90c0900 0x5616e90c2cf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90a7890 0x5616e90b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 --2- 172.21.15.16:0/3137168291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90ace00 0x5616e90b0f50 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7faced7fa700 0 client.17544 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 >> 172.21.15.16:0/3137168291 conn(0x5616e9005210 msgr2=0x5616e90a15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 shutdown_connections 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 1 -- 172.21.15.16:0/3137168291 wait complete. 2024-05-08T23:29:01.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.140+0000 7fad18206b80 20 client.17544 trim_cache size 0 max 16384 2024-05-08T23:29:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.160+0000 7fad18206b80 1 Processor -- start 2024-05-08T23:29:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.160+0000 7fad18206b80 1 -- start start 2024-05-08T23:29:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad18206b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad18206b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad18206b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e90abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad18206b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e90c7ec0 con 0x5616e90a7890 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad18206b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e909c800 con 0x5616e90ad720 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad18206b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e90c3350 con 0x5616e90ace00 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90a1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35408/0 (socket says 172.21.15.16:35408) 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 -- 172.21.15.16:0/2172951749 learned_addr learned my addr 172.21.15.16:0/2172951749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad095e7700 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e90abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 -- 172.21.15.16:0/2172951749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 msgr2=0x5616e90b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 -- 172.21.15.16:0/2172951749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 msgr2=0x5616e90abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e90abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 -- 172.21.15.16:0/2172951749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e8fbd560 con 0x5616e90a7890 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e90abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7fad09de8700 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90a1d20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x5616e90c35c0 tx=0x7facf4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8297c2f168f5da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.161+0000 7facf8ff9700 1 -- 172.21.15.16:0/2172951749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4025070 con 0x5616e90a7890 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7facf8ff9700 1 -- 172.21.15.16:0/2172951749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facf4015070 con 0x5616e90a7890 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7facf8ff9700 1 -- 172.21.15.16:0/2172951749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4027860 con 0x5616e90a7890 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 -- 172.21.15.16:0/2172951749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 msgr2=0x5616e90a1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90a1d20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x5616e90c35c0 tx=0x7facf4013000 comp rx=0 tx=0).stop 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 -- 172.21.15.16:0/2172951749 shutdown_connections 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e90abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90a1d20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 --2- 172.21.15.16:0/2172951749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 -- 172.21.15.16:0/2172951749 >> 172.21.15.16:0/2172951749 conn(0x5616e9005210 msgr2=0x5616e90af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 -- 172.21.15.16:0/2172951749 shutdown_connections 2024-05-08T23:29:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.162+0000 7fad18206b80 1 -- 172.21.15.16:0/2172951749 wait complete. 2024-05-08T23:29:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 -- start start 2024-05-08T23:29:01.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 Processor -- start 2024-05-08T23:29:01.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e9115d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e90b6d60 con 0x5616e90a7890 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e91162d0 con 0x5616e90ad720 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e9116490 con 0x5616e90ace00 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad18206b80 10 client.0 _ll_register_callbacks cb 0x5616e8f9cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad09de8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad09de8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35410/0 (socket says 172.21.15.16:35410) 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.163+0000 7fad09de8700 1 -- 172.21.15.16:0/1801591596 learned_addr learned my addr 172.21.15.16:0/1801591596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad0a5e9700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e9115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad095e7700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad09de8700 1 -- 172.21.15.16:0/1801591596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 msgr2=0x5616e90b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad09de8700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad09de8700 1 -- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 msgr2=0x5616e9115d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad09de8700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e9115d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad09de8700 1 -- 172.21.15.16:0/1801591596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf400b850 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad09de8700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90b60b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7facf4010d50 tx=0x7facf40274e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3848140c2718ee25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7faced7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7faced7fa700 1 -- 172.21.15.16:0/1801591596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4025050 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7faced7fa700 1 -- 172.21.15.16:0/1801591596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7facf4017070 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad0009acf0 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7faced7fa700 1 -- 172.21.15.16:0/1801591596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf40277b0 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad18206b80 10 client.17550 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.164+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5616e8e7ccd0 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7faced7fa700 1 -- 172.21.15.16:0/1801591596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad0009acf0 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7faced7fa700 1 -- 172.21.15.16:0/1801591596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facf4011a50 con 0x5616e90a7890 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad18206b80 10 client.17550 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad18206b80 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:01.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad18206b80 20 client.17550 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad18206b80 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e9117bc0 0x5616e9119fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad18206b80 4 client.17550 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616e8fefec0 con 0x5616e9117bc0 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad095e7700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e9117bc0 0x5616e9119fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7faced7fa700 10 client.17550 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.165+0000 7fad095e7700 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e9117bc0 0x5616e9119fb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd07cf6da3a486c server_cookie=15dfe81c668005d4 in_seq=0 out_seq=0 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7faced7fa700 1 -- 172.21.15.16:0/1801591596 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5616e8fefec0 con 0x5616e9117bc0 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7faced7fa700 10 client.17550 handle_command_reply: tid=0 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 client.17550 shutdown 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 msgr2=0x5616e90b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90b60b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7facf4010d50 tx=0x7facf40274e0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 shutdown_connections 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5616e9117bc0 0x5616e9119fb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5616e90ad720 0x5616e9115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616e90a7890 0x5616e90b60b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7faced7fa700 0 client.17550 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 --2- 172.21.15.16:0/1801591596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616e90ace00 0x5616e90b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 >> 172.21.15.16:0/1801591596 conn(0x5616e9005210 msgr2=0x5616e90abb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 shutdown_connections 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 1 -- 172.21.15.16:0/1801591596 wait complete. 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.166+0000 7fad18206b80 20 client.17550 trim_cache size 0 max 16384 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:29:01.172 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:29:01.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7facf00a1290 msgr2=0x7facf00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7facf00a1290 0x7facf00a3750 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fad04067bb0 tx=0x7facf400b410 comp rx=0 tx=0).stop 2024-05-08T23:29:01.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 msgr2=0x7fad041978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 0x7fad041978b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fad00009f50 tx=0x7fad00004730 comp rx=0 tx=0).stop 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 shutdown_connections 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7facf00a1290 0x7facf00a3750 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad04105f90 0x7fad0419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad041000f0 0x7fad04197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 --2- 172.21.15.16:0/1554387733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad04100ab0 0x7fad041978b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 >> 172.21.15.16:0/1554387733 conn(0x7fad04076ba0 msgr2=0x7fad040feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 shutdown_connections 2024-05-08T23:29:01.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.167+0000 7fad0c04c700 1 -- 172.21.15.16:0/1554387733 wait complete. 2024-05-08T23:29:01.183 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:29:01.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.319+0000 7f3734089700 1 -- 172.21.15.16:0/17833965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c105060 msgr2=0x7f372c10cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.319+0000 7f3734089700 1 --2- 172.21.15.16:0/17833965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c105060 0x7f372c10cbe0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3724009990 tx=0x7f3724009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 -- 172.21.15.16:0/17833965 shutdown_connections 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 --2- 172.21.15.16:0/17833965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c105060 0x7f372c10cbe0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 --2- 172.21.15.16:0/17833965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f372c0fff40 0x7f372c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 --2- 172.21.15.16:0/17833965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c1008f0 0x7f372c104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 -- 172.21.15.16:0/17833965 >> 172.21.15.16:0/17833965 conn(0x7f372c0fb7e0 msgr2=0x7f372c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 -- 172.21.15.16:0/17833965 shutdown_connections 2024-05-08T23:29:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 -- 172.21.15.16:0/17833965 wait complete. 2024-05-08T23:29:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 Processor -- start 2024-05-08T23:29:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.320+0000 7f3734089700 1 -- start start 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3734089700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f372c0fff40 0x7f372c1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3734089700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c105060 0x7f372c101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3734089700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 0x7f372c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3734089700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c1a59c0 con 0x7f372c102450 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3734089700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c1a5b00 con 0x7f372c105060 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3734089700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c1a5cc0 con 0x7f372c0fff40 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 0x7f372c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 0x7f372c1a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35432/0 (socket says 172.21.15.16:35432) 2024-05-08T23:29:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 -- 172.21.15.16:0/2869484867 learned_addr learned my addr 172.21.15.16:0/2869484867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3731e25700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f372c0fff40 0x7f372c1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 -- 172.21.15.16:0/2869484867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f372c0fff40 msgr2=0x7f372c1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f372c0fff40 0x7f372c1019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 -- 172.21.15.16:0/2869484867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c105060 msgr2=0x7f372c101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c105060 0x7f372c101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 -- 172.21.15.16:0/2869484867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240095f0 con 0x7f372c102450 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f3732626700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 0x7f372c1a53f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3724005b40 tx=0x7f37240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372401d070 con 0x7f372c102450 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.321+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3724021410 con 0x7f372c102450 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.322+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372c1a5f40 con 0x7f372c102450 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.322+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724017650 con 0x7f372c102450 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.322+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c1a6360 con 0x7f372c102450 2024-05-08T23:29:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.323+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372c109340 con 0x7f372c102450 2024-05-08T23:29:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.324+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f372400fa90 con 0x7f372c102450 2024-05-08T23:29:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.324+0000 7f372affd700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f37140a1290 0x7f37140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.325+0000 7f3731e25700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f37140a1290 0x7f37140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.325+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f37240cec50 con 0x7f372c102450 2024-05-08T23:29:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.325+0000 7f3731e25700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f37140a1290 0x7f37140a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f371c009c10 tx=0x7f371c009040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.328+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3724097d30 con 0x7f372c102450 2024-05-08T23:29:01.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:01 smithi016 ceph-mon[42541]: pgmap v292: 129 pgs: 129 active+clean; 15 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-05-08T23:29:01.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:01 smithi189 ceph-mon[46919]: pgmap v292: 129 pgs: 129 active+clean; 15 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-05-08T23:29:01.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:01 smithi155 ceph-mon[47105]: pgmap v292: 129 pgs: 129 active+clean; 15 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-05-08T23:29:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.600+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f372c101410 con 0x7f372c102450 2024-05-08T23:29:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.601+0000 7f372affd700 1 -- 172.21.15.16:0/2869484867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f372c101410 con 0x7f372c102450 2024-05-08T23:29:01.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:29:01.601 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:29:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f37140a1290 msgr2=0x7f37140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f37140a1290 0x7f37140a3750 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f371c009c10 tx=0x7f371c009040 comp rx=0 tx=0).stop 2024-05-08T23:29:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 msgr2=0x7f372c1a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 0x7f372c1a53f0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3724005b40 tx=0x7f37240057f0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 shutdown_connections 2024-05-08T23:29:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f37140a1290 0x7f37140a3750 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c102450 0x7f372c1a53f0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f372c0fff40 0x7f372c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 --2- 172.21.15.16:0/2869484867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f372c105060 0x7f372c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 >> 172.21.15.16:0/2869484867 conn(0x7f372c0fb7e0 msgr2=0x7f372c0683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 shutdown_connections 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.603+0000 7f3734089700 1 -- 172.21.15.16:0/2869484867 wait complete. 2024-05-08T23:29:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:29:01.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:29:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 -- 172.21.15.16:0/3742319996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 msgr2=0x7ffad80697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 --2- 172.21.15.16:0/3742319996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad80697e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009990 tx=0x7ffac8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 -- 172.21.15.16:0/3742319996 shutdown_connections 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 --2- 172.21.15.16:0/3742319996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffad81062a0 0x7ffad810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 --2- 172.21.15.16:0/3742319996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad80697e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 --2- 172.21.15.16:0/3742319996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8069db0 0x7ffad8105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 -- 172.21.15.16:0/3742319996 >> 172.21.15.16:0/3742319996 conn(0x7ffad8076ba0 msgr2=0x7ffad8076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 -- 172.21.15.16:0/3742319996 shutdown_connections 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.756+0000 7ffadddd5700 1 -- 172.21.15.16:0/3742319996 wait complete. 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 Processor -- start 2024-05-08T23:29:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 -- start start 2024-05-08T23:29:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad81001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffad81062a0 0x7ffad8100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8100c40 0x7ffad81a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a5a50 con 0x7ffad8069400 2024-05-08T23:29:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a5bc0 con 0x7ffad81062a0 2024-05-08T23:29:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffadddd5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a5d80 con 0x7ffad8100c40 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad81001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad81001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35450/0 (socket says 172.21.15.16:35450) 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffad77fe700 1 -- 172.21.15.16:0/2673209780 learned_addr learned my addr 172.21.15.16:0/2673209780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffad81062a0 0x7ffad8100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.757+0000 7ffad7fff700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8100c40 0x7ffad81a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad77fe700 1 -- 172.21.15.16:0/2673209780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8100c40 msgr2=0x7ffad81a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad77fe700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8100c40 0x7ffad81a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad77fe700 1 -- 172.21.15.16:0/2673209780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffad81062a0 msgr2=0x7ffad8100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad77fe700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffad81062a0 0x7ffad8100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad77fe700 1 -- 172.21.15.16:0/2673209780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80095f0 con 0x7ffad8069400 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad7fff700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8100c40 0x7ffad81a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad77fe700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad81001c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ffac8004730 tx=0x7ffac8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad4ff9700 1 -- 172.21.15.16:0/2673209780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac801d070 con 0x7ffad8069400 2024-05-08T23:29:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad4ff9700 1 -- 172.21.15.16:0/2673209780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffac800be80 con 0x7ffad8069400 2024-05-08T23:29:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad81a6000 con 0x7ffad8069400 2024-05-08T23:29:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffad4ff9700 1 -- 172.21.15.16:0/2673209780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80173f0 con 0x7ffad8069400 2024-05-08T23:29:01.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.758+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81a6420 con 0x7ffad8069400 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.760+0000 7ffad4ff9700 1 -- 172.21.15.16:0/2673209780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffac8022700 con 0x7ffad8069400 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.761+0000 7ffad4ff9700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ffac40a1290 0x7ffac40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.761+0000 7ffad4ff9700 1 -- 172.21.15.16:0/2673209780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ffac80ce510 con 0x7ffad8069400 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.761+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ffac40a1290 0x7ffac40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.761+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ffac40a1290 0x7ffac40a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffad81018f0 tx=0x7ffac00073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 Processor -- start 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 -- start start 2024-05-08T23:29:01.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e97720 0x563cb8e91350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e91890 0x563cb8e91cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8de8150 con 0x563cb8e96e00 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8de5b10 con 0x563cb8e91890 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8de1400 con 0x563cb8e97720 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35452/0 (socket says 172.21.15.16:35452) 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.762+0000 7ffad77fe700 1 -- 172.21.15.16:0/1236951914 learned_addr learned my addr 172.21.15.16:0/1236951914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad7fff700 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e91890 0x563cb8e91cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad77fe700 1 -- 172.21.15.16:0/1236951914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e97720 msgr2=0x563cb8e91350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e97720 0x563cb8e91350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad77fe700 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e97720 0x563cb8e91350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad77fe700 1 -- 172.21.15.16:0/1236951914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e91890 msgr2=0x563cb8e91cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad77fe700 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e91890 0x563cb8e91cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad77fe700 1 -- 172.21.15.16:0/1236951914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb8c623b0 con 0x563cb8e96e00 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad7fff700 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e91890 0x563cb8e91cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffad77fe700 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e971e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ffac8097d20 tx=0x7ffac809ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54352bc98cfcb3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffacdffb700 1 -- 172.21.15.16:0/1236951914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac809f0e0 con 0x563cb8e96e00 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffacdffb700 1 -- 172.21.15.16:0/1236951914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffac80a36a0 con 0x563cb8e96e00 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.763+0000 7ffacdffb700 1 -- 172.21.15.16:0/1236951914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac809b770 con 0x563cb8e96e00 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/1236951914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 msgr2=0x563cb8e971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e971e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ffac8097d20 tx=0x7ffac809ac10 comp rx=0 tx=0).stop 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/1236951914 shutdown_connections 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e91890 0x563cb8e91cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e971e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/1236951914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e97720 0x563cb8e91350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/1236951914 >> 172.21.15.16:0/1236951914 conn(0x563cb8def210 msgr2=0x563cb8e8bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/1236951914 shutdown_connections 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/1236951914 wait complete. 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 -- start start 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.764+0000 7ffae9f8fb80 1 Processor -- start 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8e9ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e9b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8d7d560 con 0x563cb8e96e00 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8d7d200 con 0x563cb8e97720 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8c646a0 con 0x563cb8e91890 2024-05-08T23:29:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffae9f8fb80 10 client.0 _ll_register_callbacks cb 0x563cb8d86b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e9b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e9b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35458/0 (socket says 172.21.15.16:35458) 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 -- 172.21.15.16:0/442541597 learned_addr learned my addr 172.21.15.16:0/442541597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad77fe700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8e9ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad7fff700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 -- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 msgr2=0x563cb8e9ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8e9ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 -- 172.21.15.16:0/442541597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 msgr2=0x563cb8e9b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e9b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 -- 172.21.15.16:0/442541597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81a6420 con 0x563cb8e96e00 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad7fff700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e9b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e9b080 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004630 tx=0x7ffac00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ed8354c8e31a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.765+0000 7ffab6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffab6ffd700 1 -- 172.21.15.16:0/442541597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0003b10 con 0x563cb8e96e00 2024-05-08T23:29:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffab6ffd700 1 -- 172.21.15.16:0/442541597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ffac0017070 con 0x563cb8e96e00 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cb8d3a270 con 0x563cb8e96e00 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffab6ffd700 1 -- 172.21.15.16:0/442541597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac0003c70 con 0x563cb8e96e00 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 10 client.17568 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563cb8c0eb60 con 0x563cb8e96e00 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffab6ffd700 1 -- 172.21.15.16:0/442541597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563cb8d3a270 con 0x563cb8e96e00 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffab6ffd700 1 -- 172.21.15.16:0/442541597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffac001f400 con 0x563cb8e96e00 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 10 client.17568 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 10 client.17568 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 20 client.17568 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8eaaf80 0x563cb8ead3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 4 client.17568 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.766+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cb8e60810 con 0x563cb8eaaf80 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.767+0000 7ffad77fe700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8eaaf80 0x563cb8ead3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.767+0000 7ffab6ffd700 10 client.17568 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.767+0000 7ffad77fe700 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8eaaf80 0x563cb8ead3a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba4c93ed09d7d5cb server_cookie=8d5be4ce62a82bbc in_seq=0 out_seq=0 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffab6ffd700 1 -- 172.21.15.16:0/442541597 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cb8e60810 con 0x563cb8eaaf80 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffab6ffd700 10 client.17568 handle_command_reply: tid=0 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 client.17568 shutdown 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 msgr2=0x563cb8e9b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e9b080 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004630 tx=0x7ffac00047c0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 shutdown_connections 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8eaaf80 0x563cb8ead3a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e9b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8e9ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/442541597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8e9b080 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffab6ffd700 0 client.17568 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 >> 172.21.15.16:0/442541597 conn(0x563cb8def210 msgr2=0x563cb8e8b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 shutdown_connections 2024-05-08T23:29:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/442541597 wait complete. 2024-05-08T23:29:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.768+0000 7ffae9f8fb80 20 client.17568 trim_cache size 0 max 16384 2024-05-08T23:29:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.782+0000 7ffae9f8fb80 1 Processor -- start 2024-05-08T23:29:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.782+0000 7ffae9f8fb80 1 -- start start 2024-05-08T23:29:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8ea6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8ea6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e8bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8e9c140 con 0x563cb8e96e00 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8e9db10 con 0x563cb8e97720 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8e88dc0 con 0x563cb8e91890 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8ea6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8ea6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35466/0 (socket says 172.21.15.16:35466) 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 -- 172.21.15.16:0/446382268 learned_addr learned my addr 172.21.15.16:0/446382268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad77fe700 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8ea6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad7fff700 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e8bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 -- 172.21.15.16:0/446382268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 msgr2=0x563cb8ea6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8ea6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 -- 172.21.15.16:0/446382268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 msgr2=0x563cb8e8bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e8bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 -- 172.21.15.16:0/446382268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb8da7560 con 0x563cb8e96e00 2024-05-08T23:29:01.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8ea6fe0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000c00 tx=0x7ffac0007220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1d183e23d16c78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7fface7fc700 1 -- 172.21.15.16:0/446382268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac001fa80 con 0x563cb8e96e00 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7fface7fc700 1 -- 172.21.15.16:0/446382268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffac0003da0 con 0x563cb8e96e00 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.783+0000 7fface7fc700 1 -- 172.21.15.16:0/446382268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0011710 con 0x563cb8e96e00 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/446382268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 msgr2=0x563cb8ea6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8ea6fe0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000c00 tx=0x7ffac0007220 comp rx=0 tx=0).stop 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/446382268 shutdown_connections 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8e8bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e91890 0x563cb8ea6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/446382268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e96e00 0x563cb8ea6fe0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/446382268 >> 172.21.15.16:0/446382268 conn(0x563cb8def210 msgr2=0x563cb8e99430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/446382268 shutdown_connections 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/446382268 wait complete. 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.784+0000 7ffae9f8fb80 1 -- start start 2024-05-08T23:29:01.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 Processor -- start 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 0x563cb8e9fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e96e00 0x563cb8ea03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8ea08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8ea2820 con 0x563cb8e91890 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8f00480 con 0x563cb8e97720 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8f00610 con 0x563cb8e96e00 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffae9f8fb80 10 client.0 _ll_register_callbacks cb 0x563cb8d86b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 0x563cb8e9fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 0x563cb8e9fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35476/0 (socket says 172.21.15.16:35476) 2024-05-08T23:29:01.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 -- 172.21.15.16:0/4081631425 learned_addr learned my addr 172.21.15.16:0/4081631425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e96e00 0x563cb8ea03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad7fff700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8ea08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 -- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e96e00 msgr2=0x563cb8ea03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e96e00 0x563cb8ea03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 -- 172.21.15.16:0/4081631425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 msgr2=0x563cb8ea08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8ea08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.785+0000 7ffad77fe700 1 -- 172.21.15.16:0/4081631425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cb8c0eb60 con 0x563cb8e91890 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e96e00 0x563cb8ea03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffad77fe700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 0x563cb8e9fe60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ffac809afb0 tx=0x7ffac8005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3f88a6014c3bba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffab6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffab6ffd700 1 -- 172.21.15.16:0/4081631425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8022470 con 0x563cb8e91890 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffab6ffd700 1 -- 172.21.15.16:0/4081631425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ffac8099650 con 0x563cb8e91890 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac80a36a0 con 0x563cb8e91890 2024-05-08T23:29:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffab6ffd700 1 -- 172.21.15.16:0/4081631425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac80ce130 con 0x563cb8e91890 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 10 client.17574 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563cb8c66cd0 con 0x563cb8e91890 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffab6ffd700 1 -- 172.21.15.16:0/4081631425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffac80a36a0 con 0x563cb8e91890 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffab6ffd700 1 -- 172.21.15.16:0/4081631425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffac809b5a0 con 0x563cb8e91890 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 10 client.17574 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 10 client.17574 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 20 client.17574 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8f01ae0 0x563cb8f03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 4 client.17574 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.786+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563cb8dd9ec0 con 0x563cb8f01ae0 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.787+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8f01ae0 0x563cb8f03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.787+0000 7ffad6ffd700 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8f01ae0 0x563cb8f03ed0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=475200be94170252 server_cookie=8d8fd1e003faa780 in_seq=0 out_seq=0 2024-05-08T23:29:01.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.787+0000 7ffab6ffd700 10 client.17574 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.787+0000 7ffab6ffd700 1 -- 172.21.15.16:0/4081631425 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cb8dd9ec0 con 0x563cb8f01ae0 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.787+0000 7ffab6ffd700 10 client.17574 handle_command_reply: tid=0 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.787+0000 7ffae9f8fb80 1 client.17574 shutdown 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 msgr2=0x563cb8e9fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 0x563cb8e9fe60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ffac809afb0 tx=0x7ffac8005940 comp rx=0 tx=0).stop 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 shutdown_connections 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x563cb8f01ae0 0x563cb8f03ed0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cb8e97720 0x563cb8ea08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cb8e91890 0x563cb8e9fe60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffab6ffd700 0 client.17574 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 --2- 172.21.15.16:0/4081631425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8e96e00 0x563cb8ea03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 >> 172.21.15.16:0/4081631425 conn(0x563cb8def210 msgr2=0x563cb8e95c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 shutdown_connections 2024-05-08T23:29:01.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 1 -- 172.21.15.16:0/4081631425 wait complete. 2024-05-08T23:29:01.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffae9f8fb80 20 client.17574 trim_cache size 0 max 16384 2024-05-08T23:29:01.794 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:29:01.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ffac40a1290 msgr2=0x7ffac40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffadddd5700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ffac40a1290 0x7ffac40a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffad81018f0 tx=0x7ffac00073d0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 msgr2=0x7ffad81001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffadddd5700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad81001c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ffac8004730 tx=0x7ffac8004760 comp rx=0 tx=0).stop 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.788+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 shutdown_connections 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ffac40a1290 0x7ffac40a3750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad8100c40 0x7ffad81a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffad8069400 0x7ffad81001c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 --2- 172.21.15.16:0/2673209780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffad81062a0 0x7ffad8100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 >> 172.21.15.16:0/2673209780 conn(0x7ffad8076ba0 msgr2=0x7ffad80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 shutdown_connections 2024-05-08T23:29:01.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.789+0000 7ffadddd5700 1 -- 172.21.15.16:0/2673209780 wait complete. 2024-05-08T23:29:01.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 -- 172.21.15.16:0/3064681340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac101b10 msgr2=0x7f28ac105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 --2- 172.21.15.16:0/3064681340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac101b10 0x7f28ac105b50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2894009990 tx=0x7f2894009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 -- 172.21.15.16:0/3064681340 shutdown_connections 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 --2- 172.21.15.16:0/3064681340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28ac106280 0x7f28ac10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 --2- 172.21.15.16:0/3064681340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac101160 0x7f28ac101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 --2- 172.21.15.16:0/3064681340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac101b10 0x7f28ac105b50 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 -- 172.21.15.16:0/3064681340 >> 172.21.15.16:0/3064681340 conn(0x7f28ac0fc9e0 msgr2=0x7f28ac0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 -- 172.21.15.16:0/3064681340 shutdown_connections 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.941+0000 7f28b210b700 1 -- 172.21.15.16:0/3064681340 wait complete. 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 Processor -- start 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 -- start start 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28ac101160 0x7f28ac1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 0x7f28ac197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac19bc50 0x7f28ac19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac19c6d0 con 0x7f28ac106280 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac19c840 con 0x7f28ac19bc50 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28b210b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac19ca00 con 0x7f28ac101160 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 0x7f28ac197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 0x7f28ac197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35498/0 (socket says 172.21.15.16:35498) 2024-05-08T23:29:01.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28a3fff700 1 -- 172.21.15.16:0/1855042506 learned_addr learned my addr 172.21.15.16:0/1855042506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28ac101160 0x7f28ac1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.942+0000 7f28abfff700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac19bc50 0x7f28ac19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a3fff700 1 -- 172.21.15.16:0/1855042506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28ac101160 msgr2=0x7f28ac1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a3fff700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28ac101160 0x7f28ac1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a3fff700 1 -- 172.21.15.16:0/1855042506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac19bc50 msgr2=0x7f28ac19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a3fff700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac19bc50 0x7f28ac19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a3fff700 1 -- 172.21.15.16:0/1855042506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940095f0 con 0x7f28ac106280 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28abfff700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac19bc50 0x7f28ac19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a3fff700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 0x7f28ac197900 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2894005770 tx=0x7f28940057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a97fa700 1 -- 172.21.15.16:0/1855042506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289401d070 con 0x7f28ac106280 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac1a3f00 con 0x7f28ac106280 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a97fa700 1 -- 172.21.15.16:0/1855042506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2894022470 con 0x7f28ac106280 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28a97fa700 1 -- 172.21.15.16:0/1855042506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28940173f0 con 0x7f28ac106280 2024-05-08T23:29:01.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.943+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac1a42c0 con 0x7f28ac106280 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.945+0000 7f28a97fa700 1 -- 172.21.15.16:0/1855042506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2894022800 con 0x7f28ac106280 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.946+0000 7f28a97fa700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28980a1290 0x7f28980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.946+0000 7f28a97fa700 1 -- 172.21.15.16:0/1855042506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f28940ce300 con 0x7f28ac106280 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.946+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28980a1290 0x7f28980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28980a1290 0x7f28980a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f289c009c60 tx=0x7f289c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 Processor -- start 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 -- start start 2024-05-08T23:29:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee251f8150 con 0x55ee252a6e00 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee251f5b10 con 0x55ee252a1890 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee251f1400 con 0x55ee252a7720 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.947+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252a71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35512/0 (socket says 172.21.15.16:35512) 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 -- 172.21.15.16:0/3756062827 learned_addr learned my addr 172.21.15.16:0/3756062827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28a3fff700 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28abfff700 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 -- 172.21.15.16:0/3756062827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 msgr2=0x55ee252a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 -- 172.21.15.16:0/3756062827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 msgr2=0x55ee252a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 -- 172.21.15.16:0/3756062827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee250723b0 con 0x55ee252a6e00 2024-05-08T23:29:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28abfff700 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252a71e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f289c00e4e0 tx=0x7f289c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7e5bc51b7f02a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28a1ffb700 1 -- 172.21.15.16:0/3756062827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c00ee00 con 0x55ee252a6e00 2024-05-08T23:29:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28a1ffb700 1 -- 172.21.15.16:0/3756062827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f289c021070 con 0x55ee252a6e00 2024-05-08T23:29:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.948+0000 7f28a1ffb700 1 -- 172.21.15.16:0/3756062827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c01d780 con 0x55ee252a6e00 2024-05-08T23:29:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 -- 172.21.15.16:0/3756062827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 msgr2=0x55ee252a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252a71e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f289c00e4e0 tx=0x7f289c014000 comp rx=0 tx=0).stop 2024-05-08T23:29:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 -- 172.21.15.16:0/3756062827 shutdown_connections 2024-05-08T23:29:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252a71e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/3756062827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 -- 172.21.15.16:0/3756062827 >> 172.21.15.16:0/3756062827 conn(0x55ee251ff210 msgr2=0x55ee2529bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 -- 172.21.15.16:0/3756062827 shutdown_connections 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 -- 172.21.15.16:0/3756062827 wait complete. 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.949+0000 7f28be2c5b80 1 -- start start 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 Processor -- start 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 0x55ee252aab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a6e00 0x55ee252ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2518d560 con 0x55ee252a1890 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2518d200 con 0x55ee252a6e00 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee250746a0 con 0x55ee252a7720 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28be2c5b80 10 client.0 _ll_register_callbacks cb 0x55ee25196b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 0x55ee252aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 0x55ee252aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35516/0 (socket says 172.21.15.16:35516) 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 -- 172.21.15.16:0/2295470412 learned_addr learned my addr 172.21.15.16:0/2295470412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a6e00 0x55ee252ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28abfff700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 -- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 msgr2=0x55ee252ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 -- 172.21.15.16:0/2295470412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a6e00 msgr2=0x55ee252ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.950+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a6e00 0x55ee252ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28ab7fe700 1 -- 172.21.15.16:0/2295470412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c017070 con 0x55ee252a1890 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28abfff700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 0x55ee252aab90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f289c00ee10 tx=0x7f289c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a2d6cbbcfddd83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f288affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f288affd700 1 -- 172.21.15.16:0/2295470412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c0054b0 con 0x55ee252a1890 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f288affd700 1 -- 172.21.15.16:0/2295470412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f289c005610 con 0x55ee252a1890 2024-05-08T23:29:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee2514a270 con 0x55ee252a1890 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f288affd700 1 -- 172.21.15.16:0/2295470412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f289c003680 con 0x55ee252a1890 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28be2c5b80 10 client.17586 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee2501eb60 con 0x55ee252a1890 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f288affd700 1 -- 172.21.15.16:0/2295470412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee2514a270 con 0x55ee252a1890 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f288affd700 1 -- 172.21.15.16:0/2295470412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f289c01c830 con 0x55ee252a1890 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28be2c5b80 10 client.17586 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.951+0000 7f28be2c5b80 10 client.17586 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f28be2c5b80 20 client.17586 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee252bb0c0 0x55ee252bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f28be2c5b80 4 client.17586 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee25270810 con 0x55ee252bb0c0 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee252bb0c0 0x55ee252bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f288affd700 10 client.17586 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.952+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee252bb0c0 0x55ee252bd4e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5920b53d0033fca7 server_cookie=e45c271faaac29fb in_seq=0 out_seq=0 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f288affd700 1 -- 172.21.15.16:0/2295470412 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee25270810 con 0x55ee252bb0c0 2024-05-08T23:29:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f288affd700 10 client.17586 handle_command_reply: tid=0 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 client.17586 shutdown 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 msgr2=0x55ee252aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 0x55ee252aab90 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f289c00ee10 tx=0x7f289c00eef0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 shutdown_connections 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee252ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a1890 0x55ee252aab90 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee252bb0c0 0x55ee252bd4e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2295470412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a6e00 0x55ee252ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f288affd700 0 client.17586 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 >> 172.21.15.16:0/2295470412 conn(0x55ee251ff210 msgr2=0x55ee2529b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 shutdown_connections 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2295470412 wait complete. 2024-05-08T23:29:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.953+0000 7f28be2c5b80 20 client.17586 trim_cache size 0 max 16384 2024-05-08T23:29:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.967+0000 7f28be2c5b80 1 Processor -- start 2024-05-08T23:29:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 -- start start 2024-05-08T23:29:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee2529d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee2529bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee252c26f0 con 0x55ee252a6e00 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee2529cc90 con 0x55ee252a1890 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee252a90e0 con 0x55ee252a7720 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee2529d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee2529d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35518/0 (socket says 172.21.15.16:35518) 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 -- 172.21.15.16:0/2059841438 learned_addr learned my addr 172.21.15.16:0/2059841438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28abfff700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee2529bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 -- 172.21.15.16:0/2059841438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 msgr2=0x55ee2529bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee2529bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 -- 172.21.15.16:0/2059841438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 msgr2=0x55ee252bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252bd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 -- 172.21.15.16:0/2059841438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee251b7560 con 0x55ee252a6e00 2024-05-08T23:29:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28abfff700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee2529bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.968+0000 7f28a3fff700 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee2529d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f289409b030 tx=0x7f28940cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e32cbbf58fd9d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28a27fc700 1 -- 172.21.15.16:0/2059841438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289409eed0 con 0x55ee252a6e00 2024-05-08T23:29:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28a27fc700 1 -- 172.21.15.16:0/2059841438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28940a3490 con 0x55ee252a6e00 2024-05-08T23:29:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28a27fc700 1 -- 172.21.15.16:0/2059841438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289409b560 con 0x55ee252a6e00 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2059841438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 msgr2=0x55ee2529d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee2529d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f289409b030 tx=0x7f28940cd040 comp rx=0 tx=0).stop 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2059841438 shutdown_connections 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a7720 0x55ee2529bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a1890 0x55ee252bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/2059841438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee2529d990 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2059841438 >> 172.21.15.16:0/2059841438 conn(0x55ee251ff210 msgr2=0x55ee25298210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2059841438 shutdown_connections 2024-05-08T23:29:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.969+0000 7f28be2c5b80 1 -- 172.21.15.16:0/2059841438 wait complete. 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 -- start start 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 Processor -- start 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a1890 0x55ee252af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a7720 0x55ee25309580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee25309ac0 con 0x55ee252a6e00 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee25309c30 con 0x55ee252a7720 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee25309df0 con 0x55ee252a1890 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28be2c5b80 10 client.0 _ll_register_callbacks cb 0x55ee25196b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28a3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35526/0 (socket says 172.21.15.16:35526) 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.970+0000 7f28a3fff700 1 -- 172.21.15.16:0/255576362 learned_addr learned my addr 172.21.15.16:0/255576362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a1890 0x55ee252af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28abfff700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a7720 0x55ee25309580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28a3fff700 1 -- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a1890 msgr2=0x55ee252af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28a3fff700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a1890 0x55ee252af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28a3fff700 1 -- 172.21.15.16:0/255576362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a7720 msgr2=0x55ee25309580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28a3fff700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a7720 0x55ee25309580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28a3fff700 1 -- 172.21.15.16:0/255576362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac1a42c0 con 0x55ee252a6e00 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28a3fff700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252afa80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2894005740 tx=0x7f289409b220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b926510c208f9e39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f288affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f288affd700 1 -- 172.21.15.16:0/255576362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28940a10d0 con 0x55ee252a6e00 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f288affd700 1 -- 172.21.15.16:0/255576362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f28940b2030 con 0x55ee252a6e00 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f289c021070 con 0x55ee252a6e00 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f288affd700 1 -- 172.21.15.16:0/255576362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28940b92b0 con 0x55ee252a6e00 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28be2c5b80 10 client.17592 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.971+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee25076cd0 con 0x55ee252a6e00 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f288affd700 1 -- 172.21.15.16:0/255576362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28940a9020 con 0x55ee252a6e00 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f288affd700 1 -- 172.21.15.16:0/255576362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f289409b300 con 0x55ee252a6e00 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28be2c5b80 10 client.17592 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28be2c5b80 10 client.17592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28be2c5b80 20 client.17592 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee25311100 0x55ee25313520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28be2c5b80 4 client.17592 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ee251e9ec0 con 0x55ee25311100 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee25311100 0x55ee25313520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f288affd700 10 client.17592 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.972+0000 7f28ab7fe700 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee25311100 0x55ee25313520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d697a8be23336d87 server_cookie=469d7d8700c7443 in_seq=0 out_seq=0 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f288affd700 1 -- 172.21.15.16:0/255576362 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ee251e9ec0 con 0x55ee25311100 2024-05-08T23:29:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f288affd700 10 client.17592 handle_command_reply: tid=0 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 client.17592 shutdown 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 msgr2=0x55ee252afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252afa80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2894005740 tx=0x7f289409b220 comp rx=0 tx=0).stop 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 shutdown_connections 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55ee25311100 0x55ee25313520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ee252a7720 0x55ee25309580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f288affd700 0 client.17592 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee252a1890 0x55ee252af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 --2- 172.21.15.16:0/255576362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee252a6e00 0x55ee252afa80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 >> 172.21.15.16:0/255576362 conn(0x55ee251ff210 msgr2=0x55ee252ab470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 shutdown_connections 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 1 -- 172.21.15.16:0/255576362 wait complete. 2024-05-08T23:29:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28be2c5b80 20 client.17592 trim_cache size 0 max 16384 2024-05-08T23:29:01.980 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:29:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28980a1290 msgr2=0x7f28980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.973+0000 7f28b210b700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28980a1290 0x7f28980a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f289c009c60 tx=0x7f289c009260 comp rx=0 tx=0).stop 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 msgr2=0x7f28ac197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 0x7f28ac197900 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2894005770 tx=0x7f28940057a0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 shutdown_connections 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f28980a1290 0x7f28980a3750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28ac19bc50 0x7f28ac19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28ac101160 0x7f28ac1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 --2- 172.21.15.16:0/1855042506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac106280 0x7f28ac197900 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 >> 172.21.15.16:0/1855042506 conn(0x7f28ac0fc9e0 msgr2=0x7f28ac0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 shutdown_connections 2024-05-08T23:29:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:01.974+0000 7f28b210b700 1 -- 172.21.15.16:0/1855042506 wait complete. 2024-05-08T23:29:01.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:29:02.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.126+0000 7fedfbabd700 1 -- 172.21.15.16:0/1548073441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 msgr2=0x7fedf410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.126+0000 7fedfbabd700 1 --2- 172.21.15.16:0/1548073441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf410de00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b0a0 tx=0x7fedec00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 -- 172.21.15.16:0/1548073441 shutdown_connections 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 --2- 172.21.15.16:0/1548073441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf410de00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 --2- 172.21.15.16:0/1548073441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf4101160 0x7fedf4101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 --2- 172.21.15.16:0/1548073441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedf4101b10 0x7fedf4105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 -- 172.21.15.16:0/1548073441 >> 172.21.15.16:0/1548073441 conn(0x7fedf40fc9e0 msgr2=0x7fedf40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 -- 172.21.15.16:0/1548073441 shutdown_connections 2024-05-08T23:29:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 -- 172.21.15.16:0/1548073441 wait complete. 2024-05-08T23:29:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 Processor -- start 2024-05-08T23:29:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.127+0000 7fedfbabd700 1 -- start start 2024-05-08T23:29:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfbabd700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf4101160 0x7fedf4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfbabd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfbabd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedf419bd00 0x7fedf419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfbabd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419c780 con 0x7fedf4106280 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfbabd700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419c8f0 con 0x7fedf4101160 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfbabd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf419cab0 con 0x7fedf419bd00 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf41979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35540/0 (socket says 172.21.15.16:35540) 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 -- 172.21.15.16:0/3987276310 learned_addr learned my addr 172.21.15.16:0/3987276310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedfa05a700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedf419bd00 0x7fedf419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9859700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf4101160 0x7fedf4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 -- 172.21.15.16:0/3987276310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedf419bd00 msgr2=0x7fedf419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedf419bd00 0x7fedf419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 -- 172.21.15.16:0/3987276310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf4101160 msgr2=0x7fedf4197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf4101160 0x7fedf4197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 -- 172.21.15.16:0/3987276310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec009d00 con 0x7fedf4106280 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.128+0000 7fedf9058700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf41979b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e970 tx=0x7fedf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.129+0000 7fede6ffd700 1 -- 172.21.15.16:0/3987276310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0009790 con 0x7fedf4106280 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.129+0000 7fede6ffd700 1 -- 172.21.15.16:0/3987276310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedf000c7e0 con 0x7fedf4106280 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.129+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf41a3fb0 con 0x7fedf4106280 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.129+0000 7fede6ffd700 1 -- 172.21.15.16:0/3987276310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0005440 con 0x7fedf4106280 2024-05-08T23:29:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.129+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf41a4490 con 0x7fedf4106280 2024-05-08T23:29:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.131+0000 7fede6ffd700 1 -- 172.21.15.16:0/3987276310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fedf000c950 con 0x7fedf4106280 2024-05-08T23:29:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.131+0000 7fede6ffd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fede00a1290 0x7fede00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.132+0000 7fede6ffd700 1 -- 172.21.15.16:0/3987276310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fedf0014070 con 0x7fedf4106280 2024-05-08T23:29:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.132+0000 7fedf9859700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fede00a1290 0x7fede00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.132+0000 7fedf9859700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fede00a1290 0x7fede00a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fede8009c60 tx=0x7fede8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.132+0000 7fee07c77b80 1 Processor -- start 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.132+0000 7fee07c77b80 1 -- start start 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda9720 0x55e93cda3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 0x55e93cda3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fee07c77b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ccfa150 con 0x55e93cda3890 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fee07c77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ccf7b10 con 0x55e93cda9720 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fee07c77b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ccf3400 con 0x55e93cda8e00 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 0x55e93cda3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 0x55e93cda3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35556/0 (socket says 172.21.15.16:35556) 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedfa05a700 1 -- 172.21.15.16:0/3166470250 learned_addr learned my addr 172.21.15.16:0/3166470250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9859700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9058700 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda9720 0x55e93cda3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9859700 1 -- 172.21.15.16:0/3166470250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda9720 msgr2=0x55e93cda3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9859700 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda9720 0x55e93cda3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9859700 1 -- 172.21.15.16:0/3166470250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 msgr2=0x55e93cda3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9859700 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 0x55e93cda3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedf9859700 1 -- 172.21.15.16:0/3166470250 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93cb743b0 con 0x55e93cda8e00 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.133+0000 7fedfa05a700 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 0x55e93cda3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.134+0000 7fedf9859700 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda91e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fede800e3b0 tx=0x7fede8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7440f016d5a2eb24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.134+0000 7feddffff700 1 -- 172.21.15.16:0/3166470250 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede800ed50 con 0x55e93cda8e00 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.134+0000 7feddffff700 1 -- 172.21.15.16:0/3166470250 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fede8021070 con 0x55e93cda8e00 2024-05-08T23:29:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.134+0000 7feddffff700 1 -- 172.21.15.16:0/3166470250 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8003dd0 con 0x55e93cda8e00 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.134+0000 7fee07c77b80 1 -- 172.21.15.16:0/3166470250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 msgr2=0x55e93cda91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.134+0000 7fee07c77b80 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda91e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fede800e3b0 tx=0x7fede8014000 comp rx=0 tx=0).stop 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 -- 172.21.15.16:0/3166470250 shutdown_connections 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda3890 0x55e93cda3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda91e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 --2- 172.21.15.16:0/3166470250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda9720 0x55e93cda3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 -- 172.21.15.16:0/3166470250 >> 172.21.15.16:0/3166470250 conn(0x55e93cd01210 msgr2=0x55e93cd9ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 -- 172.21.15.16:0/3166470250 shutdown_connections 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 -- 172.21.15.16:0/3166470250 wait complete. 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 -- start start 2024-05-08T23:29:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.135+0000 7fee07c77b80 1 Processor -- start 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cc8f560 con 0x55e93cda9720 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cc8f200 con 0x55e93cda8e00 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cb766a0 con 0x55e93cda3890 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fee07c77b80 10 client.0 _ll_register_callbacks cb 0x55e93cc98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35564/0 (socket says 172.21.15.16:35564) 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 -- 172.21.15.16:0/4163123578 learned_addr learned my addr 172.21.15.16:0/4163123578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedf9859700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedf9058700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 -- 172.21.15.16:0/4163123578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 msgr2=0x55e93cdacbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdacbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 -- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 msgr2=0x55e93cdad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 -- 172.21.15.16:0/4163123578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf410e670 con 0x55e93cda9720 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedf9058700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7fedfa05a700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdad670 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fedec007990 tx=0x7fedec00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d446739569998fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7feddd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.136+0000 7feddd7fa700 1 -- 172.21.15.16:0/4163123578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00e040 con 0x55e93cda9720 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7feddd7fa700 1 -- 172.21.15.16:0/4163123578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fedec007c20 con 0x55e93cda9720 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e93cc4c270 con 0x55e93cda9720 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7feddd7fa700 1 -- 172.21.15.16:0/4163123578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedec007de0 con 0x55e93cda9720 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 10 client.17604 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fede8017070 con 0x55e93cda9720 2024-05-08T23:29:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7feddd7fa700 1 -- 172.21.15.16:0/4163123578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e93cc4c270 con 0x55e93cda9720 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7feddd7fa700 1 -- 172.21.15.16:0/4163123578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fedec023030 con 0x55e93cda9720 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 10 client.17604 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 20 client.17604 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93cdbd240 0x55e93cdbf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 4 client.17604 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e93cd72810 con 0x55e93cdbd240 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.137+0000 7fedf9859700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93cdbd240 0x55e93cdbf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.138+0000 7fedf9859700 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93cdbd240 0x55e93cdbf660 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33eb92a48e5bf5f server_cookie=d8018e4bbc477407 in_seq=0 out_seq=0 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.138+0000 7feddd7fa700 10 client.17604 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7feddd7fa700 1 -- 172.21.15.16:0/4163123578 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e93cd72810 con 0x55e93cdbd240 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7feddd7fa700 10 client.17604 handle_command_reply: tid=0 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 client.17604 shutdown 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 msgr2=0x55e93cdad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdad670 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fedec007990 tx=0x7fedec00ba50 comp rx=0 tx=0).stop 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 shutdown_connections 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdad670 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdacbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93cdbd240 0x55e93cdbf660 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 --2- 172.21.15.16:0/4163123578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 >> 172.21.15.16:0/4163123578 conn(0x55e93cd01210 msgr2=0x55e93cd9d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7feddd7fa700 0 client.17604 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 shutdown_connections 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 1 -- 172.21.15.16:0/4163123578 wait complete. 2024-05-08T23:29:02.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.139+0000 7fee07c77b80 20 client.17604 trim_cache size 0 max 16384 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.155+0000 7fee07c77b80 1 Processor -- start 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.155+0000 7fee07c77b80 1 -- start start 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.155+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda3890 0x55e93cd9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.155+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.155+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cda8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.155+0000 7fee07c77b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cdbd0b0 con 0x55e93cda9720 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fee07c77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cdae220 con 0x55e93cda3890 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fee07c77b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93cdc4870 con 0x55e93cda8e00 2024-05-08T23:29:02.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cda8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cda8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35570/0 (socket says 172.21.15.16:35570) 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 -- 172.21.15.16:0/2487537074 learned_addr learned my addr 172.21.15.16:0/2487537074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedf9859700 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda3890 0x55e93cd9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedf9058700 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 -- 172.21.15.16:0/2487537074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 msgr2=0x55e93cda7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 -- 172.21.15.16:0/2487537074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda3890 msgr2=0x55e93cd9dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda3890 0x55e93cd9dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 -- 172.21.15.16:0/2487537074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93cb20b60 con 0x55e93cda9720 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedf9859700 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda3890 0x55e93cd9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.156+0000 7fedfa05a700 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cda8210 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fedec009690 tx=0x7fedec0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba96b5230e6a890 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.158+0000 7fede4da4700 1 -- 172.21.15.16:0/2487537074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00e070 con 0x55e93cda9720 2024-05-08T23:29:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.158+0000 7fede4da4700 1 -- 172.21.15.16:0/2487537074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fedec003a00 con 0x55e93cda9720 2024-05-08T23:29:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.158+0000 7fede4da4700 1 -- 172.21.15.16:0/2487537074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec0053c0 con 0x55e93cda9720 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 -- 172.21.15.16:0/2487537074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 msgr2=0x55e93cda8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cda8210 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fedec009690 tx=0x7fedec0099e0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 -- 172.21.15.16:0/2487537074 shutdown_connections 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cda8210 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda3890 0x55e93cd9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 --2- 172.21.15.16:0/2487537074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda8e00 0x55e93cda7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 -- 172.21.15.16:0/2487537074 >> 172.21.15.16:0/2487537074 conn(0x55e93cd01210 msgr2=0x55e93cd9f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 -- 172.21.15.16:0/2487537074 shutdown_connections 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 -- 172.21.15.16:0/2487537074 wait complete. 2024-05-08T23:29:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.159+0000 7fee07c77b80 1 -- start start 2024-05-08T23:29:02.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 Processor -- start 2024-05-08T23:29:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdb2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdb2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdb2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ce16b50 con 0x55e93cda9720 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ce16c90 con 0x55e93cda8e00 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93ce16dd0 con 0x55e93cda3890 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fee07c77b80 10 client.0 _ll_register_callbacks cb 0x55e93cc98b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.160+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdb2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdb2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35584/0 (socket says 172.21.15.16:35584) 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 -- 172.21.15.16:0/558273876 learned_addr learned my addr 172.21.15.16:0/558273876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedf9058700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdb2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedf9859700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdb2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 -- 172.21.15.16:0/558273876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 msgr2=0x55e93cdb2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdb2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 -- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 msgr2=0x55e93cdb2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdb2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 -- 172.21.15.16:0/558273876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede8017070 con 0x55e93cda9720 2024-05-08T23:29:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fedfa05a700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdb2bd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b780 tx=0x7fedec007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdc7ca6a987fd35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7feddd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7feddd7fa700 1 -- 172.21.15.16:0/558273876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec00e070 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7feddd7fa700 1 -- 172.21.15.16:0/558273876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fedec005070 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fede8021070 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7feddd7fa700 1 -- 172.21.15.16:0/558273876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedec005230 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fee07c77b80 10 client.17610 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.161+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e93ccb9560 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7feddd7fa700 1 -- 172.21.15.16:0/558273876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fede8021070 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7feddd7fa700 1 -- 172.21.15.16:0/558273876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fedec023070 con 0x55e93cda9720 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fee07c77b80 10 client.17610 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fee07c77b80 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fee07c77b80 20 client.17610 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fee07c77b80 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93ce18010 0x55e93ce1a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fee07c77b80 4 client.17610 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e93ccebec0 con 0x55e93ce18010 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fedf9859700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93ce18010 0x55e93ce1a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7feddd7fa700 10 client.17610 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.162+0000 7fedf9859700 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93ce18010 0x55e93ce1a400 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9203f7e43dbfbb server_cookie=640e2f4dae5003c7 in_seq=0 out_seq=0 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7feddd7fa700 1 -- 172.21.15.16:0/558273876 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e93ccebec0 con 0x55e93ce18010 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7feddd7fa700 10 client.17610 handle_command_reply: tid=0 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 client.17610 shutdown 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 msgr2=0x55e93cdb2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdb2bd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fedec00b780 tx=0x7fedec007f70 comp rx=0 tx=0).stop 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 shutdown_connections 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e93cda9720 0x55e93cdb2bd0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93cda3890 0x55e93cdb2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55e93ce18010 0x55e93ce1a400 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 --2- 172.21.15.16:0/558273876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e93cda8e00 0x55e93cdb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7feddd7fa700 0 client.17610 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:02.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 >> 172.21.15.16:0/558273876 conn(0x55e93cd01210 msgr2=0x55e93cda7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 shutdown_connections 2024-05-08T23:29:02.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 1 -- 172.21.15.16:0/558273876 wait complete. 2024-05-08T23:29:02.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.163+0000 7fee07c77b80 20 client.17610 trim_cache size 0 max 16384 2024-05-08T23:29:02.167 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fede00a1290 msgr2=0x7fede00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fede00a1290 0x7fede00a3750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fede8009c60 tx=0x7fede8009260 comp rx=0 tx=0).stop 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 msgr2=0x7fedf41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf41979b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e970 tx=0x7fedf000ec80 comp rx=0 tx=0).stop 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 shutdown_connections 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fede00a1290 0x7fede00a3750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedf419bd00 0x7fedf419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedf4101160 0x7fedf4197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 --2- 172.21.15.16:0/3987276310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf4106280 0x7fedf41979b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 >> 172.21.15.16:0/3987276310 conn(0x7fedf40fc9e0 msgr2=0x7fedf40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 shutdown_connections 2024-05-08T23:29:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:02.164+0000 7fedfbabd700 1 -- 172.21.15.16:0/3987276310 wait complete. 2024-05-08T23:29:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:02 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2869484867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:29:02.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:02 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2869484867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:29:02.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:02 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2869484867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:29:03.177 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:29:03.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:29:03.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 -- 172.21.15.16:0/190135374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 msgr2=0x7f6d9c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 --2- 172.21.15.16:0/190135374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c101540 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009990 tx=0x7f6d90009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 -- 172.21.15.16:0/190135374 shutdown_connections 2024-05-08T23:29:03.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 --2- 172.21.15.16:0/190135374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c106280 0x7f6d9c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 --2- 172.21.15.16:0/190135374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c101540 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 --2- 172.21.15.16:0/190135374 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c101b10 0x7f6d9c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 -- 172.21.15.16:0/190135374 >> 172.21.15.16:0/190135374 conn(0x7f6d9c0fc9e0 msgr2=0x7f6d9c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 -- 172.21.15.16:0/190135374 shutdown_connections 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.328+0000 7f6da3075700 1 -- 172.21.15.16:0/190135374 wait complete. 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 Processor -- start 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 -- start start 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c106280 0x7f6d9c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c19bc60 0x7f6d9c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c19c6e0 con 0x7f6d9c101160 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c19c850 con 0x7f6d9c106280 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da3075700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c19ca10 con 0x7f6d9c19bc60 2024-05-08T23:29:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35600/0 (socket says 172.21.15.16:35600) 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 -- 172.21.15.16:0/2192006365 learned_addr learned my addr 172.21.15.16:0/2192006365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c106280 0x7f6d9c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da1612700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c19bc60 0x7f6d9c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 -- 172.21.15.16:0/2192006365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c19bc60 msgr2=0x7f6d9c19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c19bc60 0x7f6d9c19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 -- 172.21.15.16:0/2192006365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c106280 msgr2=0x7f6d9c1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c106280 0x7f6d9c1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da0e11700 1 -- 172.21.15.16:0/2192006365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d900095f0 con 0x7f6d9c101160 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6da1612700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c19bc60 0x7f6d9c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.329+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c106280 0x7f6d9c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.330+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c197380 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6d900036c0 tx=0x7f6d90005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.330+0000 7f6d99ffb700 1 -- 172.21.15.16:0/2192006365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9001d070 con 0x7f6d9c101160 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.330+0000 7f6d99ffb700 1 -- 172.21.15.16:0/2192006365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d9000fc00 con 0x7f6d9c101160 2024-05-08T23:29:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.330+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c1a3f10 con 0x7f6d9c101160 2024-05-08T23:29:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.330+0000 7f6d99ffb700 1 -- 172.21.15.16:0/2192006365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d90022c60 con 0x7f6d9c101160 2024-05-08T23:29:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.330+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c1a4330 con 0x7f6d9c101160 2024-05-08T23:29:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.333+0000 7f6d99ffb700 1 -- 172.21.15.16:0/2192006365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6d90022470 con 0x7f6d9c101160 2024-05-08T23:29:03.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.334+0000 7f6d99ffb700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d8c0a1310 0x7f6d8c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.334+0000 7f6d99ffb700 1 -- 172.21.15.16:0/2192006365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6d900ce550 con 0x7f6d9c101160 2024-05-08T23:29:03.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.334+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d8c0a1310 0x7f6d8c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.334+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d8c0a1310 0x7f6d8c0a37d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005950 tx=0x7f6d880058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.334+0000 7f6daf22fb80 1 Processor -- start 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 -- start start 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1417a2e80 0x55b1417a3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b14179d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b14179d910 0x55b14179dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1416f41d0 con 0x55b1417a37a0 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1416f1b90 con 0x55b1417a2e80 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1416ed480 con 0x55b14179d910 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b14179d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b14179d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35608/0 (socket says 172.21.15.16:35608) 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 -- 172.21.15.16:0/4193618996 learned_addr learned my addr 172.21.15.16:0/4193618996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6da0e11700 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1417a2e80 0x55b1417a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6da1612700 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b14179d910 0x55b14179dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 -- 172.21.15.16:0/4193618996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b14179d910 msgr2=0x55b14179dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b14179d910 0x55b14179dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 -- 172.21.15.16:0/4193618996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1417a2e80 msgr2=0x55b1417a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1417a2e80 0x55b1417a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 -- 172.21.15.16:0/4193618996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b14156e430 con 0x55b1417a37a0 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.335+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b14179d3d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b5d0 tx=0x7f6d88012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c602ee943cf20e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6d86ffd700 1 -- 172.21.15.16:0/4193618996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88017040 con 0x55b1417a37a0 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6d86ffd700 1 -- 172.21.15.16:0/4193618996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d8800fed0 con 0x55b1417a37a0 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6d86ffd700 1 -- 172.21.15.16:0/4193618996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8801fbc0 con 0x55b1417a37a0 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 -- 172.21.15.16:0/4193618996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 msgr2=0x55b14179d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b14179d3d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b5d0 tx=0x7f6d88012000 comp rx=0 tx=0).stop 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 -- 172.21.15.16:0/4193618996 shutdown_connections 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b14179d910 0x55b14179dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b1417a2e80 0x55b1417a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/4193618996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b14179d3d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.336+0000 7f6daf22fb80 1 -- 172.21.15.16:0/4193618996 >> 172.21.15.16:0/4193618996 conn(0x55b1416fb290 msgr2=0x55b141797e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.337+0000 7f6daf22fb80 1 -- 172.21.15.16:0/4193618996 shutdown_connections 2024-05-08T23:29:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.337+0000 7f6daf22fb80 1 -- 172.21.15.16:0/4193618996 wait complete. 2024-05-08T23:29:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.337+0000 7f6daf22fb80 1 -- start start 2024-05-08T23:29:03.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.337+0000 7f6daf22fb80 1 Processor -- start 2024-05-08T23:29:03.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1416895e0 con 0x55b1417a37a0 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141689280 con 0x55b14179d910 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b141570720 con 0x55b1417a2e80 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6daf22fb80 10 client.0 _ll_register_callbacks cb 0x55b141692bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35622/0 (socket says 172.21.15.16:35622) 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 -- 172.21.15.16:0/2720016031 learned_addr learned my addr 172.21.15.16:0/2720016031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 -- 172.21.15.16:0/2720016031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 msgr2=0x55b1417a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 -- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 msgr2=0x55b1417a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417a97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 -- 172.21.15.16:0/2720016031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8800fbb0 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6da1612700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a7200 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6d94007b70 tx=0x7f6d94007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a246a2550d7f1f37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.338+0000 7f6d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2720016031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94010860 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2720016031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6d940109c0 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1416462f0 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2720016031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d94009890 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 10 client.17622 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b14151abe0 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2720016031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b1416462f0 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2720016031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d94016bf0 con 0x55b1417a37a0 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 10 client.17622 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:03.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 20 client.17622 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1417b7300 0x55b1417b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 4 client.17622 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.339+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b14176c890 con 0x55b1417b7300 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.340+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1417b7300 0x55b1417b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.340+0000 7f6d7bfff700 10 client.17622 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.340+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1417b7300 0x55b1417b9720 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f61290fca3e1c4 server_cookie=359a11af5596a583 in_seq=0 out_seq=0 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2720016031 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b14176c890 con 0x55b1417b7300 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6d7bfff700 10 client.17622 handle_command_reply: tid=0 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 client.17622 shutdown 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 msgr2=0x55b1417a7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a7200 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6d94007b70 tx=0x7f6d94007e80 comp rx=0 tx=0).stop 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 shutdown_connections 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a7200 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1417b7300 0x55b1417b9720 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2720016031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6d7bfff700 0 client.17622 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 >> 172.21.15.16:0/2720016031 conn(0x55b1416fb290 msgr2=0x55b141797620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 shutdown_connections 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2720016031 wait complete. 2024-05-08T23:29:03.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.341+0000 7f6daf22fb80 20 client.17622 trim_cache size 0 max 16384 2024-05-08T23:29:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 Processor -- start 2024-05-08T23:29:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 -- start start 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b14179d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1417a7a60 con 0x55b1417a37a0 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1417a80f0 con 0x55b14179d910 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1417a15b0 con 0x55b1417a2e80 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6da1612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6da1612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35628/0 (socket says 172.21.15.16:35628) 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6da1612700 1 -- 172.21.15.16:0/1768050288 learned_addr learned my addr 172.21.15.16:0/1768050288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.360+0000 7f6da1612700 1 -- 172.21.15.16:0/1768050288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 msgr2=0x55b1417a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6da0e11700 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b14179d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6da1612700 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6da1612700 1 -- 172.21.15.16:0/1768050288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 msgr2=0x55b14179d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6da1612700 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b14179d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6da1612700 1 -- 172.21.15.16:0/1768050288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b14179dfc0 con 0x55b1417a37a0 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6da1612700 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a9300 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005b80 tx=0x7f6d94010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9109678fbf091f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6d877fe700 1 -- 172.21.15.16:0/1768050288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400d040 con 0x55b1417a37a0 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6d877fe700 1 -- 172.21.15.16:0/1768050288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d940097e0 con 0x55b1417a37a0 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.361+0000 7f6d877fe700 1 -- 172.21.15.16:0/1768050288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94015cb0 con 0x55b1417a37a0 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 -- 172.21.15.16:0/1768050288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 msgr2=0x55b1417a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a9300 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f6d94005b80 tx=0x7f6d94010440 comp rx=0 tx=0).stop 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 -- 172.21.15.16:0/1768050288 shutdown_connections 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417a9300 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b14179d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/1768050288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 -- 172.21.15.16:0/1768050288 >> 172.21.15.16:0/1768050288 conn(0x55b1416fb290 msgr2=0x55b1417a3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 -- 172.21.15.16:0/1768050288 shutdown_connections 2024-05-08T23:29:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 -- 172.21.15.16:0/1768050288 wait complete. 2024-05-08T23:29:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 -- start start 2024-05-08T23:29:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 Processor -- start 2024-05-08T23:29:03.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.362+0000 7f6daf22fb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1417ac3d0 con 0x55b1417a37a0 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1417ac540 con 0x55b14179d910 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6daf22fb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1417ac700 con 0x55b1417a2e80 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6daf22fb80 10 client.0 _ll_register_callbacks cb 0x55b141692bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35634/0 (socket says 172.21.15.16:35634) 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 -- 172.21.15.16:0/2169067245 learned_addr learned my addr 172.21.15.16:0/2169067245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6d9bfff700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 -- 172.21.15.16:0/2169067245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 msgr2=0x55b1417ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 -- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 msgr2=0x55b1417ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 -- 172.21.15.16:0/2169067245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b14151abe0 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6da1612700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417abdf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000f80 tx=0x7f6d940159b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5665acfe80eb9c49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2169067245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94004a30 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2169067245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6d94004b90 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.363+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d8800fed0 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2169067245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d94004d90 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 10 client.17628 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b1416b35e0 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2169067245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d8800fed0 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2169067245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d94005320 con 0x55b1417a37a0 2024-05-08T23:29:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 10 client.17628 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 20 client.17628 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b141813f80 0x55b141816370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 4 client.17628 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b1416e5f40 con 0x55b141813f80 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.364+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b141813f80 0x55b141816370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.365+0000 7f6da0e11700 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b141813f80 0x55b141816370 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe74b480c645855 server_cookie=bc395254bdb619d5 in_seq=0 out_seq=0 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.365+0000 7f6d7bfff700 10 client.17628 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6d7bfff700 1 -- 172.21.15.16:0/2169067245 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b1416e5f40 con 0x55b141813f80 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6d7bfff700 10 client.17628 handle_command_reply: tid=0 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 client.17628 shutdown 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 msgr2=0x55b1417abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417abdf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000f80 tx=0x7f6d940159b0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 shutdown_connections 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1417a37a0 0x55b1417abdf0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b141813f80 0x55b141816370 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b14179d910 0x55b1417ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 --2- 172.21.15.16:0/2169067245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b1417a2e80 0x55b1417ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6d7bfff700 0 client.17628 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 >> 172.21.15.16:0/2169067245 conn(0x55b1416fb290 msgr2=0x55b1417a44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 shutdown_connections 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 1 -- 172.21.15.16:0/2169067245 wait complete. 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.366+0000 7f6daf22fb80 20 client.17628 trim_cache size 0 max 16384 2024-05-08T23:29:03.371 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:29:03.372 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:29:03.372 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "9817fe9b-22ae-4808-81d6-c163cdb07b34", 2024-05-08T23:29:03.372 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:29:03.372 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d8c0a1310 msgr2=0x7f6d8c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d8c0a1310 0x7f6d8c0a37d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005950 tx=0x7f6d880058e0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 msgr2=0x7f6d9c197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c197380 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6d900036c0 tx=0x7f6d90005e80 comp rx=0 tx=0).stop 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 shutdown_connections 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d8c0a1310 0x7f6d8c0a37d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d9c19bc60 0x7f6d9c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c101160 0x7f6d9c197380 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 --2- 172.21.15.16:0/2192006365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d9c106280 0x7f6d9c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 >> 172.21.15.16:0/2192006365 conn(0x7f6d9c0fc9e0 msgr2=0x7f6d9c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 shutdown_connections 2024-05-08T23:29:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.367+0000 7f6da3075700 1 -- 172.21.15.16:0/2192006365 wait complete. 2024-05-08T23:29:03.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:29:03.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:03 smithi016 ceph-mon[42541]: pgmap v293: 129 pgs: 129 active+clean; 15 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:03.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:03 smithi189 ceph-mon[46919]: pgmap v293: 129 pgs: 129 active+clean; 15 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:03.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:03 smithi155 ceph-mon[47105]: pgmap v293: 129 pgs: 129 active+clean; 15 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 -- 172.21.15.16:0/3732142951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8105160 msgr2=0x7f46e810cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 --2- 172.21.15.16:0/3732142951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8105160 0x7f46e810cce0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f46e000b0a0 tx=0x7f46e000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 -- 172.21.15.16:0/3732142951 shutdown_connections 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 --2- 172.21.15.16:0/3732142951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8105160 0x7f46e810cce0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 --2- 172.21.15.16:0/3732142951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8100060 0x7f46e8100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 --2- 172.21.15.16:0/3732142951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8100a10 0x7f46e8104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 -- 172.21.15.16:0/3732142951 >> 172.21.15.16:0/3732142951 conn(0x7f46e8076f40 msgr2=0x7f46e8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 -- 172.21.15.16:0/3732142951 shutdown_connections 2024-05-08T23:29:03.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.519+0000 7f46ed334700 1 -- 172.21.15.16:0/3732142951 wait complete. 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.520+0000 7f46ed334700 1 Processor -- start 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.520+0000 7f46ed334700 1 -- start start 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8100060 0x7f46e8196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8105160 0x7f46e81968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 0x7f46e819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819b6c0 con 0x7f46e819ac40 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819b830 con 0x7f46e8105160 2024-05-08T23:29:03.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819b9f0 con 0x7f46e8100060 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 0x7f46e819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 0x7f46e819b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35642/0 (socket says 172.21.15.16:35642) 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 -- 172.21.15.16:0/1886761853 learned_addr learned my addr 172.21.15.16:0/1886761853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8100060 0x7f46e8196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e67fc700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8105160 0x7f46e81968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 -- 172.21.15.16:0/1886761853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8100060 msgr2=0x7f46e8196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8100060 0x7f46e8196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 -- 172.21.15.16:0/1886761853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8105160 msgr2=0x7f46e81968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8105160 0x7f46e81968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 -- 172.21.15.16:0/1886761853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009d00 con 0x7f46e819ac40 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e67fc700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8105160 0x7f46e81968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46e77fe700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 0x7f46e819b0c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f46e0004de0 tx=0x7f46e0004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46dffff700 1 -- 172.21.15.16:0/1886761853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000e040 con 0x7f46e819ac40 2024-05-08T23:29:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46dffff700 1 -- 172.21.15.16:0/1886761853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46e001b450 con 0x7f46e819ac40 2024-05-08T23:29:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46dffff700 1 -- 172.21.15.16:0/1886761853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00052a0 con 0x7f46e819ac40 2024-05-08T23:29:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.521+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e81a2ef0 con 0x7f46e819ac40 2024-05-08T23:29:03.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.522+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e81a3320 con 0x7f46e819ac40 2024-05-08T23:29:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.524+0000 7f46dffff700 1 -- 172.21.15.16:0/1886761853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f46e001b5c0 con 0x7f46e819ac40 2024-05-08T23:29:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.525+0000 7f46dffff700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f46d40a1290 0x7f46d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.525+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f46d40a1290 0x7f46d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.525+0000 7f46dffff700 1 -- 172.21.15.16:0/1886761853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f46e00ce080 con 0x7f46e819ac40 2024-05-08T23:29:03.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.525+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f46d40a1290 0x7f46d40a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009c60 tx=0x7f46d8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 Processor -- start 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 -- start start 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b5b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 0x557335b66320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b66860 0x557335b68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335ab1150 con 0x557335b5b980 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335aaeb10 con 0x557335b5b060 2024-05-08T23:29:03.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335aaa400 con 0x557335b66860 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46e67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 0x557335b66320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46e67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 0x557335b66320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35654/0 (socket says 172.21.15.16:35654) 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46e67fc700 1 -- 172.21.15.16:0/4218919821 learned_addr learned my addr 172.21.15.16:0/4218919821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.526+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b5b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e77fe700 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b66860 0x557335b68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e67fc700 1 -- 172.21.15.16:0/4218919821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b66860 msgr2=0x557335b68c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e67fc700 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b66860 0x557335b68c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e67fc700 1 -- 172.21.15.16:0/4218919821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 msgr2=0x557335b5b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e67fc700 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b5b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e67fc700 1 -- 172.21.15.16:0/4218919821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55733592b3b0 con 0x557335b5b980 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b5b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46e67fc700 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 0x557335b66320 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d10 tx=0x7f46d000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d76ffb7694cfd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46dd7fa700 1 -- 172.21.15.16:0/4218919821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d000fe50 con 0x557335b5b980 2024-05-08T23:29:03.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46dd7fa700 1 -- 172.21.15.16:0/4218919821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46d0009e60 con 0x557335b5b980 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.527+0000 7f46dd7fa700 1 -- 172.21.15.16:0/4218919821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d0003e00 con 0x557335b5b980 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 -- 172.21.15.16:0/4218919821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 msgr2=0x557335b66320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 0x557335b66320 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009d10 tx=0x7f46d000f640 comp rx=0 tx=0).stop 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 -- 172.21.15.16:0/4218919821 shutdown_connections 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b66860 0x557335b68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b5b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/4218919821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b980 0x557335b66320 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 -- 172.21.15.16:0/4218919821 >> 172.21.15.16:0/4218919821 conn(0x557335ab8210 msgr2=0x557335b577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 -- 172.21.15.16:0/4218919821 shutdown_connections 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 -- 172.21.15.16:0/4218919821 wait complete. 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 -- start start 2024-05-08T23:29:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.528+0000 7f46f94eeb80 1 Processor -- start 2024-05-08T23:29:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 0x557335b635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b66860 0x557335b64040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335a46560 con 0x557335b5b060 2024-05-08T23:29:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335a46200 con 0x557335b66860 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55733592d6a0 con 0x557335b5b980 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46f94eeb80 10 client.0 _ll_register_callbacks cb 0x557335a4fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 0x557335b635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 0x557335b635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35670/0 (socket says 172.21.15.16:35670) 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 -- 172.21.15.16:0/1945845480 learned_addr learned my addr 172.21.15.16:0/1945845480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e67fc700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e77fe700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b66860 0x557335b64040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 -- 172.21.15.16:0/1945845480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 msgr2=0x557335b63b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 -- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b66860 msgr2=0x557335b64040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b66860 0x557335b64040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 -- 172.21.15.16:0/1945845480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d0009940 con 0x557335b5b060 2024-05-08T23:29:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e77fe700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b66860 0x557335b64040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 0x557335b635c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f46d8003930 tx=0x7f46d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6893c69fd987d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.529+0000 7f46ce7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46ce7fc700 1 -- 172.21.15.16:0/1945845480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d801e8b0 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46ce7fc700 1 -- 172.21.15.16:0/1945845480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f46d801ea10 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557335a03270 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46ce7fc700 1 -- 172.21.15.16:0/1945845480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46d801ebd0 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 10 client.17640 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5573358d7b60 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46ce7fc700 1 -- 172.21.15.16:0/1945845480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557335a03270 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46ce7fc700 1 -- 172.21.15.16:0/1945845480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46d801c430 con 0x557335b5b060 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 10 client.17640 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 20 client.17640 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335b76510 0x557335b78930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 4 client.17640 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557335b29810 con 0x557335b76510 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.530+0000 7f46e67fc700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335b76510 0x557335b78930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.531+0000 7f46ce7fc700 10 client.17640 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.531+0000 7f46e67fc700 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335b76510 0x557335b78930 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3041bbb8961061a1 server_cookie=dbc2af4da5ff0875 in_seq=0 out_seq=0 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.531+0000 7f46ce7fc700 1 -- 172.21.15.16:0/1945845480 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557335b29810 con 0x557335b76510 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.531+0000 7f46ce7fc700 10 client.17640 handle_command_reply: tid=0 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.531+0000 7f46f94eeb80 1 client.17640 shutdown 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 msgr2=0x557335b635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 0x557335b635c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f46d8003930 tx=0x7f46d80047c0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 shutdown_connections 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335b76510 0x557335b78930 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b66860 0x557335b64040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b5b060 0x557335b635c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46ce7fc700 0 client.17640 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/1945845480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 >> 172.21.15.16:0/1945845480 conn(0x557335ab8210 msgr2=0x557335b5d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 shutdown_connections 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 1 -- 172.21.15.16:0/1945845480 wait complete. 2024-05-08T23:29:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.532+0000 7f46f94eeb80 20 client.17640 trim_cache size 0 max 16384 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 Processor -- start 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 -- start start 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335b6d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335b4f880 con 0x557335b66860 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335b5cb30 con 0x557335b5b060 2024-05-08T23:29:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335b7db50 con 0x557335b5b980 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.550+0000 7f46e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335b6d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335b6d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35678/0 (socket says 172.21.15.16:35678) 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 -- 172.21.15.16:0/3415214365 learned_addr learned my addr 172.21.15.16:0/3415214365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e67fc700 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 -- 172.21.15.16:0/3415214365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 msgr2=0x557335b63e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 -- 172.21.15.16:0/3415214365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 msgr2=0x557335b68c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b68c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 -- 172.21.15.16:0/3415214365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557335a70560 con 0x557335b66860 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46e77fe700 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335b6d450 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f46e00cb4e0 tx=0x7f46e009acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d4ff3f8431ab10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46ddffb700 1 -- 172.21.15.16:0/3415214365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0019e30 con 0x557335b66860 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46ddffb700 1 -- 172.21.15.16:0/3415214365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46e009be90 con 0x557335b66860 2024-05-08T23:29:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.551+0000 7f46ddffb700 1 -- 172.21.15.16:0/3415214365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00a3300 con 0x557335b66860 2024-05-08T23:29:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 -- 172.21.15.16:0/3415214365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 msgr2=0x557335b6d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335b6d450 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f46e00cb4e0 tx=0x7f46e009acd0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 -- 172.21.15.16:0/3415214365 shutdown_connections 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335b6d450 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/3415214365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b63e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 -- 172.21.15.16:0/3415214365 >> 172.21.15.16:0/3415214365 conn(0x557335ab8210 msgr2=0x557335b5e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 -- 172.21.15.16:0/3415214365 shutdown_connections 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 -- 172.21.15.16:0/3415214365 wait complete. 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 -- start start 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.552+0000 7f46f94eeb80 1 Processor -- start 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b6b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335bcba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335b6c360 con 0x557335b66860 2024-05-08T23:29:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335bcbf50 con 0x557335b5b060 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557335bcc110 con 0x557335b5b980 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46f94eeb80 10 client.0 _ll_register_callbacks cb 0x557335a4fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335bcba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335bcba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35684/0 (socket says 172.21.15.16:35684) 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 -- 172.21.15.16:0/2929414320 learned_addr learned my addr 172.21.15.16:0/2929414320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e67fc700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b6b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 -- 172.21.15.16:0/2929414320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 msgr2=0x557335b6bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b6bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 -- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 msgr2=0x557335b6b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b6b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e77fe700 1 -- 172.21.15.16:0/2929414320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e81a3320 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.553+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b6b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46e77fe700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335bcba10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f46e009ae10 tx=0x7f46e00cb3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ef0a8b3d7fcf9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46ce7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46ce7fc700 1 -- 172.21.15.16:0/2929414320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e00b9070 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46ce7fc700 1 -- 172.21.15.16:0/2929414320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f46e0019e30 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46d0009e60 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46ce7fc700 1 -- 172.21.15.16:0/2929414320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e001bab0 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 10 client.17646 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55733592fcd0 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46ce7fc700 1 -- 172.21.15.16:0/2929414320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46d0009e60 con 0x557335b66860 2024-05-08T23:29:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46ce7fc700 1 -- 172.21.15.16:0/2929414320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46e009a370 con 0x557335b66860 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 10 client.17646 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 20 client.17646 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335bcd380 0x557335bcf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 4 client.17646 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.554+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557335aa2ec0 con 0x557335bcd380 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335bcd380 0x557335bcf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46ce7fc700 10 client.17646 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46e6ffd700 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335bcd380 0x557335bcf770 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7cfd3c87baecad server_cookie=b034bdedec7272bb in_seq=0 out_seq=0 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46ce7fc700 1 -- 172.21.15.16:0/2929414320 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557335aa2ec0 con 0x557335bcd380 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46ce7fc700 10 client.17646 handle_command_reply: tid=0 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 client.17646 shutdown 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 msgr2=0x557335bcba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335bcba10 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f46e009ae10 tx=0x7f46e00cb3a0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 shutdown_connections 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557335b66860 0x557335bcba10 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x557335bcd380 0x557335bcf770 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557335b5b060 0x557335b6b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 --2- 172.21.15.16:0/2929414320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557335b5b980 0x557335b6bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 >> 172.21.15.16:0/2929414320 conn(0x557335ab8210 msgr2=0x557335b53d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46ce7fc700 0 client.17646 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.555+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 shutdown_connections 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46f94eeb80 1 -- 172.21.15.16:0/2929414320 wait complete. 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46f94eeb80 20 client.17646 trim_cache size 0 max 16384 2024-05-08T23:29:03.562 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: "9817fe9b-22ae-4808-81d6-c163cdb07b34": { 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "9817fe9b-22ae-4808-81d6-c163cdb07b34", 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:29:03.563 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f46d40a1290 msgr2=0x7f46d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f46d40a1290 0x7f46d40a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009c60 tx=0x7f46d8009260 comp rx=0 tx=0).stop 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 msgr2=0x7f46e819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 0x7f46e819b0c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f46e0004de0 tx=0x7f46e0004e10 comp rx=0 tx=0).stop 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 shutdown_connections 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f46d40a1290 0x7f46d40a3750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e819ac40 0x7f46e819b0c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8100060 0x7f46e8196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 --2- 172.21.15.16:0/1886761853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46e8105160 0x7f46e81968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 >> 172.21.15.16:0/1886761853 conn(0x7f46e8076f40 msgr2=0x7f46e80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:03.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 shutdown_connections 2024-05-08T23:29:03.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:03.556+0000 7f46ed334700 1 -- 172.21.15.16:0/1886761853 wait complete. 2024-05-08T23:29:03.569 INFO:tasks.cephfs.filesystem:scrub status for tag:9817fe9b-22ae-4808-81d6-c163cdb07b34 - {'path': '/', 'tag': '9817fe9b-22ae-4808-81d6-c163cdb07b34', 'options': 'recursive,force'} 2024-05-08T23:29:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:04 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:29:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:04 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:04 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:29:04.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:04 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:04 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:29:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:04 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:04 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:29:04.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:04 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:04.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:04 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:29:04.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:04 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:04.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:04 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:29:04.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:04 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:05.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:05 smithi189 ceph-mon[46919]: pgmap v294: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 110 KiB/s wr, 32 op/s 2024-05-08T23:29:05.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:05 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:29:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:05 smithi155 ceph-mon[47105]: pgmap v294: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 110 KiB/s wr, 32 op/s 2024-05-08T23:29:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:05 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:29:05.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:05 smithi016 ceph-mon[42541]: pgmap v294: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 110 KiB/s wr, 32 op/s 2024-05-08T23:29:05.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:05 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:29:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:06 smithi155 ceph-mon[47105]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:29:06.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:06 smithi016 ceph-mon[42541]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:29:06.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:06 smithi189 ceph-mon[46919]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T23:29:07.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:07 smithi155 ceph-mon[47105]: pgmap v296: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 33 op/s 2024-05-08T23:29:07.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:07 smithi016 ceph-mon[42541]: pgmap v296: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 33 op/s 2024-05-08T23:29:07.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:07 smithi189 ceph-mon[46919]: pgmap v296: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 33 op/s 2024-05-08T23:29:09.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:09 smithi155 ceph-mon[47105]: pgmap v297: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 33 op/s 2024-05-08T23:29:09.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:09 smithi016 ceph-mon[42541]: pgmap v297: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 33 op/s 2024-05-08T23:29:09.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:09 smithi189 ceph-mon[46919]: pgmap v297: 129 pgs: 129 active+clean; 15 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 33 op/s 2024-05-08T23:29:11.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:11 smithi155 ceph-mon[47105]: pgmap v298: 129 pgs: 129 active+clean; 15 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 33 op/s 2024-05-08T23:29:11.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:11 smithi016 ceph-mon[42541]: pgmap v298: 129 pgs: 129 active+clean; 15 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 33 op/s 2024-05-08T23:29:11.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:11 smithi189 ceph-mon[46919]: pgmap v298: 129 pgs: 129 active+clean; 15 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 33 op/s 2024-05-08T23:29:13.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:13 smithi155 ceph-mon[47105]: pgmap v299: 129 pgs: 129 active+clean; 15 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 33 op/s 2024-05-08T23:29:13.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:13 smithi016 ceph-mon[42541]: pgmap v299: 129 pgs: 129 active+clean; 15 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 33 op/s 2024-05-08T23:29:13.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:13 smithi189 ceph-mon[46919]: pgmap v299: 129 pgs: 129 active+clean; 15 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 110 KiB/s wr, 33 op/s 2024-05-08T23:29:14.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:14 smithi155 ceph-mon[47105]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:29:15.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:14 smithi016 ceph-mon[42541]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:29:15.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:14 smithi189 ceph-mon[46919]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T23:29:15.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:15 smithi155 ceph-mon[47105]: pgmap v301: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 13 op/s 2024-05-08T23:29:16.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:15 smithi016 ceph-mon[42541]: pgmap v301: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 13 op/s 2024-05-08T23:29:16.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:15 smithi189 ceph-mon[46919]: pgmap v301: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 13 op/s 2024-05-08T23:29:17.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:17 smithi155 ceph-mon[47105]: pgmap v302: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-05-08T23:29:18.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:17 smithi016 ceph-mon[42541]: pgmap v302: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-05-08T23:29:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:17 smithi189 ceph-mon[46919]: pgmap v302: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-05-08T23:29:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:19 smithi016 ceph-mon[42541]: pgmap v303: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-05-08T23:29:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:19 smithi189 ceph-mon[46919]: pgmap v303: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-05-08T23:29:20.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:19 smithi155 ceph-mon[47105]: pgmap v303: 129 pgs: 129 active+clean; 16 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 12 op/s 2024-05-08T23:29:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:21 smithi016 ceph-mon[42541]: pgmap v304: 129 pgs: 129 active+clean; 16 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-05-08T23:29:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:21 smithi189 ceph-mon[46919]: pgmap v304: 129 pgs: 129 active+clean; 16 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-05-08T23:29:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:21 smithi155 ceph-mon[47105]: pgmap v304: 129 pgs: 129 active+clean; 16 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-05-08T23:29:24.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:23 smithi016 ceph-mon[42541]: pgmap v305: 129 pgs: 129 active+clean; 16 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-05-08T23:29:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:23 smithi189 ceph-mon[46919]: pgmap v305: 129 pgs: 129 active+clean; 16 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-05-08T23:29:24.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:23 smithi155 ceph-mon[47105]: pgmap v305: 129 pgs: 129 active+clean; 16 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 12 op/s 2024-05-08T23:29:26.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:25 smithi016 ceph-mon[42541]: pgmap v306: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 80 KiB/s wr, 17 op/s 2024-05-08T23:29:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:25 smithi189 ceph-mon[46919]: pgmap v306: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 80 KiB/s wr, 17 op/s 2024-05-08T23:29:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:25 smithi155 ceph-mon[47105]: pgmap v306: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 80 KiB/s wr, 17 op/s 2024-05-08T23:29:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:27 smithi016 ceph-mon[42541]: pgmap v307: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-05-08T23:29:28.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:27 smithi189 ceph-mon[46919]: pgmap v307: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-05-08T23:29:28.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:27 smithi155 ceph-mon[47105]: pgmap v307: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-05-08T23:29:30.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:29 smithi016 ceph-mon[42541]: pgmap v308: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-05-08T23:29:30.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:29 smithi189 ceph-mon[46919]: pgmap v308: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-05-08T23:29:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:29 smithi155 ceph-mon[47105]: pgmap v308: 129 pgs: 129 active+clean; 16 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-05-08T23:29:32.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:31 smithi016 ceph-mon[42541]: pgmap v309: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 70 KiB/s wr, 15 op/s 2024-05-08T23:29:32.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:31 smithi189 ceph-mon[46919]: pgmap v309: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 70 KiB/s wr, 15 op/s 2024-05-08T23:29:32.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:31 smithi155 ceph-mon[47105]: pgmap v309: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 70 KiB/s wr, 15 op/s 2024-05-08T23:29:33.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 -- 172.21.15.16:0/1649561579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c100a10 msgr2=0x7fe81c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 --2- 172.21.15.16:0/1649561579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c100a10 0x7fe81c104a50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe810009990 tx=0x7fe810009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 -- 172.21.15.16:0/1649561579 shutdown_connections 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 --2- 172.21.15.16:0/1649561579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe81c105180 0x7fe81c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 --2- 172.21.15.16:0/1649561579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 0x7fe81c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 --2- 172.21.15.16:0/1649561579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c100a10 0x7fe81c104a50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 -- 172.21.15.16:0/1649561579 >> 172.21.15.16:0/1649561579 conn(0x7fe81c076fa0 msgr2=0x7fe81c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 -- 172.21.15.16:0/1649561579 shutdown_connections 2024-05-08T23:29:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 -- 172.21.15.16:0/1649561579 wait complete. 2024-05-08T23:29:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.717+0000 7fe821692700 1 Processor -- start 2024-05-08T23:29:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 -- start start 2024-05-08T23:29:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 0x7fe81c10d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 0x7fe81c10db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe81c10e0c0 0x7fe81c1a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1a5b10 con 0x7fe81c105180 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1a5c80 con 0x7fe81c100060 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe821692700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1a5e40 con 0x7fe81c10e0c0 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 0x7fe81c10db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 0x7fe81c10db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59938/0 (socket says 172.21.15.16:59938) 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 -- 172.21.15.16:0/244870677 learned_addr learned my addr 172.21.15.16:0/244870677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81affd700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 0x7fe81c10d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe81c10e0c0 0x7fe81c1a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 -- 172.21.15.16:0/244870677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe81c10e0c0 msgr2=0x7fe81c1a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe81c10e0c0 0x7fe81c1a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 -- 172.21.15.16:0/244870677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 msgr2=0x7fe81c10d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 0x7fe81c10d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81a7fc700 1 -- 172.21.15.16:0/244870677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100095f0 con 0x7fe81c105180 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.718+0000 7fe81affd700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 0x7fe81c10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.719+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 0x7fe81c10db80 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe810005910 tx=0x7fe810005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.719+0000 7fe803fff700 1 -- 172.21.15.16:0/244870677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81001d070 con 0x7fe81c105180 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.719+0000 7fe803fff700 1 -- 172.21.15.16:0/244870677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe810022470 con 0x7fe81c105180 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.719+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c1a60c0 con 0x7fe81c105180 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.719+0000 7fe803fff700 1 -- 172.21.15.16:0/244870677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810017650 con 0x7fe81c105180 2024-05-08T23:29:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.719+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1a6490 con 0x7fe81c105180 2024-05-08T23:29:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.722+0000 7fe803fff700 1 -- 172.21.15.16:0/244870677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe8100225e0 con 0x7fe81c105180 2024-05-08T23:29:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe803fff700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe8040a1290 0x7fe8040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe803fff700 1 -- 172.21.15.16:0/244870677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fe8100ce1a0 con 0x7fe81c105180 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe81affd700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe8040a1290 0x7fe8040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 Processor -- start 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 -- start start 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e1211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e11b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e11bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e072150 con 0x561a9e121720 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e06fb10 con 0x561a9e11b890 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.723+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e06b400 con 0x561a9e120e00 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81affd700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe8040a1290 0x7fe8040a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009cb0 tx=0x7fe80c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e11b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81affd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e1211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e11b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59954/0 (socket says 172.21.15.16:59954) 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2899042365 learned_addr learned my addr 172.21.15.16:0/2899042365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e11bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2899042365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 msgr2=0x561a9e1211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e1211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2899042365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 msgr2=0x561a9e11bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e11bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2899042365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9deec3b0 con 0x561a9e121720 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e11bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e11b350 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe810097a10 tx=0x7fe81009ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2721483457922677 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe8017fa700 1 -- 172.21.15.16:0/2899042365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81009b6a0 con 0x561a9e121720 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe8017fa700 1 -- 172.21.15.16:0/2899042365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe81009afa0 con 0x561a9e121720 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.724+0000 7fe8017fa700 1 -- 172.21.15.16:0/2899042365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81009b440 con 0x561a9e121720 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2899042365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 msgr2=0x561a9e11b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e11b350 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe810097a10 tx=0x7fe81009ae90 comp rx=0 tx=0).stop 2024-05-08T23:29:33.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2899042365 shutdown_connections 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e11bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e1211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2899042365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e11b350 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2899042365 >> 172.21.15.16:0/2899042365 conn(0x561a9e079210 msgr2=0x561a9e115df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2899042365 shutdown_connections 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2899042365 wait complete. 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.725+0000 7fe82d84cb80 1 -- start start 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 Processor -- start 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e11b890 0x561a9e124a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e120e00 0x561a9e124f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e1254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e007560 con 0x561a9e121720 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e007200 con 0x561a9e120e00 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9deee6a0 con 0x561a9e11b890 2024-05-08T23:29:33.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe82d84cb80 10 client.0 _ll_register_callbacks cb 0x561a9e010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e1254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e1254d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59956/0 (socket says 172.21.15.16:59956) 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 -- 172.21.15.16:0/4190207835 learned_addr learned my addr 172.21.15.16:0/4190207835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81affd700 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e11b890 0x561a9e124a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 -- 172.21.15.16:0/4190207835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e11b890 msgr2=0x561a9e124a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e11b890 0x561a9e124a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 -- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e120e00 msgr2=0x561a9e124f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e120e00 0x561a9e124f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.726+0000 7fe81b7fe700 1 -- 172.21.15.16:0/4190207835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1a6490 con 0x561a9e121720 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e1254d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe814007920 tx=0x7fe814007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9abc634723ee6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe7feff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe7feff5700 1 -- 172.21.15.16:0/4190207835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814010860 con 0x561a9e121720 2024-05-08T23:29:33.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe7feff5700 1 -- 172.21.15.16:0/4190207835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fe8140109c0 con 0x561a9e121720 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a9dfc4270 con 0x561a9e121720 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe7feff5700 1 -- 172.21.15.16:0/4190207835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe814010b80 con 0x561a9e121720 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 10 client.17658 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561a9de98b60 con 0x561a9e121720 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe7feff5700 1 -- 172.21.15.16:0/4190207835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561a9dfc4270 con 0x561a9e121720 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe7feff5700 1 -- 172.21.15.16:0/4190207835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe814015ce0 con 0x561a9e121720 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 10 client.17658 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 20 client.17658 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e134f30 0x561a9e137350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 4 client.17658 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.727+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a9e0ea810 con 0x561a9e134f30 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.728+0000 7fe81affd700 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e134f30 0x561a9e137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.728+0000 7fe81affd700 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e134f30 0x561a9e137350 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54516f480bb2b6d server_cookie=597601eebf1b3e73 in_seq=0 out_seq=0 2024-05-08T23:29:33.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.728+0000 7fe7feff5700 10 client.17658 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe7feff5700 1 -- 172.21.15.16:0/4190207835 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a9e0ea810 con 0x561a9e134f30 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe7feff5700 10 client.17658 handle_command_reply: tid=0 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 client.17658 shutdown 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 msgr2=0x561a9e1254d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e1254d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe814007920 tx=0x7fe814007ce0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 shutdown_connections 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e121720 0x561a9e1254d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e11b890 0x561a9e124a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e134f30 0x561a9e137350 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/4190207835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e120e00 0x561a9e124f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe7feff5700 0 client.17658 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 >> 172.21.15.16:0/4190207835 conn(0x561a9e079210 msgr2=0x561a9e1155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 shutdown_connections 2024-05-08T23:29:33.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 1 -- 172.21.15.16:0/4190207835 wait complete. 2024-05-08T23:29:33.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.729+0000 7fe82d84cb80 20 client.17658 trim_cache size 0 max 16384 2024-05-08T23:29:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 Processor -- start 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 -- start start 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e115d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 0x561a9e12bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e121720 0x561a9e12c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e131a50 con 0x561a9e120e00 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e112dc0 con 0x561a9e11b890 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e13c550 con 0x561a9e121720 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 0x561a9e12bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 0x561a9e12bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59962/0 (socket says 172.21.15.16:59962) 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2066978052 learned_addr learned my addr 172.21.15.16:0/2066978052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e121720 0x561a9e12c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81affd700 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e115d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2066978052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e121720 msgr2=0x561a9e12c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e121720 0x561a9e12c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2066978052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 msgr2=0x561a9e115d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e115d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.748+0000 7fe81a7fc700 1 -- 172.21.15.16:0/2066978052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9e031560 con 0x561a9e120e00 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 0x561a9e12bed0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe81009aca0 tx=0x7fe810009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ceb9f88d17b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe801ffb700 1 -- 172.21.15.16:0/2066978052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81009b6a0 con 0x561a9e120e00 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe801ffb700 1 -- 172.21.15.16:0/2066978052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8100a3390 con 0x561a9e120e00 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe801ffb700 1 -- 172.21.15.16:0/2066978052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81009b200 con 0x561a9e120e00 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2066978052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 msgr2=0x561a9e12bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 0x561a9e12bed0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe81009aca0 tx=0x7fe810009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.749+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2066978052 shutdown_connections 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e121720 0x561a9e12c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e11b890 0x561a9e115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/2066978052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e120e00 0x561a9e12bed0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2066978052 >> 172.21.15.16:0/2066978052 conn(0x561a9e079210 msgr2=0x561a9e123340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2066978052 shutdown_connections 2024-05-08T23:29:33.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 -- 172.21.15.16:0/2066978052 wait complete. 2024-05-08T23:29:33.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 -- start start 2024-05-08T23:29:33.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 Processor -- start 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 0x561a9e12c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e11de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.750+0000 7fe82d84cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e121720 0x561a9e11c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e11e780 con 0x561a9e11b890 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e11ce70 con 0x561a9e121720 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe82d84cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9e11d030 con 0x561a9e120e00 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe82d84cb80 10 client.0 _ll_register_callbacks cb 0x561a9e010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 0x561a9e12c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 0x561a9e12c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59968/0 (socket says 172.21.15.16:59968) 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 -- 172.21.15.16:0/3093045399 learned_addr learned my addr 172.21.15.16:0/3093045399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e11de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81b7fe700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e121720 0x561a9e11c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 -- 172.21.15.16:0/3093045399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 msgr2=0x561a9e11de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e11de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 -- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e121720 msgr2=0x561a9e11c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e121720 0x561a9e11c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 -- 172.21.15.16:0/3093045399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a9deec3b0 con 0x561a9e11b890 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e11de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe81affd700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 0x561a9e12c410 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe80c011e70 tx=0x7fe80c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7615f2b35d8bc3c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.751+0000 7fe7feff5700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe7feff5700 1 -- 172.21.15.16:0/3093045399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c029d00 con 0x561a9e11b890 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe7feff5700 1 -- 172.21.15.16:0/3093045399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fe80c018040 con 0x561a9e11b890 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe81009afa0 con 0x561a9e11b890 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe7feff5700 1 -- 172.21.15.16:0/3093045399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe80c029e60 con 0x561a9e11b890 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 10 client.17664 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561a9def0cd0 con 0x561a9e11b890 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe7feff5700 1 -- 172.21.15.16:0/3093045399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe81009afa0 con 0x561a9e11b890 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe7feff5700 1 -- 172.21.15.16:0/3093045399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe80c003ae0 con 0x561a9e11b890 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 10 client.17664 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 20 client.17664 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e18c200 0x561a9e18e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 4 client.17664 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a9e063ec0 con 0x561a9e18c200 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.752+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e18c200 0x561a9e18e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe7feff5700 10 client.17664 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe81a7fc700 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e18c200 0x561a9e18e5f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd6757e1e2395d server_cookie=e535723c1aeb7697 in_seq=0 out_seq=0 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe7feff5700 1 -- 172.21.15.16:0/3093045399 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561a9e063ec0 con 0x561a9e18c200 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe7feff5700 10 client.17664 handle_command_reply: tid=0 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 client.17664 shutdown 2024-05-08T23:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 msgr2=0x561a9e12c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 0x561a9e12c410 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe80c011e70 tx=0x7fe80c0047c0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 shutdown_connections 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561a9e18c200 0x561a9e18e5f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561a9e121720 0x561a9e11c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe7feff5700 0 client.17664 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a9e11b890 0x561a9e12c410 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 --2- 172.21.15.16:0/3093045399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a9e120e00 0x561a9e11de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 >> 172.21.15.16:0/3093045399 conn(0x561a9e079210 msgr2=0x561a9e11fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 shutdown_connections 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 1 -- 172.21.15.16:0/3093045399 wait complete. 2024-05-08T23:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.753+0000 7fe82d84cb80 20 client.17664 trim_cache size 0 max 16384 2024-05-08T23:29:33.761 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:29:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:29:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:29:33.761 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe8040a1290 msgr2=0x7fe8040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe8040a1290 0x7fe8040a3750 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009cb0 tx=0x7fe80c009260 comp rx=0 tx=0).stop 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 msgr2=0x7fe81c10db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 0x7fe81c10db80 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe810005910 tx=0x7fe810005e80 comp rx=0 tx=0).stop 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 shutdown_connections 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fe8040a1290 0x7fe8040a3750 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe81c10e0c0 0x7fe81c1a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe81c100060 0x7fe81c10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 --2- 172.21.15.16:0/244870677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81c105180 0x7fe81c10db80 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 >> 172.21.15.16:0/244870677 conn(0x7fe81c076fa0 msgr2=0x7fe81c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 shutdown_connections 2024-05-08T23:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.754+0000 7fe821692700 1 -- 172.21.15.16:0/244870677 wait complete. 2024-05-08T23:29:33.768 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:29:33.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:29:33.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:33 smithi016 ceph-mon[42541]: pgmap v310: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 -- 172.21.15.16:0/291313322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0101a10 msgr2=0x7f7ec0105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/291313322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0101a10 0x7f7ec0105a50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009990 tx=0x7f7ea8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 -- 172.21.15.16:0/291313322 shutdown_connections 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/291313322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec0106180 0x7f7ec010dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/291313322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec0101060 0x7f7ec0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/291313322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0101a10 0x7f7ec0105a50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 -- 172.21.15.16:0/291313322 >> 172.21.15.16:0/291313322 conn(0x7f7ec0078360 msgr2=0x7f7ec0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 -- 172.21.15.16:0/291313322 shutdown_connections 2024-05-08T23:29:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.906+0000 7f7ec5cee700 1 -- 172.21.15.16:0/291313322 wait complete. 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 Processor -- start 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 -- start start 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec0101060 0x7f7ec0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 0x7f7ec0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec019bb30 0x7f7ec019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec019c5b0 con 0x7f7ec0106180 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec019c720 con 0x7f7ec019bb30 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ec5cee700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec019c8e0 con 0x7f7ec0101060 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ebeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 0x7f7ec0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ebeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 0x7f7ec0197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59982/0 (socket says 172.21.15.16:59982) 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.907+0000 7f7ebeffd700 1 -- 172.21.15.16:0/39309308 learned_addr learned my addr 172.21.15.16:0/39309308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebf7fe700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec0101060 0x7f7ec0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebffff700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec019bb30 0x7f7ec019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebeffd700 1 -- 172.21.15.16:0/39309308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec0101060 msgr2=0x7f7ec0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebeffd700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec0101060 0x7f7ec0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebeffd700 1 -- 172.21.15.16:0/39309308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec019bb30 msgr2=0x7f7ec019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebeffd700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec019bb30 0x7f7ec019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebeffd700 1 -- 172.21.15.16:0/39309308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80095f0 con 0x7f7ec0106180 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebffff700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec019bb30 0x7f7ec019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebeffd700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 0x7f7ec0197790 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005910 tx=0x7f7ea8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea801d070 con 0x7f7ec0106180 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ea800bea0 con 0x7f7ec0106180 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80173f0 con 0x7f7ec0106180 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec01a3de0 con 0x7f7ec0106180 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.908+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec01a41b0 con 0x7f7ec0106180 2024-05-08T23:29:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.909+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec0109450 con 0x7f7ec0106180 2024-05-08T23:29:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.910+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7ea8022650 con 0x7f7ec0106180 2024-05-08T23:29:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.911+0000 7f7ebcff9700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7eac0a1310 0x7f7eac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.911+0000 7f7ebf7fe700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7eac0a1310 0x7f7eac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.911+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f7ea80cea40 con 0x7f7ec0106180 2024-05-08T23:29:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.912+0000 7f7ebf7fe700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7eac0a1310 0x7f7eac0a37d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009c60 tx=0x7f7eb0009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:33.915+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ea8097c90 con 0x7f7ec0106180 2024-05-08T23:29:34.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:33 smithi189 ceph-mon[46919]: pgmap v310: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.187+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec0102530 con 0x7f7ec0106180 2024-05-08T23:29:34.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.187+0000 7f7ebcff9700 1 -- 172.21.15.16:0/39309308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f7ea809b2b0 con 0x7f7ec0106180 2024-05-08T23:29:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:29:34.188 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:29:34.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7eac0a1310 msgr2=0x7f7eac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7eac0a1310 0x7f7eac0a37d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009c60 tx=0x7f7eb0009260 comp rx=0 tx=0).stop 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 msgr2=0x7f7ec0197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 0x7f7ec0197790 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005910 tx=0x7f7ea8004730 comp rx=0 tx=0).stop 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 shutdown_connections 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7eac0a1310 0x7f7eac0a37d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ec019bb30 0x7f7ec019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec0101060 0x7f7ec0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 --2- 172.21.15.16:0/39309308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ec0106180 0x7f7ec0197790 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 >> 172.21.15.16:0/39309308 conn(0x7f7ec0078360 msgr2=0x7f7ec00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 shutdown_connections 2024-05-08T23:29:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.190+0000 7f7ec5cee700 1 -- 172.21.15.16:0/39309308 wait complete. 2024-05-08T23:29:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:29:34.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:29:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:33 smithi155 ceph-mon[47105]: pgmap v310: 129 pgs: 129 active+clean; 16 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 -- 172.21.15.16:0/1429772509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08105f90 msgr2=0x7f5d08106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/1429772509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08105f90 0x7f5d08106370 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80099c0 tx=0x7f5cf8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 -- 172.21.15.16:0/1429772509 shutdown_connections 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/1429772509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d08100ab0 0x7f5d0810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/1429772509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08105f90 0x7f5d08106370 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/1429772509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d081000f0 0x7f5d08100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 -- 172.21.15.16:0/1429772509 >> 172.21.15.16:0/1429772509 conn(0x7f5d08076ba0 msgr2=0x7f5d08076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 -- 172.21.15.16:0/1429772509 shutdown_connections 2024-05-08T23:29:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.343+0000 7f5d0cb17700 1 -- 172.21.15.16:0/1429772509 wait complete. 2024-05-08T23:29:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.344+0000 7f5d0cb17700 1 Processor -- start 2024-05-08T23:29:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 -- start start 2024-05-08T23:29:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d081000f0 0x7f5d081980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 0x7f5d08198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d08105f90 0x7f5d081a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08198e90 con 0x7f5d08100ab0 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d081a36a0 con 0x7f5d081000f0 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0cb17700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d081a37e0 con 0x7f5d08105f90 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 0x7f5d08198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 0x7f5d08198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60004/0 (socket says 172.21.15.16:60004) 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3049691965 learned_addr learned my addr 172.21.15.16:0/3049691965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d081000f0 0x7f5d081980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d06d9d700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d08105f90 0x7f5d081a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3049691965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d08105f90 msgr2=0x7f5d081a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d08105f90 0x7f5d081a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3049691965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d081000f0 msgr2=0x7f5d081980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d081000f0 0x7f5d081980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.345+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3049691965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80095f0 con 0x7f5d08100ab0 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.346+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 0x7f5d08198620 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00d980 tx=0x7f5cfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.346+0000 7f5cf77fe700 1 -- 172.21.15.16:0/3049691965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc004a30 con 0x7f5d08100ab0 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.346+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d081a3a70 con 0x7f5d08100ab0 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.346+0000 7f5cf77fe700 1 -- 172.21.15.16:0/3049691965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cfc0054b0 con 0x7f5d08100ab0 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.346+0000 7f5cf77fe700 1 -- 172.21.15.16:0/3049691965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc009bf0 con 0x7f5d08100ab0 2024-05-08T23:29:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.346+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d081a3f30 con 0x7f5d08100ab0 2024-05-08T23:29:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.348+0000 7f5cf77fe700 1 -- 172.21.15.16:0/3049691965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5cfc009d50 con 0x7f5d08100ab0 2024-05-08T23:29:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.349+0000 7f5cf77fe700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5cf00a1290 0x7f5cf00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.349+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5cf00a1290 0x7f5cf00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.349+0000 7f5cf77fe700 1 -- 172.21.15.16:0/3049691965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5cfc0ccf20 con 0x7f5d08100ab0 2024-05-08T23:29:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.349+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5cf00a1290 0x7f5cf00a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009990 tx=0x7f5cf8009f10 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 Processor -- start 2024-05-08T23:29:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 -- start start 2024-05-08T23:29:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x5598016601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x55980165a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165a890 0x55980165acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598015b1150 con 0x55980165fe00 2024-05-08T23:29:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598015aeb10 con 0x559801660720 2024-05-08T23:29:34.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598015aa400 con 0x55980165a890 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x5598016601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x5598016601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60020/0 (socket says 172.21.15.16:60020) 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d0659c700 1 -- 172.21.15.16:0/3021985070 learned_addr learned my addr 172.21.15.16:0/3021985070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d06d9d700 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165a890 0x55980165acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.350+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x55980165a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5d0659c700 1 -- 172.21.15.16:0/3021985070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165a890 msgr2=0x55980165acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165a890 0x55980165acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5d0659c700 1 -- 172.21.15.16:0/3021985070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 msgr2=0x55980165a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x55980165a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5d0659c700 1 -- 172.21.15.16:0/3021985070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980142b3b0 con 0x55980165fe00 2024-05-08T23:29:34.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x5598016601e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5d08067bb0 tx=0x7f5cf8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a320d8e8f8fa0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5cf4ff9700 1 -- 172.21.15.16:0/3021985070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800bdf0 con 0x55980165fe00 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5cf4ff9700 1 -- 172.21.15.16:0/3021985070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cf801e040 con 0x55980165fe00 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.351+0000 7f5cf4ff9700 1 -- 172.21.15.16:0/3021985070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800f6c0 con 0x55980165fe00 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3021985070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 msgr2=0x5598016601e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x5598016601e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f5d08067bb0 tx=0x7f5cf8020000 comp rx=0 tx=0).stop 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3021985070 shutdown_connections 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165a890 0x55980165acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x5598016601e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3021985070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x55980165a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3021985070 >> 172.21.15.16:0/3021985070 conn(0x5598015b8210 msgr2=0x559801654df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3021985070 shutdown_connections 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3021985070 wait complete. 2024-05-08T23:29:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.352+0000 7f5d18cd1b80 1 -- start start 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 Processor -- start 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x559801663b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x559801664040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801664580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801546560 con 0x55980165fe00 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801546200 con 0x55980165a890 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55980142d6a0 con 0x559801660720 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d18cd1b80 10 client.0 _ll_register_callbacks cb 0x55980154fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x559801664040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x559801664040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60032/0 (socket says 172.21.15.16:60032) 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 -- 172.21.15.16:0/2093193970 learned_addr learned my addr 172.21.15.16:0/2093193970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d0659c700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x559801663b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d06d9d700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801664580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 -- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 msgr2=0x559801664580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801664580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 -- 172.21.15.16:0/2093193970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 msgr2=0x559801663b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x559801663b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 -- 172.21.15.16:0/2093193970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0810d550 con 0x55980165fe00 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d0659c700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x559801663b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x559801664040 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009520 tx=0x7f5cfc09a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751cfd86301edcc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.353+0000 7f5cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5cedffb700 1 -- 172.21.15.16:0/2093193970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc09a0f0 con 0x55980165fe00 2024-05-08T23:29:34.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5cedffb700 1 -- 172.21.15.16:0/2093193970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5cfc0a2110 con 0x55980165fe00 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559801503270 con 0x55980165fe00 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5cedffb700 1 -- 172.21.15.16:0/2093193970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cfc099180 con 0x55980165fe00 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 10 client.17682 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5598013d7b60 con 0x55980165fe00 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5cedffb700 1 -- 172.21.15.16:0/2093193970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cfc063080 con 0x55980165fe00 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5cedffb700 1 -- 172.21.15.16:0/2093193970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5cfc00ebc0 con 0x55980165fe00 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 10 client.17682 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 10 client.17682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 20 client.17682 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559801673f70 0x559801676390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 4 client.17682 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.354+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559801629810 con 0x559801673f70 2024-05-08T23:29:34.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.355+0000 7f5d0659c700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559801673f70 0x559801676390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.355+0000 7f5d0659c700 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559801673f70 0x559801676390 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b5b5bb248554739 server_cookie=648544074056ee60 in_seq=0 out_seq=0 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.355+0000 7f5cedffb700 10 client.17682 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5cedffb700 1 -- 172.21.15.16:0/2093193970 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559801629810 con 0x559801673f70 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5cedffb700 10 client.17682 handle_command_reply: tid=0 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 client.17682 shutdown 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 msgr2=0x559801664040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x559801664040 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009520 tx=0x7f5cfc09a390 comp rx=0 tx=0).stop 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 shutdown_connections 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801664580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x559801673f70 0x559801676390 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x559801663b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/2093193970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x559801664040 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5cedffb700 0 client.17682 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:34.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 >> 172.21.15.16:0/2093193970 conn(0x5598015b8210 msgr2=0x5598016545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 shutdown_connections 2024-05-08T23:29:34.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/2093193970 wait complete. 2024-05-08T23:29:34.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.356+0000 7f5d18cd1b80 20 client.17682 trim_cache size 0 max 16384 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 Processor -- start 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 -- start start 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x55980165a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x55980165f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801654d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801665170 con 0x55980165fe00 2024-05-08T23:29:34.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016558e0 con 0x55980165a890 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559801656e60 con 0x559801660720 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.370+0000 7f5d05d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x55980165f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x55980165f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60046/0 (socket says 172.21.15.16:60046) 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3403587174 learned_addr learned my addr 172.21.15.16:0/3403587174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d06d9d700 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801654d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d0659c700 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x55980165a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3403587174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 msgr2=0x559801654d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801654d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3403587174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 msgr2=0x55980165a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x55980165a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 -- 172.21.15.16:0/3403587174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55980165af40 con 0x55980165fe00 2024-05-08T23:29:34.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x55980165f320 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc099e50 tx=0x7f5cfc005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5675a00d2f14e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5cf57fa700 1 -- 172.21.15.16:0/3403587174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0a2110 con 0x55980165fe00 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5cf57fa700 1 -- 172.21.15.16:0/3403587174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559801503270 con 0x55980165fe00 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.371+0000 7f5cf57fa700 1 -- 172.21.15.16:0/3403587174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0993e0 con 0x55980165fe00 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3403587174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 msgr2=0x55980165f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x55980165f320 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc099e50 tx=0x7f5cfc005870 comp rx=0 tx=0).stop 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3403587174 shutdown_connections 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559801660720 0x559801654d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55980165a890 0x55980165a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/3403587174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165fe00 0x55980165f320 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3403587174 >> 172.21.15.16:0/3403587174 conn(0x5598015b8210 msgr2=0x55980165ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3403587174 shutdown_connections 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/3403587174 wait complete. 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 -- start start 2024-05-08T23:29:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.372+0000 7f5d18cd1b80 1 Processor -- start 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 0x559801668ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165fe00 0x559801668ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x559801669530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016c9490 con 0x55980165a890 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016c95d0 con 0x559801660720 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598016c9710 con 0x55980165fe00 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d18cd1b80 10 client.0 _ll_register_callbacks cb 0x55980154fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 0x559801668ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 0x559801668ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60062/0 (socket says 172.21.15.16:60062) 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 -- 172.21.15.16:0/395802813 learned_addr learned my addr 172.21.15.16:0/395802813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165fe00 0x559801668ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d06d9d700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x559801669530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 -- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165fe00 msgr2=0x559801668ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165fe00 0x559801668ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 -- 172.21.15.16:0/395802813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 msgr2=0x559801669530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x559801669530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 -- 172.21.15.16:0/395802813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598013d7b60 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5d0659c700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 0x559801668ab0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009d10 tx=0x7f5cf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f266049445e2569 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5cedffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.373+0000 7f5cedffb700 1 -- 172.21.15.16:0/395802813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf803eea0 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5cedffb700 1 -- 172.21.15.16:0/395802813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5cf800be00 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cf801e040 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5cedffb700 1 -- 172.21.15.16:0/395802813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf8004700 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 10 client.17688 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5598016514e0 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5cedffb700 1 -- 172.21.15.16:0/395802813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cf801e040 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5cedffb700 1 -- 172.21.15.16:0/395802813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5cf800f460 con 0x55980165a890 2024-05-08T23:29:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 10 client.17688 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 10 client.17688 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:29:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 20 client.17688 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5598016caa20 0x5598016cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 4 client.17688 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:29:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.374+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598015a2ec0 con 0x5598016caa20 2024-05-08T23:29:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.375+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5598016caa20 0x5598016cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d05d9b700 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5598016caa20 0x5598016cce10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb3ce87a6a9543df server_cookie=7d6bbf5664105b66 in_seq=0 out_seq=0 2024-05-08T23:29:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5cedffb700 10 client.17688 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5cedffb700 1 -- 172.21.15.16:0/395802813 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598015a2ec0 con 0x5598016caa20 2024-05-08T23:29:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5cedffb700 10 client.17688 handle_command_reply: tid=0 2024-05-08T23:29:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 client.17688 shutdown 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 msgr2=0x559801668ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 0x559801668ab0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009d10 tx=0x7f5cf80057f0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 shutdown_connections 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5598016caa20 0x5598016cce10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559801660720 0x559801669530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55980165a890 0x559801668ab0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5cedffb700 0 client.17688 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 --2- 172.21.15.16:0/395802813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55980165fe00 0x559801668ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 >> 172.21.15.16:0/395802813 conn(0x5598015b8210 msgr2=0x5598016643e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 shutdown_connections 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.376+0000 7f5d18cd1b80 1 -- 172.21.15.16:0/395802813 wait complete. 2024-05-08T23:29:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d18cd1b80 20 client.17688 trim_cache size 0 max 16384 2024-05-08T23:29:34.384 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:29:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5cf00a1290 msgr2=0x7f5cf00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5cf00a1290 0x7f5cf00a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009990 tx=0x7f5cf8009f10 comp rx=0 tx=0).stop 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 msgr2=0x7f5d08198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 0x7f5d08198620 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00d980 tx=0x7f5cfc00dc90 comp rx=0 tx=0).stop 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 shutdown_connections 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5cf00a1290 0x7f5cf00a3750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d08105f90 0x7f5d081a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d081000f0 0x7f5d081980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 --2- 172.21.15.16:0/3049691965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d08100ab0 0x7f5d08198620 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 >> 172.21.15.16:0/3049691965 conn(0x7f5d08076ba0 msgr2=0x7f5d080ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 shutdown_connections 2024-05-08T23:29:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.377+0000 7f5d0cb17700 1 -- 172.21.15.16:0/3049691965 wait complete. 2024-05-08T23:29:34.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:29:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 -- 172.21.15.16:0/2941365256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100a30 msgr2=0x7fef3c104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 --2- 172.21.15.16:0/2941365256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100a30 0x7fef3c104a70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fef30009990 tx=0x7fef30009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 -- 172.21.15.16:0/2941365256 shutdown_connections 2024-05-08T23:29:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 --2- 172.21.15.16:0/2941365256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef3c1051a0 0x7fef3c10cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 --2- 172.21.15.16:0/2941365256 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef3c100080 0x7fef3c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 --2- 172.21.15.16:0/2941365256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100a30 0x7fef3c104a70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 -- 172.21.15.16:0/2941365256 >> 172.21.15.16:0/2941365256 conn(0x7fef3c076f40 msgr2=0x7fef3c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 -- 172.21.15.16:0/2941365256 shutdown_connections 2024-05-08T23:29:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 -- 172.21.15.16:0/2941365256 wait complete. 2024-05-08T23:29:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 Processor -- start 2024-05-08T23:29:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.530+0000 7fef42a61700 1 -- start start 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef42a61700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 0x7fef3c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef42a61700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef3c1051a0 0x7fef3c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef42a61700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef3c19bbd0 0x7fef3c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef42a61700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c19c650 con 0x7fef3c100080 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef42a61700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c19c7c0 con 0x7fef3c19bbd0 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef42a61700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c19c980 con 0x7fef3c1051a0 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 0x7fef3c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 0x7fef3c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60082/0 (socket says 172.21.15.16:60082) 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 -- 172.21.15.16:0/3476940433 learned_addr learned my addr 172.21.15.16:0/3476940433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef3c1051a0 0x7fef3c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef40ffe700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef3c19bbd0 0x7fef3c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 -- 172.21.15.16:0/3476940433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef3c1051a0 msgr2=0x7fef3c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef3c1051a0 0x7fef3c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 -- 172.21.15.16:0/3476940433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef3c19bbd0 msgr2=0x7fef3c19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef3c19bbd0 0x7fef3c19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.531+0000 7fef3bfff700 1 -- 172.21.15.16:0/3476940433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300095f0 con 0x7fef3c100080 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.532+0000 7fef3bfff700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 0x7fef3c1972f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00e9a0 tx=0x7fef2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.532+0000 7fef397fa700 1 -- 172.21.15.16:0/3476940433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c009790 con 0x7fef3c100080 2024-05-08T23:29:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.532+0000 7fef397fa700 1 -- 172.21.15.16:0/3476940433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef2c010430 con 0x7fef3c100080 2024-05-08T23:29:34.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.532+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c1a3ee0 con 0x7fef3c100080 2024-05-08T23:29:34.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.532+0000 7fef397fa700 1 -- 172.21.15.16:0/3476940433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00cea0 con 0x7fef3c100080 2024-05-08T23:29:34.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.532+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c1a4300 con 0x7fef3c100080 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.534+0000 7fef397fa700 1 -- 172.21.15.16:0/3476940433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fef2c0105a0 con 0x7fef3c100080 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.535+0000 7fef397fa700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fef240a1290 0x7fef240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.535+0000 7fef397fa700 1 -- 172.21.15.16:0/3476940433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fef2c014070 con 0x7fef3c100080 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.535+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fef240a1290 0x7fef240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.535+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fef240a1290 0x7fef240a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef3000b2e0 tx=0x7fef30005040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 Processor -- start 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 -- start start 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad6a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602bad64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad64cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bacbb150 con 0x5602bad6a720 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bacb8b10 con 0x5602bad69e00 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bacb4400 con 0x5602bad64890 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602bad64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602bad64350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60098/0 (socket says 172.21.15.16:60098) 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef3b7fe700 1 -- 172.21.15.16:0/664686474 learned_addr learned my addr 172.21.15.16:0/664686474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef40ffe700 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad64cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.536+0000 7fef3bfff700 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad6a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef3b7fe700 1 -- 172.21.15.16:0/664686474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 msgr2=0x5602bad64cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad64cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef3b7fe700 1 -- 172.21.15.16:0/664686474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 msgr2=0x5602bad6a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad6a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef3b7fe700 1 -- 172.21.15.16:0/664686474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602bab353b0 con 0x5602bad6a720 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602bad64350 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fef30015000 tx=0x7fef30020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec99af77096d6a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef227fc700 1 -- 172.21.15.16:0/664686474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30004500 con 0x5602bad6a720 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef227fc700 1 -- 172.21.15.16:0/664686474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef3001e040 con 0x5602bad6a720 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.537+0000 7fef227fc700 1 -- 172.21.15.16:0/664686474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3000f750 con 0x5602bad6a720 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/664686474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 msgr2=0x5602bad64350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602bad64350 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fef30015000 tx=0x7fef30020000 comp rx=0 tx=0).stop 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/664686474 shutdown_connections 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad64cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad6a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/664686474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602bad64350 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/664686474 >> 172.21.15.16:0/664686474 conn(0x5602bacc2210 msgr2=0x5602bad5edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/664686474 shutdown_connections 2024-05-08T23:29:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/664686474 wait complete. 2024-05-08T23:29:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 -- start start 2024-05-08T23:29:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.538+0000 7fef4ec1bb80 1 Processor -- start 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad70750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad6da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad6e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bac50560 con 0x5602bad64890 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bac50200 con 0x5602bad6a720 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bab376a0 con 0x5602bad69e00 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef4ec1bb80 10 client.0 _ll_register_callbacks cb 0x5602bac59b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad70750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad70750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60108/0 (socket says 172.21.15.16:60108) 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 -- 172.21.15.16:0/7401759 learned_addr learned my addr 172.21.15.16:0/7401759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef40ffe700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad6e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad6da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 -- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 msgr2=0x5602bad6da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad6da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 -- 172.21.15.16:0/7401759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 msgr2=0x5602bad6e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad6e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 -- 172.21.15.16:0/7401759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c10d590 con 0x5602bad64890 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef40ffe700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad6e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef3bfff700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad70750 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fef2c099f00 tx=0x7fef2c0109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188418fcaef90aef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.539+0000 7fef1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef1b7fe700 1 -- 172.21.15.16:0/7401759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c09dda0 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef1b7fe700 1 -- 172.21.15.16:0/7401759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fef2c0ade40 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602bac0d270 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef1b7fe700 1 -- 172.21.15.16:0/7401759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef2c0a23f0 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 10 client.17700 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5602baae1b60 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef1b7fe700 1 -- 172.21.15.16:0/7401759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef2c0b8020 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef1b7fe700 1 -- 172.21.15.16:0/7401759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fef2c094b80 con 0x5602bad64890 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 10 client.17700 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 10 client.17700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 20 client.17700 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602bad7e1c0 0x5602bad805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 4 client.17700 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.540+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602bad33810 con 0x5602bad7e1c0 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.541+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602bad7e1c0 0x5602bad805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.541+0000 7fef1b7fe700 10 client.17700 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.541+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602bad7e1c0 0x5602bad805b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13c575bf42555dca server_cookie=8a327a17c713aa97 in_seq=0 out_seq=0 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef1b7fe700 1 -- 172.21.15.16:0/7401759 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602bad33810 con 0x5602bad7e1c0 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef1b7fe700 10 client.17700 handle_command_reply: tid=0 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 client.17700 shutdown 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 msgr2=0x5602bad70750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad70750 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fef2c099f00 tx=0x7fef2c0109b0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 shutdown_connections 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602bad7e1c0 0x5602bad805b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad6e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad70750 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef1b7fe700 0 client.17700 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/7401759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad6da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 >> 172.21.15.16:0/7401759 conn(0x5602bacc2210 msgr2=0x5602bad5e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 shutdown_connections 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/7401759 wait complete. 2024-05-08T23:29:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.542+0000 7fef4ec1bb80 20 client.17700 trim_cache size 0 max 16384 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 Processor -- start 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 -- start start 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad79bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad5ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bad59ac0 con 0x5602bad64890 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bad6e9c0 con 0x5602bad6a720 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.556+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bad59800 con 0x5602bad69e00 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad64350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60116/0 (socket says 172.21.15.16:60116) 2024-05-08T23:29:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 -- 172.21.15.16:0/1489402506 learned_addr learned my addr 172.21.15.16:0/1489402506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad79bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 -- 172.21.15.16:0/1489402506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 msgr2=0x5602bad79bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad79bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef40ffe700 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad5ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 -- 172.21.15.16:0/1489402506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 msgr2=0x5602bad5ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad5ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 -- 172.21.15.16:0/1489402506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602bac7a560 con 0x5602bad64890 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef40ffe700 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef3bfff700 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad64350 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0969e0 tx=0x7fef2c005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ca9a9a61ae7ebd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef22ffd700 1 -- 172.21.15.16:0/1489402506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c09dda0 con 0x5602bad64890 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef22ffd700 1 -- 172.21.15.16:0/1489402506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef2c0b8040 con 0x5602bad64890 2024-05-08T23:29:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.557+0000 7fef22ffd700 1 -- 172.21.15.16:0/1489402506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c094b80 con 0x5602bad64890 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/1489402506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 msgr2=0x5602bad64350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad64350 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0969e0 tx=0x7fef2c005b40 comp rx=0 tx=0).stop 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/1489402506 shutdown_connections 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad6a720 0x5602bad5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad64890 0x5602bad64350 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/1489402506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad69e00 0x5602bad79bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/1489402506 >> 172.21.15.16:0/1489402506 conn(0x5602bacc2210 msgr2=0x5602bad64c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/1489402506 shutdown_connections 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/1489402506 wait complete. 2024-05-08T23:29:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.558+0000 7fef4ec1bb80 1 -- start start 2024-05-08T23:29:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 Processor -- start 2024-05-08T23:29:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad71620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602badd3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602bad71f00 con 0x5602bad6a720 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602badd3b90 con 0x5602bad69e00 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602badd3cd0 con 0x5602bad64890 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef4ec1bb80 10 client.0 _ll_register_callbacks cb 0x5602bac59b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602badd3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602badd3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60120/0 (socket says 172.21.15.16:60120) 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 -- 172.21.15.16:0/338022406 learned_addr learned my addr 172.21.15.16:0/338022406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef3b7fe700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad71620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef3bfff700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 -- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 msgr2=0x5602bad710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 -- 172.21.15.16:0/338022406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 msgr2=0x5602bad71620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad71620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 -- 172.21.15.16:0/338022406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602baae1b60 con 0x5602bad6a720 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef40ffe700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602badd3650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fef34008e90 tx=0x7fef340076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687a4b9a480b5bed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.559+0000 7fef1b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef1b7fe700 1 -- 172.21.15.16:0/338022406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3400e040 con 0x5602bad6a720 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef1b7fe700 1 -- 172.21.15.16:0/338022406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fef34004db0 con 0x5602bad6a720 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef3001e040 con 0x5602bad6a720 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef1b7fe700 1 -- 172.21.15.16:0/338022406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef34007b20 con 0x5602bad6a720 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 10 client.17706 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:34.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5602bab39cd0 con 0x5602bad6a720 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef1b7fe700 1 -- 172.21.15.16:0/338022406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef3001e040 con 0x5602bad6a720 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef1b7fe700 1 -- 172.21.15.16:0/338022406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fef34018420 con 0x5602bad6a720 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 10 client.17706 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 10 client.17706 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 20 client.17706 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602badd53f0 0x5602badd7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 4 client.17706 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5602bacacec0 con 0x5602badd53f0 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.560+0000 7fef3bfff700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602badd53f0 0x5602badd7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef1b7fe700 10 client.17706 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef3bfff700 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602badd53f0 0x5602badd7810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc3d9e8db1f5f173 server_cookie=a08ccf3bf6c20b92 in_seq=0 out_seq=0 2024-05-08T23:29:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef1b7fe700 1 -- 172.21.15.16:0/338022406 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5602bacacec0 con 0x5602badd53f0 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef1b7fe700 10 client.17706 handle_command_reply: tid=0 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 client.17706 shutdown 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 msgr2=0x5602badd3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602badd3650 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fef34008e90 tx=0x7fef340076c0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 shutdown_connections 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602bad6a720 0x5602badd3650 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602bad64890 0x5602bad710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x5602badd53f0 0x5602badd7810 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 --2- 172.21.15.16:0/338022406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5602bad69e00 0x5602bad71620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 >> 172.21.15.16:0/338022406 conn(0x5602bacc2210 msgr2=0x5602bad67bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef1b7fe700 0 client.17706 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 shutdown_connections 2024-05-08T23:29:34.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.561+0000 7fef4ec1bb80 1 -- 172.21.15.16:0/338022406 wait complete. 2024-05-08T23:29:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef4ec1bb80 20 client.17706 trim_cache size 0 max 16384 2024-05-08T23:29:34.570 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fef240a1290 msgr2=0x7fef240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fef240a1290 0x7fef240a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef3000b2e0 tx=0x7fef30005040 comp rx=0 tx=0).stop 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 msgr2=0x7fef3c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 0x7fef3c1972f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00e9a0 tx=0x7fef2c00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 shutdown_connections 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fef240a1290 0x7fef240a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef3c19bbd0 0x7fef3c19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef3c100080 0x7fef3c1972f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 --2- 172.21.15.16:0/3476940433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef3c1051a0 0x7fef3c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 >> 172.21.15.16:0/3476940433 conn(0x7fef3c076f40 msgr2=0x7fef3c069850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 shutdown_connections 2024-05-08T23:29:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.562+0000 7fef42a61700 1 -- 172.21.15.16:0/3476940433 wait complete. 2024-05-08T23:29:34.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.714+0000 7fc1a6282700 1 -- 172.21.15.16:0/1110759231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0100a10 msgr2=0x7fc1a0104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.714+0000 7fc1a6282700 1 --2- 172.21.15.16:0/1110759231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0100a10 0x7fc1a0104a30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc188009990 tx=0x7fc188009ca0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 -- 172.21.15.16:0/1110759231 shutdown_connections 2024-05-08T23:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 --2- 172.21.15.16:0/1110759231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0105160 0x7fc1a010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 --2- 172.21.15.16:0/1110759231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0100060 0x7fc1a0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 --2- 172.21.15.16:0/1110759231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0100a10 0x7fc1a0104a30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 -- 172.21.15.16:0/1110759231 >> 172.21.15.16:0/1110759231 conn(0x7fc1a0076f40 msgr2=0x7fc1a0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 -- 172.21.15.16:0/1110759231 shutdown_connections 2024-05-08T23:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 -- 172.21.15.16:0/1110759231 wait complete. 2024-05-08T23:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 Processor -- start 2024-05-08T23:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.715+0000 7fc1a6282700 1 -- start start 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a6282700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0100060 0x7fc1a0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a6282700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 0x7fc1a0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a6282700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0198bc0 0x7fc1a01a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a6282700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01a3860 con 0x7fc1a0105160 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a6282700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01a39d0 con 0x7fc1a0100060 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a6282700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a01a3b90 con 0x7fc1a0198bc0 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 0x7fc1a0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 0x7fc1a0198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60140/0 (socket says 172.21.15.16:60140) 2024-05-08T23:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 -- 172.21.15.16:0/139176529 learned_addr learned my addr 172.21.15.16:0/139176529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a481f700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0198bc0 0x7fc1a01a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19ffff700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0100060 0x7fc1a0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 -- 172.21.15.16:0/139176529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0198bc0 msgr2=0x7fc1a01a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0198bc0 0x7fc1a01a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 -- 172.21.15.16:0/139176529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0100060 msgr2=0x7fc1a0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0100060 0x7fc1a0198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 -- 172.21.15.16:0/139176529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880095f0 con 0x7fc1a0105160 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc1a481f700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0198bc0 0x7fc1a01a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19ffff700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0100060 0x7fc1a0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.716+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 0x7fc1a0198680 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc1880095a0 tx=0x7fc188004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.717+0000 7fc19d7fa700 1 -- 172.21.15.16:0/139176529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18801d070 con 0x7fc1a0105160 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.717+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a01a3e10 con 0x7fc1a0105160 2024-05-08T23:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.717+0000 7fc19d7fa700 1 -- 172.21.15.16:0/139176529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc188005030 con 0x7fc1a0105160 2024-05-08T23:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.717+0000 7fc19d7fa700 1 -- 172.21.15.16:0/139176529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188004b00 con 0x7fc1a0105160 2024-05-08T23:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.717+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a01a41d0 con 0x7fc1a0105160 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.719+0000 7fc19d7fa700 1 -- 172.21.15.16:0/139176529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc1880051a0 con 0x7fc1a0105160 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.720+0000 7fc19d7fa700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc18c0a1290 0x7fc18c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.720+0000 7fc19ffff700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc18c0a1290 0x7fc18c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.720+0000 7fc19d7fa700 1 -- 172.21.15.16:0/139176529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc1880ce070 con 0x7fc1a0105160 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.720+0000 7fc1b243cb80 1 Processor -- start 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.720+0000 7fc19ffff700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc18c0a1290 0x7fc18c0a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc190009be0 tx=0x7fc190009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.720+0000 7fc1b243cb80 1 -- start start 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca3d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca38170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ac98a150 con 0x5640aca3d260 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ac987b10 con 0x5640aca3db80 2024-05-08T23:29:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ac983400 con 0x5640aca37cf0 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca3d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca3d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60154/0 (socket says 172.21.15.16:60154) 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 -- 172.21.15.16:0/3985848736 learned_addr learned my addr 172.21.15.16:0/3985848736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1a481f700 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca38170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 -- 172.21.15.16:0/3985848736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 msgr2=0x5640aca38170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca38170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 -- 172.21.15.16:0/3985848736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 msgr2=0x5640aca37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 -- 172.21.15.16:0/3985848736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640ac8043b0 con 0x5640aca3d260 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc19ffff700 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca3d640 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc190010d00 tx=0x7fc190013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e688b3789816ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.721+0000 7fc1967fc700 1 -- 172.21.15.16:0/3985848736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190012b60 con 0x5640aca3d260 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1967fc700 1 -- 172.21.15.16:0/3985848736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc190012cc0 con 0x5640aca3d260 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1967fc700 1 -- 172.21.15.16:0/3985848736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19001e5d0 con 0x5640aca3d260 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3985848736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 msgr2=0x5640aca3d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca3d640 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc190010d00 tx=0x7fc190013000 comp rx=0 tx=0).stop 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3985848736 shutdown_connections 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca38170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca3d640 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3985848736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3985848736 >> 172.21.15.16:0/3985848736 conn(0x5640ac991210 msgr2=0x5640aca2ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3985848736 shutdown_connections 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.722+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3985848736 wait complete. 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 -- start start 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 Processor -- start 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca37cf0 0x5640aca40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca418d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ac91f560 con 0x5640aca3d260 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ac91f200 con 0x5640aca37cf0 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ac8066a0 con 0x5640aca3db80 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1b243cb80 10 client.0 _ll_register_callbacks cb 0x5640ac928b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca41390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60160/0 (socket says 172.21.15.16:60160) 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc19f7fe700 1 -- 172.21.15.16:0/2258865254 learned_addr learned my addr 172.21.15.16:0/2258865254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc19ffff700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca37cf0 0x5640aca40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.723+0000 7fc1a481f700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc19f7fe700 1 -- 172.21.15.16:0/2258865254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 msgr2=0x5640aca418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc19f7fe700 1 -- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca37cf0 msgr2=0x5640aca40e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca37cf0 0x5640aca40e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc19f7fe700 1 -- 172.21.15.16:0/2258865254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1900109b0 con 0x5640aca3d260 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca41390 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc188005450 tx=0x7fc18809b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4250ca7084b04d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc17f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc17f7fe700 1 -- 172.21.15.16:0/2258865254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18809b270 con 0x5640aca3d260 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc17f7fe700 1 -- 172.21.15.16:0/2258865254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc188004e70 con 0x5640aca3d260 2024-05-08T23:29:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640ac8dc270 con 0x5640aca3d260 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc17f7fe700 1 -- 172.21.15.16:0/2258865254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1880a3200 con 0x5640aca3d260 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc1b243cb80 10 client.17718 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5640ac7b0b60 con 0x5640aca3d260 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.724+0000 7fc17f7fe700 1 -- 172.21.15.16:0/2258865254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5640ac8dc270 con 0x5640aca3d260 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc17f7fe700 1 -- 172.21.15.16:0/2258865254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc18809fb60 con 0x5640aca3d260 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc1b243cb80 10 client.17718 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc1b243cb80 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc1b243cb80 20 client.17718 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640aca51140 0x5640aca53530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc1b243cb80 4 client.17718 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640aca02810 con 0x5640aca51140 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc19ffff700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640aca51140 0x5640aca53530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc17f7fe700 10 client.17718 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.725+0000 7fc19ffff700 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640aca51140 0x5640aca53530 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1cb76abe2b2f8e server_cookie=9d155920d4ca1336 in_seq=0 out_seq=0 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc17f7fe700 1 -- 172.21.15.16:0/2258865254 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640aca02810 con 0x5640aca51140 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc17f7fe700 10 client.17718 handle_command_reply: tid=0 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 client.17718 shutdown 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 msgr2=0x5640aca41390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca41390 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc188005450 tx=0x7fc18809b590 comp rx=0 tx=0).stop 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 shutdown_connections 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640aca51140 0x5640aca53530 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca37cf0 0x5640aca40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/2258865254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca41390 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 >> 172.21.15.16:0/2258865254 conn(0x5640ac991210 msgr2=0x5640aca2d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc17f7fe700 0 client.17718 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 shutdown_connections 2024-05-08T23:29:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.726+0000 7fc1b243cb80 1 -- 172.21.15.16:0/2258865254 wait complete. 2024-05-08T23:29:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.727+0000 7fc1b243cb80 20 client.17718 trim_cache size 0 max 16384 2024-05-08T23:29:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 Processor -- start 2024-05-08T23:29:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 -- start start 2024-05-08T23:29:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 0x5640aca4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 0x5640aca4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640aca40c60 con 0x5640aca37cf0 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640aca3d890 con 0x5640aca3d260 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640aca439c0 con 0x5640aca3db80 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc19ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 0x5640aca4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 0x5640aca4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 0x5640aca4d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:40360/0 (socket says 172.21.15.16:40360) 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc19ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 0x5640aca4d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60174/0 (socket says 172.21.15.16:60174) 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.743+0000 7fc19f7fe700 1 -- 172.21.15.16:0/3816120550 learned_addr learned my addr 172.21.15.16:0/3816120550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc1a481f700 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19f7fe700 1 -- 172.21.15.16:0/3816120550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 msgr2=0x5640aca2dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca2dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19f7fe700 1 -- 172.21.15.16:0/3816120550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 msgr2=0x5640aca4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 0x5640aca4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19f7fe700 1 -- 172.21.15.16:0/3816120550 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640ac949560 con 0x5640aca3d260 2024-05-08T23:29:34.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19ffff700 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 0x5640aca4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:29:34.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.744+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 0x5640aca4d020 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc188097880 tx=0x7fc188009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce7f2442c058504 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.748+0000 7fc196ffd700 1 -- 172.21.15.16:0/3816120550 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188004e70 con 0x5640aca3d260 2024-05-08T23:29:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.748+0000 7fc196ffd700 1 -- 172.21.15.16:0/3816120550 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1880a3200 con 0x5640aca3d260 2024-05-08T23:29:34.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.748+0000 7fc196ffd700 1 -- 172.21.15.16:0/3816120550 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18809a520 con 0x5640aca3d260 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3816120550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 msgr2=0x5640aca4d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 0x5640aca4d020 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc188097880 tx=0x7fc188009310 comp rx=0 tx=0).stop 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3816120550 shutdown_connections 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca3db80 0x5640aca2dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca37cf0 0x5640aca4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/3816120550 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3d260 0x5640aca4d020 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3816120550 >> 172.21.15.16:0/3816120550 conn(0x5640ac991210 msgr2=0x5640aca380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3816120550 shutdown_connections 2024-05-08T23:29:34.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 -- 172.21.15.16:0/3816120550 wait complete. 2024-05-08T23:29:34.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.749+0000 7fc1b243cb80 1 -- start start 2024-05-08T23:29:34.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 Processor -- start 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca39290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640aca39990 con 0x5640aca3d260 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640aca39b00 con 0x5640aca3db80 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640aca39cc0 con 0x5640aca37cf0 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1b243cb80 10 client.0 _ll_register_callbacks cb 0x5640ac928b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc19f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60180/0 (socket says 172.21.15.16:60180) 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc19f7fe700 1 -- 172.21.15.16:0/137007787 learned_addr learned my addr 172.21.15.16:0/137007787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc19ffff700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.750+0000 7fc1a481f700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca39290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc19f7fe700 1 -- 172.21.15.16:0/137007787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 msgr2=0x5640aca489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc19f7fe700 1 -- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 msgr2=0x5640aca39290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca39290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc19f7fe700 1 -- 172.21.15.16:0/137007787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640ac8043b0 con 0x5640aca3d260 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc19f7fe700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca38c60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc188005cd0 tx=0x7fc18809fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf72c80db3c536f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc17f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc17f7fe700 1 -- 172.21.15.16:0/137007787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18809b270 con 0x5640aca3d260 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc17f7fe700 1 -- 172.21.15.16:0/137007787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc188004e70 con 0x5640aca3d260 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc190012cc0 con 0x5640aca3d260 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc17f7fe700 1 -- 172.21.15.16:0/137007787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18800fb40 con 0x5640aca3d260 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc1b243cb80 10 client.17724 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:34.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5640ac7b0b60 con 0x5640aca3d260 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.751+0000 7fc17f7fe700 1 -- 172.21.15.16:0/137007787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc190012cc0 con 0x5640aca3d260 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc17f7fe700 1 -- 172.21.15.16:0/137007787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc188096c00 con 0x5640aca3d260 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc1b243cb80 10 client.17724 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc1b243cb80 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc1b243cb80 20 client.17724 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640acaa7910 0x5640acaa9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc1b243cb80 4 client.17724 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5640ac97bec0 con 0x5640acaa7910 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc19ffff700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640acaa7910 0x5640acaa9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc17f7fe700 10 client.17724 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc19ffff700 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640acaa7910 0x5640acaa9d00 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e215f89a9d27 server_cookie=5ec6016e37f95d41 in_seq=0 out_seq=0 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc17f7fe700 1 -- 172.21.15.16:0/137007787 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5640ac97bec0 con 0x5640acaa7910 2024-05-08T23:29:34.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.752+0000 7fc17f7fe700 10 client.17724 handle_command_reply: tid=0 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 client.17724 shutdown 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 msgr2=0x5640aca38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca38c60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc188005cd0 tx=0x7fc18809fd90 comp rx=0 tx=0).stop 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 shutdown_connections 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5640acaa7910 0x5640acaa9d00 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5640aca3db80 0x5640aca39290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640aca37cf0 0x5640aca489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 --2- 172.21.15.16:0/137007787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640aca3d260 0x5640aca38c60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc17f7fe700 0 client.17724 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 >> 172.21.15.16:0/137007787 conn(0x5640ac991210 msgr2=0x5640aca3af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 shutdown_connections 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 1 -- 172.21.15.16:0/137007787 wait complete. 2024-05-08T23:29:34.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1b243cb80 20 client.17724 trim_cache size 0 max 16384 2024-05-08T23:29:34.759 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc18c0a1290 msgr2=0x7fc18c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.753+0000 7fc1a6282700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc18c0a1290 0x7fc18c0a3750 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc190009be0 tx=0x7fc190009260 comp rx=0 tx=0).stop 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 msgr2=0x7fc1a0198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 0x7fc1a0198680 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc1880095a0 tx=0x7fc188004500 comp rx=0 tx=0).stop 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 shutdown_connections 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc18c0a1290 0x7fc18c0a3750 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc1a0198bc0 0x7fc1a01a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a0100060 0x7fc1a0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 --2- 172.21.15.16:0/139176529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0105160 0x7fc1a0198680 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 >> 172.21.15.16:0/139176529 conn(0x7fc1a0076f40 msgr2=0x7fc1a00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 shutdown_connections 2024-05-08T23:29:34.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:34.754+0000 7fc1a6282700 1 -- 172.21.15.16:0/139176529 wait complete. 2024-05-08T23:29:34.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:34 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/39309308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:29:35.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:34 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/39309308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:29:35.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:34 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/39309308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:29:35.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:29:35.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:29:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 -- 172.21.15.16:0/1081634368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb044100ab0 msgr2=0x7fb04410ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 --2- 172.21.15.16:0/1081634368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb044100ab0 0x7fb04410ccc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00b0d0 tx=0x7fb03c00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 -- 172.21.15.16:0/1081634368 shutdown_connections 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 --2- 172.21.15.16:0/1081634368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb044100ab0 0x7fb04410ccc0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 --2- 172.21.15.16:0/1081634368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044105f90 0x7fb044106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 --2- 172.21.15.16:0/1081634368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0441000f0 0x7fb044100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 -- 172.21.15.16:0/1081634368 >> 172.21.15.16:0/1081634368 conn(0x7fb044076ba0 msgr2=0x7fb044076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 -- 172.21.15.16:0/1081634368 shutdown_connections 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.917+0000 7fb04a01b700 1 -- 172.21.15.16:0/1081634368 wait complete. 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 Processor -- start 2024-05-08T23:29:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 -- start start 2024-05-08T23:29:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 0x7fb044101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044100ab0 0x7fb044102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb044105f90 0x7fb0441025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0441a56d0 con 0x7fb0441000f0 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0441a5840 con 0x7fb044105f90 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb04a01b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0441a5a00 con 0x7fb044100ab0 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.918+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 0x7fb044101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 0x7fb044101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60192/0 (socket says 172.21.15.16:60192) 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 -- 172.21.15.16:0/659260200 learned_addr learned my addr 172.21.15.16:0/659260200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb042ffd700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044100ab0 0x7fb044102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb043fff700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb044105f90 0x7fb0441025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 -- 172.21.15.16:0/659260200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044100ab0 msgr2=0x7fb044102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044100ab0 0x7fb044102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 -- 172.21.15.16:0/659260200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb044105f90 msgr2=0x7fb0441025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb044105f90 0x7fb0441025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 -- 172.21.15.16:0/659260200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009d00 con 0x7fb0441000f0 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb043fff700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb044105f90 0x7fb0441025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb042ffd700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044100ab0 0x7fb044102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb0437fe700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 0x7fb044101b20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00b550 tx=0x7fb02c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb040ff9700 1 -- 172.21.15.16:0/659260200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c011830 con 0x7fb0441000f0 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb040ff9700 1 -- 172.21.15.16:0/659260200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb02c011e20 con 0x7fb0441000f0 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0441a5ce0 con 0x7fb0441000f0 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb040ff9700 1 -- 172.21.15.16:0/659260200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c010dd0 con 0x7fb0441000f0 2024-05-08T23:29:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.919+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0441a61a0 con 0x7fb0441000f0 2024-05-08T23:29:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.922+0000 7fb040ff9700 1 -- 172.21.15.16:0/659260200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb02c011990 con 0x7fb0441000f0 2024-05-08T23:29:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.923+0000 7fb040ff9700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb0300a1290 0x7fb0300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.923+0000 7fb040ff9700 1 -- 172.21.15.16:0/659260200 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb02c0ccac0 con 0x7fb0441000f0 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.923+0000 7fb042ffd700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb0300a1290 0x7fb0300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.923+0000 7fb0561d5b80 1 Processor -- start 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb0300a1290 0x7fb0300a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb044103250 tx=0x7fb034008040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 -- start start 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fef8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fef23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fe491d0 con 0x563d1fef87a0 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fe46b90 con 0x563d1fef7e80 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fe42480 con 0x563d1fef2910 2024-05-08T23:29:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fef23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fef23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60196/0 (socket says 172.21.15.16:60196) 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 -- 172.21.15.16:0/1571909351 learned_addr learned my addr 172.21.15.16:0/1571909351 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb0437fe700 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fef8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb043fff700 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 -- 172.21.15.16:0/1571909351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 msgr2=0x563d1fef2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 -- 172.21.15.16:0/1571909351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 msgr2=0x563d1fef8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fef8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.924+0000 7fb042ffd700 1 -- 172.21.15.16:0/1571909351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d1fcc3430 con 0x563d1fef87a0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.925+0000 7fb042ffd700 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fef23d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb03400d750 tx=0x7fb034010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1691d35da5504a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.925+0000 7fb039ffb700 1 -- 172.21.15.16:0/1571909351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034014640 con 0x563d1fef87a0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.925+0000 7fb039ffb700 1 -- 172.21.15.16:0/1571909351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb034014c30 con 0x563d1fef87a0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.925+0000 7fb039ffb700 1 -- 172.21.15.16:0/1571909351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03401fb10 con 0x563d1fef87a0 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 -- 172.21.15.16:0/1571909351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 msgr2=0x563d1fef23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fef23d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb03400d750 tx=0x7fb034010000 comp rx=0 tx=0).stop 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 -- 172.21.15.16:0/1571909351 shutdown_connections 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fef8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/1571909351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fef23d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 -- 172.21.15.16:0/1571909351 >> 172.21.15.16:0/1571909351 conn(0x563d1fe50290 msgr2=0x563d1feece70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 -- 172.21.15.16:0/1571909351 shutdown_connections 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 -- 172.21.15.16:0/1571909351 wait complete. 2024-05-08T23:29:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 -- start start 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.926+0000 7fb0561d5b80 1 Processor -- start 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fefe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fefbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fefc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fdde5e0 con 0x563d1fef87a0 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fdde280 con 0x563d1fef7e80 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fcc5720 con 0x563d1fef2910 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0561d5b80 10 client.0 _ll_register_callbacks cb 0x563d1fde7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fefc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fefc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60200/0 (socket says 172.21.15.16:60200) 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 -- 172.21.15.16:0/2389139846 learned_addr learned my addr 172.21.15.16:0/2389139846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:35.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fefe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb042ffd700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fefbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 -- 172.21.15.16:0/2389139846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 msgr2=0x563d1fefe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fefe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 -- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 msgr2=0x563d1fefbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fefbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 -- 172.21.15.16:0/2389139846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03400e040 con 0x563d1fef87a0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb043fff700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fefc200 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0095b0 tx=0x7fb03c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86a2abd82920d38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.927+0000 7fb022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb022ffd700 1 -- 172.21.15.16:0/2389139846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00e030 con 0x563d1fef87a0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb022ffd700 1 -- 172.21.15.16:0/2389139846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb03c007dd0 con 0x563d1fef87a0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d1fd9b2f0 con 0x563d1fef87a0 2024-05-08T23:29:35.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb022ffd700 1 -- 172.21.15.16:0/2389139846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb03c004e80 con 0x563d1fef87a0 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 10 client.17736 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d1fc6fbe0 con 0x563d1fef87a0 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb022ffd700 1 -- 172.21.15.16:0/2389139846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d1fd9b2f0 con 0x563d1fef87a0 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb022ffd700 1 -- 172.21.15.16:0/2389139846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb03c00b100 con 0x563d1fef87a0 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 10 client.17736 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 20 client.17736 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff0c240 0x563d1ff0e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 4 client.17736 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.928+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d1fec1890 con 0x563d1ff0c240 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.929+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff0c240 0x563d1ff0e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.929+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff0c240 0x563d1ff0e660 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd683b1e5699b7e server_cookie=9dd6ce7d0bc4563a in_seq=0 out_seq=0 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.929+0000 7fb022ffd700 10 client.17736 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:35.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb022ffd700 1 -- 172.21.15.16:0/2389139846 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d1fec1890 con 0x563d1ff0c240 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb022ffd700 10 client.17736 handle_command_reply: tid=0 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 client.17736 shutdown 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 msgr2=0x563d1fefc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fefc200 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0095b0 tx=0x7fb03c004a50 comp rx=0 tx=0).stop 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 shutdown_connections 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1fefc200 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fefe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff0c240 0x563d1ff0e660 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2389139846 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1fefbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb022ffd700 0 client.17736 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 >> 172.21.15.16:0/2389139846 conn(0x563d1fe50290 msgr2=0x563d1feec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 shutdown_connections 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2389139846 wait complete. 2024-05-08T23:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.930+0000 7fb0561d5b80 20 client.17736 trim_cache size 0 max 16384 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.948+0000 7fb0561d5b80 1 Processor -- start 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.948+0000 7fb0561d5b80 1 -- start start 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1feecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fee7b10 con 0x563d1fef87a0 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fee7880 con 0x563d1fef7e80 2024-05-08T23:29:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1fef64d0 con 0x563d1fef2910 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60206/0 (socket says 172.21.15.16:60206) 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 -- 172.21.15.16:0/2230286761 learned_addr learned my addr 172.21.15.16:0/2230286761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb042ffd700 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1feecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 -- 172.21.15.16:0/2230286761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 msgr2=0x563d1fef23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 -- 172.21.15.16:0/2230286761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 msgr2=0x563d1feecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1feecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 -- 172.21.15.16:0/2230286761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d1fef2fc0 con 0x563d1fef87a0 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.949+0000 7fb043fff700 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff031f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009580 tx=0x7fb03c005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac36c5a1cf0b34d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb03a7fc700 1 -- 172.21.15.16:0/2230286761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00e070 con 0x563d1fef87a0 2024-05-08T23:29:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb03a7fc700 1 -- 172.21.15.16:0/2230286761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb03c004ac0 con 0x563d1fef87a0 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb03a7fc700 1 -- 172.21.15.16:0/2230286761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0050c0 con 0x563d1fef87a0 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2230286761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 msgr2=0x563d1ff031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff031f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009580 tx=0x7fb03c005890 comp rx=0 tx=0).stop 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2230286761 shutdown_connections 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff031f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef2910 0x563d1fef23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2230286761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef7e80 0x563d1feecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2230286761 >> 172.21.15.16:0/2230286761 conn(0x563d1fe50290 msgr2=0x563d1fefa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.950+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2230286761 shutdown_connections 2024-05-08T23:29:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2230286761 wait complete. 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 -- start start 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 Processor -- start 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef2910 0x563d1ff02410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef7e80 0x563d1ff02950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff62710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1ff62c50 con 0x563d1fef87a0 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1ff62d90 con 0x563d1fef2910 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.951+0000 7fb0561d5b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d1ff62ed0 con 0x563d1fef7e80 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb0561d5b80 10 client.0 _ll_register_callbacks cb 0x563d1fde7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:35.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff62710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb042ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef7e80 0x563d1ff02950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb042ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef7e80 0x563d1ff02950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:56694/0 (socket says 172.21.15.16:56694) 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb0437fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef2910 0x563d1ff02410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff62710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60222/0 (socket says 172.21.15.16:60222) 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb042ffd700 1 -- 172.21.15.16:0/2312128747 learned_addr learned my addr 172.21.15.16:0/2312128747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 -- 172.21.15.16:0/2312128747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef7e80 msgr2=0x563d1ff02950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef7e80 0x563d1ff02950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 -- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef2910 msgr2=0x563d1ff02410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef2910 0x563d1ff02410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 -- 172.21.15.16:0/2312128747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d1fc6fbe0 con 0x563d1fef87a0 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb043fff700 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff62710 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005670 tx=0x7fb03c00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c1d38b7947e0cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:35.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb022ffd700 1 -- 172.21.15.16:0/2312128747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00e040 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb022ffd700 1 -- 172.21.15.16:0/2312128747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb03c0078b0 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.952+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb034014c30 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb022ffd700 1 -- 172.21.15.16:0/2312128747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb03c004e80 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 10 client.17742 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d1fe085e0 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb022ffd700 1 -- 172.21.15.16:0/2312128747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb034014c30 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb022ffd700 1 -- 172.21.15.16:0/2312128747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb03c012a80 con 0x563d1fef87a0 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 10 client.17742 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 20 client.17742 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff64150 0x563d1ff66540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 4 client.17742 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.953+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563d1fe3af40 con 0x563d1ff64150 2024-05-08T23:29:35.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.954+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff64150 0x563d1ff66540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.954+0000 7fb022ffd700 10 client.17742 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:35.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.954+0000 7fb0437fe700 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff64150 0x563d1ff66540 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d8b0207ac7d04 server_cookie=134e5079a7d1c513 in_seq=0 out_seq=0 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb022ffd700 1 -- 172.21.15.16:0/2312128747 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563d1fe3af40 con 0x563d1ff64150 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb022ffd700 10 client.17742 handle_command_reply: tid=0 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 client.17742 shutdown 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 msgr2=0x563d1ff62710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff62710 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb03c005670 tx=0x7fb03c00b3c0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 shutdown_connections 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d1fef87a0 0x563d1ff62710 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563d1ff64150 0x563d1ff66540 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563d1fef2910 0x563d1ff02410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 --2- 172.21.15.16:0/2312128747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d1fef7e80 0x563d1ff02950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 >> 172.21.15.16:0/2312128747 conn(0x563d1fe50290 msgr2=0x563d1fef6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb022ffd700 0 client.17742 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 shutdown_connections 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 1 -- 172.21.15.16:0/2312128747 wait complete. 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.956+0000 7fb0561d5b80 20 client.17742 trim_cache size 0 max 16384 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "fc84f3fc-f8ec-453a-b5b7-e9093e1a5a48", 2024-05-08T23:29:35.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:29:35.962 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:29:35.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb0300a1290 msgr2=0x7fb0300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb0300a1290 0x7fb0300a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb044103250 tx=0x7fb034008040 comp rx=0 tx=0).stop 2024-05-08T23:29:35.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 msgr2=0x7fb044101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:35.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 0x7fb044101b20 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00b550 tx=0x7fb02c00b910 comp rx=0 tx=0).stop 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 shutdown_connections 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb0300a1290 0x7fb0300a3750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb044105f90 0x7fb0441025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0441000f0 0x7fb044101b20 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 --2- 172.21.15.16:0/659260200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb044100ab0 0x7fb044102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 >> 172.21.15.16:0/659260200 conn(0x7fb044076ba0 msgr2=0x7fb044069240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 shutdown_connections 2024-05-08T23:29:35.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:35.957+0000 7fb04a01b700 1 -- 172.21.15.16:0/659260200 wait complete. 2024-05-08T23:29:35.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:29:35.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:35 smithi016 ceph-mon[42541]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 15 op/s 2024-05-08T23:29:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 -- 172.21.15.16:0/1367430826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4068c90 msgr2=0x7f71b4105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 --2- 172.21.15.16:0/1367430826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4068c90 0x7f71b4105b60 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f71a80099c0 tx=0x7f71a8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 -- 172.21.15.16:0/1367430826 shutdown_connections 2024-05-08T23:29:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 --2- 172.21.15.16:0/1367430826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b4106290 0x7f71b410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 --2- 172.21.15.16:0/1367430826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b40682e0 0x7f71b40686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 --2- 172.21.15.16:0/1367430826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4068c90 0x7f71b4105b60 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 -- 172.21.15.16:0/1367430826 >> 172.21.15.16:0/1367430826 conn(0x7f71b4076f40 msgr2=0x7f71b4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 -- 172.21.15.16:0/1367430826 shutdown_connections 2024-05-08T23:29:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.112+0000 7f71b898e700 1 -- 172.21.15.16:0/1367430826 wait complete. 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 Processor -- start 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 -- start start 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b40682e0 0x7f71b4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 0x7f71b41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b419bb70 0x7f71b419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419c5f0 con 0x7f71b4106290 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419c760 con 0x7f71b419bb70 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b898e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b419c920 con 0x7f71b40682e0 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 0x7f71b41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 0x7f71b41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60232/0 (socket says 172.21.15.16:60232) 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b1d9b700 1 -- 172.21.15.16:0/2279595430 learned_addr learned my addr 172.21.15.16:0/2279595430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b419bb70 0x7f71b419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.113+0000 7f71b259c700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b40682e0 0x7f71b4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b1d9b700 1 -- 172.21.15.16:0/2279595430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b40682e0 msgr2=0x7f71b4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b40682e0 0x7f71b4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b1d9b700 1 -- 172.21.15.16:0/2279595430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b419bb70 msgr2=0x7f71b419bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b419bb70 0x7f71b419bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b1d9b700 1 -- 172.21.15.16:0/2279595430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a80095f0 con 0x7f71b4106290 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 0x7f71b41977d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009990 tx=0x7f71a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71a37fe700 1 -- 172.21.15.16:0/2279595430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a801d070 con 0x7f71b4106290 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71a37fe700 1 -- 172.21.15.16:0/2279595430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a800fc40 con 0x7f71b4106290 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71a37fe700 1 -- 172.21.15.16:0/2279595430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8022d80 con 0x7f71b4106290 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b41a3e20 con 0x7f71b4106290 2024-05-08T23:29:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.114+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b41a4190 con 0x7f71b4106290 2024-05-08T23:29:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.117+0000 7f71a37fe700 1 -- 172.21.15.16:0/2279595430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f71a8022500 con 0x7f71b4106290 2024-05-08T23:29:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.117+0000 7f71a37fe700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f719c0a1290 0x7f719c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.118+0000 7f71a37fe700 1 -- 172.21.15.16:0/2279595430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f71a80ce120 con 0x7f71b4106290 2024-05-08T23:29:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.118+0000 7f71b259c700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f719c0a1290 0x7f719c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.118+0000 7f71b259c700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f719c0a1290 0x7f719c0a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005950 tx=0x7f71a40058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.118+0000 7f71c4b48b80 1 Processor -- start 2024-05-08T23:29:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 -- start start 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c32d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c327760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c328100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c27a150 con 0x55c15c327ca0 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c277b10 con 0x55c15c32db10 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c273400 con 0x55c15c32d1f0 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71b2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c328100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71b2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c328100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60234/0 (socket says 172.21.15.16:60234) 2024-05-08T23:29:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71b2d9d700 1 -- 172.21.15.16:0/3517106662 learned_addr learned my addr 172.21.15.16:0/3517106662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c327760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.119+0000 7f71b259c700 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c32d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b2d9d700 1 -- 172.21.15.16:0/3517106662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 msgr2=0x55c15c32d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c32d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b2d9d700 1 -- 172.21.15.16:0/3517106662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 msgr2=0x55c15c327760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c327760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b2d9d700 1 -- 172.21.15.16:0/3517106662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15c0f43b0 con 0x55c15c327ca0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c327760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c328100 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00f410 tx=0x7f71ac00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f070b6032dbbeea2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71a0ff9700 1 -- 172.21.15.16:0/3517106662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac004bb0 con 0x55c15c327ca0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71a0ff9700 1 -- 172.21.15.16:0/3517106662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71ac004d10 con 0x55c15c327ca0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.120+0000 7f71a0ff9700 1 -- 172.21.15.16:0/3517106662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac005020 con 0x55c15c327ca0 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3517106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 msgr2=0x55c15c328100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c328100 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00f410 tx=0x7f71ac00f720 comp rx=0 tx=0).stop 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3517106662 shutdown_connections 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c328100 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c32d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3517106662 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c327760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3517106662 >> 172.21.15.16:0/3517106662 conn(0x55c15c281210 msgr2=0x55c15c31ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3517106662 shutdown_connections 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3517106662 wait complete. 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.121+0000 7f71c4b48b80 1 -- start start 2024-05-08T23:29:36.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 Processor -- start 2024-05-08T23:29:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c330ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32d1f0 0x55c15c331430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32db10 0x55c15c331970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c20f560 con 0x55c15c327ca0 2024-05-08T23:29:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c20f200 con 0x55c15c32d1f0 2024-05-08T23:29:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c0f66a0 con 0x55c15c32db10 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71c4b48b80 10 client.0 _ll_register_callbacks cb 0x55c15c218b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c330ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c330ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60246/0 (socket says 172.21.15.16:60246) 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 -- 172.21.15.16:0/3727039706 learned_addr learned my addr 172.21.15.16:0/3727039706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32db10 0x55c15c331970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32d1f0 0x55c15c331430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 -- 172.21.15.16:0/3727039706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32db10 msgr2=0x55c15c331970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32db10 0x55c15c331970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 -- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32d1f0 msgr2=0x55c15c331430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32d1f0 0x55c15c331430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b259c700 1 -- 172.21.15.16:0/3727039706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b41a4190 con 0x55c15c327ca0 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.122+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32d1f0 0x55c15c331430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71b259c700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c330ef0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011470 tx=0x7f71a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ee60c77a6f7303 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f7199ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f7199ffb700 1 -- 172.21.15.16:0/3727039706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a401f630 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f7199ffb700 1 -- 172.21.15.16:0/3727039706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f71a401f790 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c15c1cc270 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f7199ffb700 1 -- 172.21.15.16:0/3727039706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a401f990 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71c4b48b80 10 client.17754 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f71ac007e50 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f7199ffb700 1 -- 172.21.15.16:0/3727039706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c15c1cc270 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f7199ffb700 1 -- 172.21.15.16:0/3727039706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71a4028a90 con 0x55c15c327ca0 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71c4b48b80 10 client.17754 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71c4b48b80 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.123+0000 7f71c4b48b80 20 client.17754 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.124+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c341670 0x55c15c343a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.124+0000 7f71c4b48b80 4 client.17754 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.124+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c15c2f2810 con 0x55c15c341670 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.124+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c341670 0x55c15c343a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.124+0000 7f7199ffb700 10 client.17754 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.124+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c341670 0x55c15c343a90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674d6675f7cfb251 server_cookie=51409fcc0e74eee9 in_seq=0 out_seq=0 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f7199ffb700 1 -- 172.21.15.16:0/3727039706 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c15c2f2810 con 0x55c15c341670 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f7199ffb700 10 client.17754 handle_command_reply: tid=0 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 client.17754 shutdown 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 msgr2=0x55c15c330ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c330ef0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f71a4011470 tx=0x7f71a4004750 comp rx=0 tx=0).stop 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 shutdown_connections 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32db10 0x55c15c331970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c330ef0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c341670 0x55c15c343a90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3727039706 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32d1f0 0x55c15c331430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 >> 172.21.15.16:0/3727039706 conn(0x55c15c281210 msgr2=0x55c15c31d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f7199ffb700 0 client.17754 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 shutdown_connections 2024-05-08T23:29:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3727039706 wait complete. 2024-05-08T23:29:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.125+0000 7f71c4b48b80 20 client.17754 trim_cache size 0 max 16384 2024-05-08T23:29:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 Processor -- start 2024-05-08T23:29:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 -- start start 2024-05-08T23:29:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c32c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c31dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c331790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c31adc0 con 0x55c15c327ca0 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c32cc00 con 0x55c15c32db10 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c348ca0 con 0x55c15c32d1f0 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c32c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c32c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60262/0 (socket says 172.21.15.16:60262) 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 -- 172.21.15.16:0/1854117259 learned_addr learned my addr 172.21.15.16:0/1854117259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c331790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 -- 172.21.15.16:0/1854117259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 msgr2=0x55c15c31dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c31dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 -- 172.21.15.16:0/1854117259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 msgr2=0x55c15c331790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c331790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.144+0000 7f71b259c700 1 -- 172.21.15.16:0/1854117259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15c0a0b60 con 0x55c15c327ca0 2024-05-08T23:29:36.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71b259c700 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c32c6c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f71a4010010 tx=0x7f71a4011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85db1d04d13205a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:36.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71a17fa700 1 -- 172.21.15.16:0/1854117259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4011b10 con 0x55c15c327ca0 2024-05-08T23:29:36.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71a17fa700 1 -- 172.21.15.16:0/1854117259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71a40288b0 con 0x55c15c327ca0 2024-05-08T23:29:36.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71a17fa700 1 -- 172.21.15.16:0/1854117259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a401ec40 con 0x55c15c327ca0 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 -- 172.21.15.16:0/1854117259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 msgr2=0x55c15c32c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c32c6c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f71a4010010 tx=0x7f71a4011ed0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 -- 172.21.15.16:0/1854117259 shutdown_connections 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c331790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c32c6c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/1854117259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c31dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 -- 172.21.15.16:0/1854117259 >> 172.21.15.16:0/1854117259 conn(0x55c15c281210 msgr2=0x55c15c32bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 -- 172.21.15.16:0/1854117259 shutdown_connections 2024-05-08T23:29:36.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.145+0000 7f71c4b48b80 1 -- 172.21.15.16:0/1854117259 wait complete. 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 -- start start 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 Processor -- start 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c38fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c3902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c334940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c334ed0 con 0x55c15c327ca0 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c335040 con 0x55c15c32db10 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15c335200 con 0x55c15c32d1f0 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71c4b48b80 10 client.0 _ll_register_callbacks cb 0x55c15c218b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c38fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71b259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c38fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60272/0 (socket says 172.21.15.16:60272) 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71b259c700 1 -- 172.21.15.16:0/3355246343 learned_addr learned my addr 172.21.15.16:0/3355246343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71b2d9d700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c334940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.146+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c3902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b259c700 1 -- 172.21.15.16:0/3355246343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 msgr2=0x55c15c3902c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b259c700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c3902c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b259c700 1 -- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 msgr2=0x55c15c334940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b259c700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c334940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b259c700 1 -- 172.21.15.16:0/3355246343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac007e50 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c3902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71b259c700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c38fd80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b480 tx=0x7f71a4028a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672d1e69e99e865 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f7199ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f7199ffb700 1 -- 172.21.15.16:0/3355246343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4028620 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f7199ffb700 1 -- 172.21.15.16:0/3355246343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f71a401e8e0 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71ac004d10 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f7199ffb700 1 -- 172.21.15.16:0/3355246343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a401eaa0 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71c4b48b80 10 client.17760 fetch_fsmap learned FSMap version 25 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c15c239560 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f7199ffb700 1 -- 172.21.15.16:0/3355246343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71ac004d10 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.147+0000 7f7199ffb700 1 -- 172.21.15.16:0/3355246343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71a4004710 con 0x55c15c327ca0 2024-05-08T23:29:36.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 10 client.17760 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 20 client.17760 populate_metadata read hostname 'smithi016' 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c398900 0x55c15c39acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 4 client.17760 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c15c26bec0 con 0x55c15c398900 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c398900 0x55c15c39acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71b1d9b700 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c398900 0x55c15c39acf0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa69b26478dde56 server_cookie=84cced91c0d07452 in_seq=0 out_seq=0 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f7199ffb700 10 client.17760 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f7199ffb700 1 -- 172.21.15.16:0/3355246343 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c15c26bec0 con 0x55c15c398900 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f7199ffb700 10 client.17760 handle_command_reply: tid=0 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.148+0000 7f71c4b48b80 1 client.17760 shutdown 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 msgr2=0x55c15c38fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c38fd80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f71a400b480 tx=0x7f71a4028a20 comp rx=0 tx=0).stop 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 shutdown_connections 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55c15c398900 0x55c15c39acf0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c15c32db10 0x55c15c334940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c15c327ca0 0x55c15c38fd80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f7199ffb700 0 client.17760 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 --2- 172.21.15.16:0/3355246343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c15c32d1f0 0x55c15c3902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 >> 172.21.15.16:0/3355246343 conn(0x55c15c281210 msgr2=0x55c15c322340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 shutdown_connections 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 1 -- 172.21.15.16:0/3355246343 wait complete. 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71c4b48b80 20 client.17760 trim_cache size 0 max 16384 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:29:36.155 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: "fc84f3fc-f8ec-453a-b5b7-e9093e1a5a48": { 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "fc84f3fc-f8ec-453a-b5b7-e9093e1a5a48", 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:29:36.156 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f719c0a1290 msgr2=0x7f719c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f719c0a1290 0x7f719c0a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005950 tx=0x7f71a40058e0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 msgr2=0x7f71b41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 0x7f71b41977d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009990 tx=0x7f71a8004730 comp rx=0 tx=0).stop 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 shutdown_connections 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f719c0a1290 0x7f719c0a3750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71b419bb70 0x7f71b419bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b40682e0 0x7f71b4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 --2- 172.21.15.16:0/2279595430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71b4106290 0x7f71b41977d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:29:36.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.149+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 >> 172.21.15.16:0/2279595430 conn(0x7f71b4076f40 msgr2=0x7f71b40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:29:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.150+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 shutdown_connections 2024-05-08T23:29:36.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:29:36.150+0000 7f71b898e700 1 -- 172.21.15.16:0/2279595430 wait complete. 2024-05-08T23:29:36.163 INFO:tasks.cephfs.filesystem:scrub status for tag:fc84f3fc-f8ec-453a-b5b7-e9093e1a5a48 - {'path': '/', 'tag': 'fc84f3fc-f8ec-453a-b5b7-e9093e1a5a48', 'options': 'recursive,force'} 2024-05-08T23:29:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:35 smithi189 ceph-mon[46919]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 15 op/s 2024-05-08T23:29:36.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:35 smithi155 ceph-mon[47105]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 68 KiB/s wr, 15 op/s 2024-05-08T23:29:37.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:36 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:29:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:36 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:36 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:29:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:36 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:36 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:29:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:36 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:36 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:29:37.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:36 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:37.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:36 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:29:37.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:36 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:37.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:36 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:29:37.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:36 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:29:38.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:37 smithi016 ceph-mon[42541]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:38.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:37 smithi189 ceph-mon[46919]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:38.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:37 smithi155 ceph-mon[47105]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:40.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:39 smithi016 ceph-mon[42541]: pgmap v313: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:40.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:39 smithi189 ceph-mon[46919]: pgmap v313: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:39 smithi155 ceph-mon[47105]: pgmap v313: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 10 op/s 2024-05-08T23:29:41.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:40 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:29:41.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:40 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:29:41.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:40 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:29:42.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:41 smithi016 ceph-mon[42541]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-05-08T23:29:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:41 smithi189 ceph-mon[46919]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-05-08T23:29:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:41 smithi155 ceph-mon[47105]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-05-08T23:29:44.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:43 smithi016 ceph-mon[42541]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:43 smithi189 ceph-mon[46919]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:43 smithi155 ceph-mon[47105]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 42 KiB/s wr, 10 op/s 2024-05-08T23:29:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:45 smithi016 ceph-mon[42541]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-05-08T23:29:46.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:45 smithi189 ceph-mon[46919]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-05-08T23:29:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:45 smithi155 ceph-mon[47105]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 15 op/s 2024-05-08T23:29:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:47 smithi016 ceph-mon[42541]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:29:48.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:47 smithi189 ceph-mon[46919]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:29:48.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:47 smithi155 ceph-mon[47105]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:29:50.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:49 smithi016 ceph-mon[42541]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:29:50.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:49 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:29:50.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:49 smithi189 ceph-mon[46919]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:29:50.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:49 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:29:50.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:49 smithi155 ceph-mon[47105]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-05-08T23:29:50.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:49 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:29:51.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:29:51.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:29:51.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 16 op/s 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: Adjusting osd_memory_target on smithi189 to 182.5M 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: Unable to set osd_memory_target on smithi016 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:29:52.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:52 smithi016 ceph-mon[42541]: Unable to set osd_memory_target on smithi189 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T23:29:52.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 16 op/s 2024-05-08T23:29:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-05-08T23:29:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:29:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-05-08T23:29:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: Adjusting osd_memory_target on smithi189 to 182.5M 2024-05-08T23:29:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: Unable to set osd_memory_target on smithi016 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:29:52.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:52 smithi189 ceph-mon[46919]: Unable to set osd_memory_target on smithi189 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 68 KiB/s wr, 16 op/s 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi155 to 182.5M 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: Unable to set osd_memory_target on smithi155 to 191375564: error parsing value: Value '191375564' is below minimum 939524096 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi016 to 183.5M 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: Adjusting osd_memory_target on smithi189 to 182.5M 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: Unable to set osd_memory_target on smithi016 to 192482304: error parsing value: Value '192482304' is below minimum 939524096 2024-05-08T23:29:52.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:52 smithi155 ceph-mon[47105]: Unable to set osd_memory_target on smithi189 to 191374848: error parsing value: Value '191374848' is below minimum 939524096 2024-05-08T23:29:54.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:54 smithi016 ceph-mon[42541]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 10 op/s 2024-05-08T23:29:54.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:54 smithi189 ceph-mon[46919]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 10 op/s 2024-05-08T23:29:54.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:54 smithi155 ceph-mon[47105]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 10 op/s 2024-05-08T23:29:56.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:56 smithi016 ceph-mon[42541]: pgmap v321: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-05-08T23:29:56.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:56 smithi189 ceph-mon[46919]: pgmap v321: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-05-08T23:29:56.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:56 smithi155 ceph-mon[47105]: pgmap v321: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-05-08T23:29:58.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:29:58 smithi016 ceph-mon[42541]: pgmap v322: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-05-08T23:29:58.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:29:58 smithi189 ceph-mon[46919]: pgmap v322: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-05-08T23:29:58.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:29:58 smithi155 ceph-mon[47105]: pgmap v322: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-05-08T23:29:59.261 INFO:tasks.workunit.client.0.smithi016.stdout:437/600: open fd = 3 2024-05-08T23:29:59.261 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.261 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.261 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.261 INFO:tasks.workunit.client.0.smithi016.stdout:438/600: open fd = 3 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:439/600: open fd = 3 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:440/600: open fd = 3 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:441/600: open fd = 3 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.262 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:442/600: open fd = 3 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:443/600: open fd = 3 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:444/600: open fd = 3 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.263 INFO:tasks.workunit.client.0.smithi016.stdout:445/600: open fd = 3 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:446/600: open fd = 3 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:447/600: open fd = 3 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:448/600: open fd = 3 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:449/600: open fd = 3 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:450/600: open fd = 3 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:451/600: open fd = 3 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:452/600: open fd = 3 2024-05-08T23:29:59.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:453/600: open fd = 3 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:454/600: open fd = 3 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:455/600: open fd = 3 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:456/600: open fd = 3 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:457/600: open fd = 3 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:458/600: open fd = 3 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:459/600: open fd = 3 2024-05-08T23:29:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:460/600: open fd = 3 2024-05-08T23:29:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:461/600: open fd = 3 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:462/600: open fd = 3 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:463/600: open fd = 3 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:464/600: open fd = 3 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:465/600: open fd = 3 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:466/600: open fd = 3 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:467/600: open fd = 3 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:468/600: open fd = 3 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:469/600: open fd = 3 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:470/600: open fd = 3 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:471/600: open fd = 3 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:472/600: open fd = 3 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.273 INFO:tasks.workunit.client.0.smithi016.stdout:473/600: open fd = 3 2024-05-08T23:29:59.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.274 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.274 INFO:tasks.workunit.client.0.smithi016.stdout:474/600: open fd = 3 2024-05-08T23:29:59.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:475/600: open fd = 3 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:476/600: open fd = 3 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:477/600: open fd = 3 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:478/600: open fd = 3 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:479/600: open fd = 3 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:480/600: open fd = 3 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:481/600: open fd = 3 2024-05-08T23:29:59.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:482/600: open fd = 3 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:483/600: open fd = 3 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:484/600: open fd = 3 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.277 INFO:tasks.workunit.client.0.smithi016.stdout:485/600: open fd = 3 2024-05-08T23:29:59.278 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:486/600: open fd = 3 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:487/600: open fd = 3 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:488/600: open fd = 3 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.279 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:489/600: open fd = 3 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:490/600: open fd = 3 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:491/600: open fd = 3 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.280 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:492/600: open fd = 3 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:493/600: open fd = 3 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:494/600: open fd = 3 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.281 INFO:tasks.workunit.client.0.smithi016.stdout:495/600: open fd = 3 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:496/600: open fd = 3 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:497/600: open fd = 3 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:498/600: open fd = 3 2024-05-08T23:29:59.282 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.283 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:29:59.283 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:30:00.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:00 smithi016 ceph-mon[42541]: pgmap v323: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-05-08T23:30:00.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:00 smithi016 ceph-mon[42541]: overall HEALTH_OK 2024-05-08T23:30:00.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:00 smithi189 ceph-mon[46919]: pgmap v323: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-05-08T23:30:00.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:00 smithi189 ceph-mon[46919]: overall HEALTH_OK 2024-05-08T23:30:00.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:00 smithi155 ceph-mon[47105]: pgmap v323: 129 pgs: 129 active+clean; 18 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-05-08T23:30:00.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:00 smithi155 ceph-mon[47105]: overall HEALTH_OK 2024-05-08T23:30:02.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:02 smithi016 ceph-mon[42541]: pgmap v324: 129 pgs: 129 active+clean; 18 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-05-08T23:30:02.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:02 smithi189 ceph-mon[46919]: pgmap v324: 129 pgs: 129 active+clean; 18 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-05-08T23:30:02.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:02 smithi155 ceph-mon[47105]: pgmap v324: 129 pgs: 129 active+clean; 18 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-05-08T23:30:03.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:03 smithi016 ceph-mon[42541]: pgmap v325: 129 pgs: 129 active+clean; 18 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 11 op/s 2024-05-08T23:30:03.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:03 smithi189 ceph-mon[46919]: pgmap v325: 129 pgs: 129 active+clean; 18 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 11 op/s 2024-05-08T23:30:03.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:03 smithi155 ceph-mon[47105]: pgmap v325: 129 pgs: 129 active+clean; 18 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 11 op/s 2024-05-08T23:30:05.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:05 smithi155 ceph-mon[47105]: pgmap v326: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:05.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:05 smithi016 ceph-mon[42541]: pgmap v326: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:05.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:05 smithi189 ceph-mon[46919]: pgmap v326: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:06.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:30:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 -- 172.21.15.16:0/2303110806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c105f90 msgr2=0x7f874c106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 --2- 172.21.15.16:0/2303110806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c105f90 0x7f874c106370 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f873c0099c0 tx=0x7f873c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 -- 172.21.15.16:0/2303110806 shutdown_connections 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 --2- 172.21.15.16:0/2303110806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c100ab0 0x7f874c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 --2- 172.21.15.16:0/2303110806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c105f90 0x7f874c106370 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 --2- 172.21.15.16:0/2303110806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c1000f0 0x7f874c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 -- 172.21.15.16:0/2303110806 >> 172.21.15.16:0/2303110806 conn(0x7f874c076ba0 msgr2=0x7f874c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 -- 172.21.15.16:0/2303110806 shutdown_connections 2024-05-08T23:30:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.318+0000 7f8750c8d700 1 -- 172.21.15.16:0/2303110806 wait complete. 2024-05-08T23:30:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.319+0000 7f8750c8d700 1 Processor -- start 2024-05-08T23:30:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.319+0000 7f8750c8d700 1 -- start start 2024-05-08T23:30:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8750c8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 0x7f874c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8750c8d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c100ab0 0x7f874c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8750c8d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c105f90 0x7f874c1a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8750c8d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c198e40 con 0x7f874c1000f0 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8750c8d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a3670 con 0x7f874c105f90 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8750c8d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a37b0 con 0x7f874c100ab0 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 0x7f874c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 0x7f874c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56588/0 (socket says 172.21.15.16:56588) 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 -- 172.21.15.16:0/182237043 learned_addr learned my addr 172.21.15.16:0/182237043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f8749d9b700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c100ab0 0x7f874c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874ad9d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c105f90 0x7f874c1a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 -- 172.21.15.16:0/182237043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c100ab0 msgr2=0x7f874c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c100ab0 0x7f874c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 -- 172.21.15.16:0/182237043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c105f90 msgr2=0x7f874c1a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c105f90 0x7f874c1a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 -- 172.21.15.16:0/182237043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0095f0 con 0x7f874c1000f0 2024-05-08T23:30:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f874a59c700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 0x7f874c1980b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f873c00f690 tx=0x7f873c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.320+0000 7f873b7fe700 1 -- 172.21.15.16:0/182237043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c01c070 con 0x7f874c1000f0 2024-05-08T23:30:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.321+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c1a39b0 con 0x7f874c1000f0 2024-05-08T23:30:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.321+0000 7f873b7fe700 1 -- 172.21.15.16:0/182237043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f873c00fe20 con 0x7f874c1000f0 2024-05-08T23:30:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.321+0000 7f873b7fe700 1 -- 172.21.15.16:0/182237043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c004970 con 0x7f874c1000f0 2024-05-08T23:30:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.321+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a3e00 con 0x7f874c1000f0 2024-05-08T23:30:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.323+0000 7f873b7fe700 1 -- 172.21.15.16:0/182237043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f873c020070 con 0x7f874c1000f0 2024-05-08T23:30:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.324+0000 7f873b7fe700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87340a1290 0x7f87340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.324+0000 7f873b7fe700 1 -- 172.21.15.16:0/182237043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f873c0cdbc0 con 0x7f874c1000f0 2024-05-08T23:30:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.324+0000 7f8749d9b700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87340a1290 0x7f87340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f8749d9b700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87340a1290 0x7f87340a3750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f874c0699a0 tx=0x7f874000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 Processor -- start 2024-05-08T23:30:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 -- start start 2024-05-08T23:30:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dfea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dfeb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898df3d150 con 0x55898dff0b30 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898df3ab10 con 0x55898dfeaca0 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f875ce47b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898df36400 con 0x55898dff0210 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f8749d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dfea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f8749d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dfea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39260/0 (socket says 172.21.15.16:39260) 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.325+0000 7f8749d9b700 1 -- 172.21.15.16:0/2882941078 learned_addr learned my addr 172.21.15.16:0/2882941078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f874a59c700 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f874ad9d700 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dfeb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8749d9b700 1 -- 172.21.15.16:0/2882941078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 msgr2=0x55898dff05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8749d9b700 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8749d9b700 1 -- 172.21.15.16:0/2882941078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 msgr2=0x55898dfeb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8749d9b700 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dfeb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8749d9b700 1 -- 172.21.15.16:0/2882941078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55898ddb73b0 con 0x55898dff0b30 2024-05-08T23:30:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f874a59c700 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8749d9b700 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dfea760 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8740010f70 tx=0x7f8740013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7dbe28eaa8a2c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.326+0000 7f8738ff9700 1 -- 172.21.15.16:0/2882941078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740011db0 con 0x55898dff0b30 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f8738ff9700 1 -- 172.21.15.16:0/2882941078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8740015070 con 0x55898dff0b30 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f8738ff9700 1 -- 172.21.15.16:0/2882941078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874001c610 con 0x55898dff0b30 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 -- 172.21.15.16:0/2882941078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 msgr2=0x55898dfea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dfea760 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8740010f70 tx=0x7f8740013000 comp rx=0 tx=0).stop 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 -- 172.21.15.16:0/2882941078 shutdown_connections 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dfeb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2882941078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dfea760 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 -- 172.21.15.16:0/2882941078 >> 172.21.15.16:0/2882941078 conn(0x55898df44210 msgr2=0x55898dfe0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 -- 172.21.15.16:0/2882941078 shutdown_connections 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.327+0000 7f875ce47b80 1 -- 172.21.15.16:0/2882941078 wait complete. 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 -- start start 2024-05-08T23:30:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 Processor -- start 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 0x55898dff4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0b30 0x55898dff49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898ded2560 con 0x55898dff0210 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898ded2200 con 0x55898dfeaca0 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898ddb96a0 con 0x55898dff0b30 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f875ce47b80 10 client.0 _ll_register_callbacks cb 0x55898dedbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.328+0000 7f8749d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 0x55898dff4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 0x55898dff4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39270/0 (socket says 172.21.15.16:39270) 2024-05-08T23:30:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 -- 172.21.15.16:0/1184375117 learned_addr learned my addr 172.21.15.16:0/1184375117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f874ad9d700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0b30 0x55898dff49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f874a59c700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 -- 172.21.15.16:0/1184375117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0b30 msgr2=0x55898dff49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0b30 0x55898dff49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 -- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 msgr2=0x55898dff3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 -- 172.21.15.16:0/1184375117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a3e00 con 0x55898dff0210 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8749d9b700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 0x55898dff4470 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f874000bfa0 tx=0x7f8740012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ae2a9f3a2a6512 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8731ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8731ffb700 1 -- 172.21.15.16:0/1184375117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874002b070 con 0x55898dff0210 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8731ffb700 1 -- 172.21.15.16:0/1184375117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f8740017070 con 0x55898dff0210 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55898de8f270 con 0x55898dff0210 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f8731ffb700 1 -- 172.21.15.16:0/1184375117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874001e420 con 0x55898dff0210 2024-05-08T23:30:06.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f875ce47b80 10 client.17772 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.329+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f8740010c20 con 0x55898dff0210 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f8731ffb700 1 -- 172.21.15.16:0/1184375117 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55898de8f270 con 0x55898dff0210 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f8731ffb700 1 -- 172.21.15.16:0/1184375117 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87400047a0 con 0x55898dff0210 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f875ce47b80 10 client.17772 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f875ce47b80 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f875ce47b80 20 client.17772 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f875ce47b80 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e0045b0 0x55898e0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f875ce47b80 4 client.17772 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55898dfb5810 con 0x55898e0045b0 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.330+0000 7f874a59c700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e0045b0 0x55898e0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.331+0000 7f874a59c700 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e0045b0 0x55898e0069d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e068a200e928cdf7 server_cookie=db30c1c6cf4449d9 in_seq=0 out_seq=0 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.331+0000 7f8731ffb700 10 client.17772 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.331+0000 7f8731ffb700 1 -- 172.21.15.16:0/1184375117 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55898dfb5810 con 0x55898e0045b0 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.331+0000 7f8731ffb700 10 client.17772 handle_command_reply: tid=0 2024-05-08T23:30:06.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.331+0000 7f875ce47b80 1 client.17772 shutdown 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 msgr2=0x55898dff4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 0x55898dff4470 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f874000bfa0 tx=0x7f8740012bc0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 shutdown_connections 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0b30 0x55898dff49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e0045b0 0x55898e0069d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 --2- 172.21.15.16:0/1184375117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0210 0x55898dff4470 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 >> 172.21.15.16:0/1184375117 conn(0x55898df44210 msgr2=0x55898dfe05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f8731ffb700 0 client.17772 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:06.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 shutdown_connections 2024-05-08T23:30:06.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 1 -- 172.21.15.16:0/1184375117 wait complete. 2024-05-08T23:30:06.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.332+0000 7f875ce47b80 20 client.17772 trim_cache size 0 max 16384 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 Processor -- start 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 -- start start 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 0x55898dfe0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dfee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dff0b30 0x55898dfee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898de8f270 con 0x55898dfeaca0 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898e001170 con 0x55898dff0b30 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f875ce47b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898dff6a30 con 0x55898dff0210 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f874a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 0x55898dfe0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f874a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 0x55898dfe0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39284/0 (socket says 172.21.15.16:39284) 2024-05-08T23:30:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f874a59c700 1 -- 172.21.15.16:0/3417350700 learned_addr learned my addr 172.21.15.16:0/3417350700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f8749d9b700 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dfee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.352+0000 7f874ad9d700 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dff0b30 0x55898dfee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f874a59c700 1 -- 172.21.15.16:0/3417350700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 msgr2=0x55898dfee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f874a59c700 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dfee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f874a59c700 1 -- 172.21.15.16:0/3417350700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dff0b30 msgr2=0x55898dfee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f874a59c700 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dff0b30 0x55898dfee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f874a59c700 1 -- 172.21.15.16:0/3417350700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55898dd63b60 con 0x55898dfeaca0 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f874a59c700 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 0x55898dfe0d20 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f873c0cb520 tx=0x7f873c0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fe9897b1aefbf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f87397fa700 1 -- 172.21.15.16:0/3417350700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c09e7a0 con 0x55898dfeaca0 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f87397fa700 1 -- 172.21.15.16:0/3417350700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f873c0a2d60 con 0x55898dfeaca0 2024-05-08T23:30:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.353+0000 7f87397fa700 1 -- 172.21.15.16:0/3417350700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c099e60 con 0x55898dfeaca0 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 -- 172.21.15.16:0/3417350700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 msgr2=0x55898dfe0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 0x55898dfe0d20 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f873c0cb520 tx=0x7f873c0ba040 comp rx=0 tx=0).stop 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 -- 172.21.15.16:0/3417350700 shutdown_connections 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dff0b30 0x55898dfee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dfeaca0 0x55898dfe0d20 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 --2- 172.21.15.16:0/3417350700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dfee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 -- 172.21.15.16:0/3417350700 >> 172.21.15.16:0/3417350700 conn(0x55898df44210 msgr2=0x55898e00b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 -- 172.21.15.16:0/3417350700 shutdown_connections 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 -- 172.21.15.16:0/3417350700 wait complete. 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 -- start start 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.354+0000 7f875ce47b80 1 Processor -- start 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dffa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898e059c30 con 0x55898dff0b30 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898e059da0 con 0x55898dfeaca0 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55898e059f60 con 0x55898dff0210 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 10 client.0 _ll_register_callbacks cb 0x55898dedbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dffa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dffa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39296/0 (socket says 172.21.15.16:39296) 2024-05-08T23:30:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 -- 172.21.15.16:0/2991106500 learned_addr learned my addr 172.21.15.16:0/2991106500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874a59c700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f8749d9b700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 -- 172.21.15.16:0/2991106500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 msgr2=0x55898dff9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 -- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 msgr2=0x55898dff95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 -- 172.21.15.16:0/2991106500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a39b0 con 0x55898dff0b30 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874a59c700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:06.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f874ad9d700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dffa030 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f8744007db0 tx=0x7f8744007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa07a9d0121f41b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f8731ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f8731ffb700 1 -- 172.21.15.16:0/2991106500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400e040 con 0x55898dff0b30 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8740015070 con 0x55898dff0b30 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.355+0000 7f8731ffb700 1 -- 172.21.15.16:0/2991106500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f8744017a20 con 0x55898dff0b30 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f8731ffb700 1 -- 172.21.15.16:0/2991106500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8744017be0 con 0x55898dff0b30 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 10 client.17778 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55898defc560 con 0x55898dff0b30 2024-05-08T23:30:06.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f8731ffb700 1 -- 172.21.15.16:0/2991106500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8740015070 con 0x55898dff0b30 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f8731ffb700 1 -- 172.21.15.16:0/2991106500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8744018420 con 0x55898dff0b30 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 10 client.17778 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 20 client.17778 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e05b7b0 0x55898e05dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 4 client.17778 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55898df2eec0 con 0x55898e05b7b0 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.356+0000 7f874a59c700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e05b7b0 0x55898e05dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f874a59c700 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e05b7b0 0x55898e05dbd0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994427e9ae22b066 server_cookie=3cfc6ef6cdc13a41 in_seq=0 out_seq=0 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f8731ffb700 10 client.17778 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f8731ffb700 1 -- 172.21.15.16:0/2991106500 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55898df2eec0 con 0x55898e05b7b0 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f8731ffb700 10 client.17778 handle_command_reply: tid=0 2024-05-08T23:30:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 client.17778 shutdown 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 msgr2=0x55898dffa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dffa030 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f8744007db0 tx=0x7f8744007e90 comp rx=0 tx=0).stop 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 shutdown_connections 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55898dff0b30 0x55898dffa030 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55898e05b7b0 0x55898e05dbd0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55898dfeaca0 0x55898dff95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 --2- 172.21.15.16:0/2991106500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55898dff0210 0x55898dff9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f8731ffb700 0 client.17778 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 >> 172.21.15.16:0/2991106500 conn(0x55898df44210 msgr2=0x55898dfeeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 shutdown_connections 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.357+0000 7f875ce47b80 1 -- 172.21.15.16:0/2991106500 wait complete. 2024-05-08T23:30:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f875ce47b80 20 client.17778 trim_cache size 0 max 16384 2024-05-08T23:30:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:30:06.365 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:30:06.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:30:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87340a1290 msgr2=0x7f87340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87340a1290 0x7f87340a3750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f874c0699a0 tx=0x7f874000b410 comp rx=0 tx=0).stop 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 msgr2=0x7f874c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 0x7f874c1980b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f873c00f690 tx=0x7f873c00f6c0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 shutdown_connections 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87340a1290 0x7f87340a3750 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f874c105f90 0x7f874c1a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f874c1000f0 0x7f874c1980b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 --2- 172.21.15.16:0/182237043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c100ab0 0x7f874c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 >> 172.21.15.16:0/182237043 conn(0x7f874c076ba0 msgr2=0x7f874c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 shutdown_connections 2024-05-08T23:30:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.358+0000 7f8750c8d700 1 -- 172.21.15.16:0/182237043 wait complete. 2024-05-08T23:30:06.371 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:30:06.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:06.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:06 smithi155 ceph-mon[47105]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:30:06.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 -- 172.21.15.16:0/832250653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 msgr2=0x7ff98810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/832250653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff98810cbe0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff98000b0a0 tx=0x7ff98000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 -- 172.21.15.16:0/832250653 shutdown_connections 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/832250653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff98810cbe0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/832250653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 0x7ff988100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/832250653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9881008f0 0x7ff988104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 -- 172.21.15.16:0/832250653 >> 172.21.15.16:0/832250653 conn(0x7ff9880fb7e0 msgr2=0x7ff9880fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 -- 172.21.15.16:0/832250653 shutdown_connections 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.508+0000 7ff98e8f3700 1 -- 172.21.15.16:0/832250653 wait complete. 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 Processor -- start 2024-05-08T23:30:06.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 -- start start 2024-05-08T23:30:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 0x7ff9881972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff9881977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff98819bb90 0x7ff98819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98819c610 con 0x7ff988105060 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98819c780 con 0x7ff9880fff40 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98e8f3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98819c940 con 0x7ff98819bb90 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff9881977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff9881977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39322/0 (socket says 172.21.15.16:39322) 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff97ffff700 1 -- 172.21.15.16:0/2666692163 learned_addr learned my addr 172.21.15.16:0/2666692163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.509+0000 7ff98ce90700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff98819bb90 0x7ff98819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff987fff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 0x7ff9881972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff97ffff700 1 -- 172.21.15.16:0/2666692163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff98819bb90 msgr2=0x7ff98819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff97ffff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff98819bb90 0x7ff98819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff97ffff700 1 -- 172.21.15.16:0/2666692163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 msgr2=0x7ff9881972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff97ffff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 0x7ff9881972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff97ffff700 1 -- 172.21.15.16:0/2666692163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009d00 con 0x7ff988105060 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff987fff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 0x7ff9881972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff97ffff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff9881977f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d950 tx=0x7ff97000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970004220 con 0x7ff988105060 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff970007620 con 0x7ff988105060 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9881a3ea0 con 0x7ff988105060 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970005530 con 0x7ff988105060 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.510+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9881a4320 con 0x7ff988105060 2024-05-08T23:30:06.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.511+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff988109340 con 0x7ff988105060 2024-05-08T23:30:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.512+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff970004750 con 0x7ff988105060 2024-05-08T23:30:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.513+0000 7ff985ffb700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9740a1290 0x7ff9740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.513+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7ff9700cceb0 con 0x7ff988105060 2024-05-08T23:30:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.513+0000 7ff987fff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9740a1290 0x7ff9740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.514+0000 7ff987fff700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9740a1290 0x7ff9740a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff978005fd0 tx=0x7ff978005dc0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.518+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff970095eb0 con 0x7ff988105060 2024-05-08T23:30:06.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:06 smithi016 ceph-mon[42541]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:30:06.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:06 smithi189 ceph-mon[46919]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T23:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.785+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff988067c10 con 0x7ff988105060 2024-05-08T23:30:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.785+0000 7ff985ffb700 1 -- 172.21.15.16:0/2666692163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7ff970016020 con 0x7ff988105060 2024-05-08T23:30:06.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:30:06.785 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9740a1290 msgr2=0x7ff9740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9740a1290 0x7ff9740a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff978005fd0 tx=0x7ff978005dc0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 msgr2=0x7ff9881977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff9881977f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff97000d950 tx=0x7ff97000dc60 comp rx=0 tx=0).stop 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 shutdown_connections 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff9740a1290 0x7ff9740a3750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff98819bb90 0x7ff98819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9880fff40 0x7ff9881972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 --2- 172.21.15.16:0/2666692163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff988105060 0x7ff9881977f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 >> 172.21.15.16:0/2666692163 conn(0x7ff9880fb7e0 msgr2=0x7ff988068330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 shutdown_connections 2024-05-08T23:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.788+0000 7ff98e8f3700 1 -- 172.21.15.16:0/2666692163 wait complete. 2024-05-08T23:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:30:06.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 -- 172.21.15.16:0/1115966078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068970 msgr2=0x7f47e810d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 --2- 172.21.15.16:0/1115966078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068970 0x7f47e810d400 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009990 tx=0x7f47d0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 -- 172.21.15.16:0/1115966078 shutdown_connections 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 --2- 172.21.15.16:0/1115966078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47e810d940 0x7f47e810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 --2- 172.21.15.16:0/1115966078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47e8068050 0x7f47e8068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 --2- 172.21.15.16:0/1115966078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068970 0x7f47e810d400 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 -- 172.21.15.16:0/1115966078 >> 172.21.15.16:0/1115966078 conn(0x7f47e8076f40 msgr2=0x7f47e8077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 -- 172.21.15.16:0/1115966078 shutdown_connections 2024-05-08T23:30:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.942+0000 7f47eec38700 1 -- 172.21.15.16:0/1115966078 wait complete. 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 Processor -- start 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 -- start start 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 0x7f47e81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47e8068970 0x7f47e81999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47e810d940 0x7f47e819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e819e3c0 con 0x7f47e8068050 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e819e530 con 0x7f47e8068970 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47eec38700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e819e6f0 con 0x7f47e810d940 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 0x7f47e81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 0x7f47e81994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39346/0 (socket says 172.21.15.16:39346) 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 -- 172.21.15.16:0/3038060016 learned_addr learned my addr 172.21.15.16:0/3038060016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47e7fff700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47e8068970 0x7f47e81999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47e810d940 0x7f47e819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 -- 172.21.15.16:0/3038060016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47e810d940 msgr2=0x7f47e819dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47e810d940 0x7f47e819dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 -- 172.21.15.16:0/3038060016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47e8068970 msgr2=0x7f47e81999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47e8068970 0x7f47e81999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.943+0000 7f47ec9d4700 1 -- 172.21.15.16:0/3038060016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00095f0 con 0x7f47e8068050 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.944+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 0x7f47e81994a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c9b0 tx=0x7f47d800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.944+0000 7f47e5ffb700 1 -- 172.21.15.16:0/3038060016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8007b00 con 0x7f47e8068050 2024-05-08T23:30:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.944+0000 7f47e5ffb700 1 -- 172.21.15.16:0/3038060016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47d8007c60 con 0x7f47e8068050 2024-05-08T23:30:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.944+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e819e9d0 con 0x7f47e8068050 2024-05-08T23:30:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.944+0000 7f47e5ffb700 1 -- 172.21.15.16:0/3038060016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8018660 con 0x7f47e8068050 2024-05-08T23:30:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.944+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e81a6210 con 0x7f47e8068050 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.946+0000 7f47e5ffb700 1 -- 172.21.15.16:0/3038060016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f47d800f450 con 0x7f47e8068050 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.947+0000 7f47e5ffb700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47d40a1290 0x7f47d40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.947+0000 7f47e5ffb700 1 -- 172.21.15.16:0/3038060016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f47d80cd1a0 con 0x7f47e8068050 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.947+0000 7f47e7fff700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47d40a1290 0x7f47d40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.947+0000 7f47e7fff700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47d40a1290 0x7f47d40a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b5c0 tx=0x7f47d0005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 Processor -- start 2024-05-08T23:30:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 -- start start 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc355b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc3555350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3555cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc34ac150 con 0x557dc355b720 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc34a9b10 con 0x557dc355ae00 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc34a5400 con 0x557dc3555890 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc3555350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc3555350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39360/0 (socket says 172.21.15.16:39360) 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47e7fff700 1 -- 172.21.15.16:0/4276008030 learned_addr learned my addr 172.21.15.16:0/4276008030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3555cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.948+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc355b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47e7fff700 1 -- 172.21.15.16:0/4276008030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 msgr2=0x557dc3555cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47e7fff700 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3555cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47e7fff700 1 -- 172.21.15.16:0/4276008030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 msgr2=0x557dc355b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47e7fff700 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc355b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47e7fff700 1 -- 172.21.15.16:0/4276008030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc33263b0 con 0x557dc355b720 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47e7fff700 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc3555350 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f47d0015000 tx=0x7f47d001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b1559f7c1611a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47deffd700 1 -- 172.21.15.16:0/4276008030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d002b070 con 0x557dc355b720 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47deffd700 1 -- 172.21.15.16:0/4276008030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47d001e040 con 0x557dc355b720 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.949+0000 7f47deffd700 1 -- 172.21.15.16:0/4276008030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d00049f0 con 0x557dc355b720 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4276008030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 msgr2=0x557dc3555350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc3555350 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f47d0015000 tx=0x7f47d001f000 comp rx=0 tx=0).stop 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4276008030 shutdown_connections 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3555cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc355b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4276008030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc3555350 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4276008030 >> 172.21.15.16:0/4276008030 conn(0x557dc34b3210 msgr2=0x557dc354fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4276008030 shutdown_connections 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4276008030 wait complete. 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 -- start start 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.950+0000 7f47fadf2b80 1 Processor -- start 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc3555890 0x557dc3561750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355ae00 0x557dc355ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc355f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc3441560 con 0x557dc355b720 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc3441200 con 0x557dc3555890 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc33286a0 con 0x557dc355ae00 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47fadf2b80 10 client.0 _ll_register_callbacks cb 0x557dc344ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc355f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc355f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39368/0 (socket says 172.21.15.16:39368) 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 -- 172.21.15.16:0/4091043897 learned_addr learned my addr 172.21.15.16:0/4091043897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc3555890 0x557dc3561750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47e7fff700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355ae00 0x557dc355ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 -- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355ae00 msgr2=0x557dc355ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355ae00 0x557dc355ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 -- 172.21.15.16:0/4091043897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc3555890 msgr2=0x557dc3561750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc3555890 0x557dc3561750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 -- 172.21.15.16:0/4091043897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e8110860 con 0x557dc355b720 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc355f180 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f47e0011d00 tx=0x7f47e000f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dfed9557503bce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.951+0000 7f47c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47c7fff700 1 -- 172.21.15.16:0/4091043897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000fac0 con 0x557dc355b720 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47c7fff700 1 -- 172.21.15.16:0/4091043897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f47e000fc20 con 0x557dc355b720 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557dc33fe270 con 0x557dc355b720 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47c7fff700 1 -- 172.21.15.16:0/4091043897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e000fde0 con 0x557dc355b720 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 10 client.17796 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557dc32d2b60 con 0x557dc355b720 2024-05-08T23:30:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47c7fff700 1 -- 172.21.15.16:0/4091043897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557dc33fe270 con 0x557dc355b720 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47c7fff700 1 -- 172.21.15.16:0/4091043897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47e00143d0 con 0x557dc355b720 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 10 client.17796 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 10 client.17796 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 20 client.17796 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc356f100 0x557dc3571520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 4 client.17796 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.952+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dc3524810 con 0x557dc356f100 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.953+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc356f100 0x557dc3571520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.953+0000 7f47c7fff700 10 client.17796 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.953+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc356f100 0x557dc3571520 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65a6a021a61f2bf server_cookie=7c48c759ab0e75bf in_seq=0 out_seq=0 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47c7fff700 1 -- 172.21.15.16:0/4091043897 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557dc3524810 con 0x557dc356f100 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47c7fff700 10 client.17796 handle_command_reply: tid=0 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 client.17796 shutdown 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 msgr2=0x557dc355f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc355f180 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f47e0011d00 tx=0x7f47e000f4b0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 shutdown_connections 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355b720 0x557dc355f180 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc356f100 0x557dc3571520 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc3555890 0x557dc3561750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/4091043897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355ae00 0x557dc355ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47c7fff700 0 client.17796 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 >> 172.21.15.16:0/4091043897 conn(0x557dc34b3210 msgr2=0x557dc354f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 shutdown_connections 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 1 -- 172.21.15.16:0/4091043897 wait complete. 2024-05-08T23:30:06.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.954+0000 7f47fadf2b80 20 client.17796 trim_cache size 0 max 16384 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 Processor -- start 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 -- start start 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 0x557dc3555350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc354fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355b720 0x557dc35660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc356bc50 con 0x557dc3555890 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc354aa90 con 0x557dc355ae00 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc354cdc0 con 0x557dc355b720 2024-05-08T23:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.968+0000 7f47ec9d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 0x557dc3555350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 0x557dc3555350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39374/0 (socket says 172.21.15.16:39374) 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 -- 172.21.15.16:0/1229712483 learned_addr learned my addr 172.21.15.16:0/1229712483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47e7fff700 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc354fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355b720 0x557dc35660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 -- 172.21.15.16:0/1229712483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355b720 msgr2=0x557dc35660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355b720 0x557dc35660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 -- 172.21.15.16:0/1229712483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 msgr2=0x557dc354fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc354fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 -- 172.21.15.16:0/1229712483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dc346b560 con 0x557dc3555890 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 0x557dc3555350 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f47d80968d0 tx=0x7f47d80cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafbe27051b180f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47df7fe700 1 -- 172.21.15.16:0/1229712483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d80b8270 con 0x557dc3555890 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47df7fe700 1 -- 172.21.15.16:0/1229712483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47d8007dd0 con 0x557dc3555890 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.969+0000 7f47df7fe700 1 -- 172.21.15.16:0/1229712483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8096570 con 0x557dc3555890 2024-05-08T23:30:06.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 -- 172.21.15.16:0/1229712483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 msgr2=0x557dc3555350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 0x557dc3555350 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f47d80968d0 tx=0x7f47d80cc040 comp rx=0 tx=0).stop 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 -- 172.21.15.16:0/1229712483 shutdown_connections 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc355b720 0x557dc35660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc3555890 0x557dc3555350 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/1229712483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355ae00 0x557dc354fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 -- 172.21.15.16:0/1229712483 >> 172.21.15.16:0/1229712483 conn(0x557dc34b3210 msgr2=0x557dc3575e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 -- 172.21.15.16:0/1229712483 shutdown_connections 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 -- 172.21.15.16:0/1229712483 wait complete. 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 -- start start 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.970+0000 7f47fadf2b80 1 Processor -- start 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3564090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 0x557dc35645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355b720 0x557dc35c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc3564d70 con 0x557dc355ae00 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc3564ee0 con 0x557dc355b720 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dc35c4b00 con 0x557dc3555890 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47fadf2b80 10 client.0 _ll_register_callbacks cb 0x557dc344ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 0x557dc35645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 0x557dc35645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39390/0 (socket says 172.21.15.16:39390) 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 -- 172.21.15.16:0/3511634121 learned_addr learned my addr 172.21.15.16:0/3511634121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3564090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47ed1d5700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355b720 0x557dc35c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 -- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 msgr2=0x557dc3564090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3564090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 -- 172.21.15.16:0/3511634121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355b720 msgr2=0x557dc35c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355b720 0x557dc35c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47e7fff700 1 -- 172.21.15.16:0/3511634121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d809ae70 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.971+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3564090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47e7fff700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 0x557dc35645d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f47d0031040 tx=0x7f47d000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852b462d236da426 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47c7fff700 1 -- 172.21.15.16:0/3511634121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d002b040 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47c7fff700 1 -- 172.21.15.16:0/3511634121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f47d0004910 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47d001e040 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47c7fff700 1 -- 172.21.15.16:0/3511634121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47d0004ad0 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 10 client.17802 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557dc332acd0 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47c7fff700 1 -- 172.21.15.16:0/3511634121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47d0042080 con 0x557dc355ae00 2024-05-08T23:30:06.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47c7fff700 1 -- 172.21.15.16:0/3511634121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47d0038cb0 con 0x557dc355ae00 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 10 client.17802 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 10 client.17802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 20 client.17802 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc35c5e90 0x557dc35c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 4 client.17802 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.972+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557dc349dec0 con 0x557dc35c5e90 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc35c5e90 0x557dc35c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47c7fff700 10 client.17802 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47ec9d4700 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc35c5e90 0x557dc35c8280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7c7f432f1c57972 server_cookie=e063cc666e74ffc9 in_seq=0 out_seq=0 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47c7fff700 1 -- 172.21.15.16:0/3511634121 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557dc349dec0 con 0x557dc35c5e90 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47c7fff700 10 client.17802 handle_command_reply: tid=0 2024-05-08T23:30:06.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 client.17802 shutdown 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 msgr2=0x557dc35645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 0x557dc35645d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f47d0031040 tx=0x7f47d000bf40 comp rx=0 tx=0).stop 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 shutdown_connections 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x557dc35c5e90 0x557dc35c8280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x557dc355b720 0x557dc35c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dc3555890 0x557dc3564090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47c7fff700 0 client.17802 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 --2- 172.21.15.16:0/3511634121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dc355ae00 0x557dc35645d0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 >> 172.21.15.16:0/3511634121 conn(0x557dc34b3210 msgr2=0x557dc355d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 shutdown_connections 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.973+0000 7f47fadf2b80 1 -- 172.21.15.16:0/3511634121 wait complete. 2024-05-08T23:30:06.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47fadf2b80 20 client.17802 trim_cache size 0 max 16384 2024-05-08T23:30:06.981 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47d40a1290 msgr2=0x7f47d40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47d40a1290 0x7f47d40a3750 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b5c0 tx=0x7f47d0005fb0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 msgr2=0x7f47e81994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 0x7f47e81994a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c9b0 tx=0x7f47d800cd70 comp rx=0 tx=0).stop 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 shutdown_connections 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f47d40a1290 0x7f47d40a3750 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47e810d940 0x7f47e819dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47e8068050 0x7f47e81994a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 --2- 172.21.15.16:0/3038060016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f47e8068970 0x7f47e81999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:06.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 >> 172.21.15.16:0/3038060016 conn(0x7f47e8076f40 msgr2=0x7f47e810ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:06.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 shutdown_connections 2024-05-08T23:30:06.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:06.974+0000 7f47eec38700 1 -- 172.21.15.16:0/3038060016 wait complete. 2024-05-08T23:30:06.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:30:07.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 -- 172.21.15.16:0/3572876737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 msgr2=0x7f87d410cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 --2- 172.21.15.16:0/3572876737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d410cce0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b0a0 tx=0x7f87cc00b3b0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 -- 172.21.15.16:0/3572876737 shutdown_connections 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 --2- 172.21.15.16:0/3572876737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d410cce0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 --2- 172.21.15.16:0/3572876737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87d4100060 0x7f87d4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 --2- 172.21.15.16:0/3572876737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d4100a10 0x7f87d4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 -- 172.21.15.16:0/3572876737 >> 172.21.15.16:0/3572876737 conn(0x7f87d4076f40 msgr2=0x7f87d4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 -- 172.21.15.16:0/3572876737 shutdown_connections 2024-05-08T23:30:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.126+0000 7f87d9038700 1 -- 172.21.15.16:0/3572876737 wait complete. 2024-05-08T23:30:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.127+0000 7f87d9038700 1 Processor -- start 2024-05-08T23:30:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.127+0000 7f87d9038700 1 -- start start 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d9038700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87d4100060 0x7f87d4197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d9038700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d9038700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d419bd70 0x7f87d419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d9038700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d419c7f0 con 0x7f87d4105160 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d9038700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d419c960 con 0x7f87d4100060 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d9038700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d419cb20 con 0x7f87d419bd70 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d41979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39408/0 (socket says 172.21.15.16:39408) 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 -- 172.21.15.16:0/1007346363 learned_addr learned my addr 172.21.15.16:0/1007346363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87d4100060 0x7f87d4197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87d359e700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d419bd70 0x7f87d419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 -- 172.21.15.16:0/1007346363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d419bd70 msgr2=0x7f87d419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d419bd70 0x7f87d419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 -- 172.21.15.16:0/1007346363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87d4100060 msgr2=0x7f87d4197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87d4100060 0x7f87d4197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 -- 172.21.15.16:0/1007346363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc009520 con 0x7f87d4105160 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.128+0000 7f87ca59c700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d41979d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00ea30 tx=0x7f87bc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.129+0000 7f87d0d99700 1 -- 172.21.15.16:0/1007346363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc009820 con 0x7f87d4105160 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.129+0000 7f87d0d99700 1 -- 172.21.15.16:0/1007346363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87bc0044b0 con 0x7f87d4105160 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.129+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc009d00 con 0x7f87d4105160 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.129+0000 7f87d0d99700 1 -- 172.21.15.16:0/1007346363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc003ce0 con 0x7f87d4105160 2024-05-08T23:30:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.129+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d41a43d0 con 0x7f87d4105160 2024-05-08T23:30:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.131+0000 7f87d0d99700 1 -- 172.21.15.16:0/1007346363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f87bc018430 con 0x7f87d4105160 2024-05-08T23:30:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.132+0000 7f87d0d99700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87c00a1290 0x7f87c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.132+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87c00a1290 0x7f87c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.132+0000 7f87d0d99700 1 -- 172.21.15.16:0/1007346363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f87bc014070 con 0x7f87d4105160 2024-05-08T23:30:07.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87c00a1290 0x7f87c00a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009c60 tx=0x7f87c4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 Processor -- start 2024-05-08T23:30:07.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 -- start start 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac4e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 0x563cfac48cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfab9f150 con 0x563cfac48890 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfab9cb10 con 0x563cfac4e720 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfab98400 con 0x563cfac4de00 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87d359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 0x563cfac48cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87d359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 0x563cfac48cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39424/0 (socket says 172.21.15.16:39424) 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87d359e700 1 -- 172.21.15.16:0/801933171 learned_addr learned my addr 172.21.15.16:0/801933171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.133+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac4e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87ca59c700 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87d359e700 1 -- 172.21.15.16:0/801933171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 msgr2=0x563cfac4e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87d359e700 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac4e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87d359e700 1 -- 172.21.15.16:0/801933171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 msgr2=0x563cfac48350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87d359e700 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac48350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87d359e700 1 -- 172.21.15.16:0/801933171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cfaa193b0 con 0x563cfac48890 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87d359e700 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 0x563cfac48cd0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b780 tx=0x7f87cc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3519346e3f1e6d35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87c959a700 1 -- 172.21.15.16:0/801933171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00e050 con 0x563cfac48890 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87c959a700 1 -- 172.21.15.16:0/801933171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87cc0076e0 con 0x563cfac48890 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.134+0000 7f87c959a700 1 -- 172.21.15.16:0/801933171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc003c60 con 0x563cfac48890 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 -- 172.21.15.16:0/801933171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 msgr2=0x563cfac48cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 0x563cfac48cd0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b780 tx=0x7f87cc0092e0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 -- 172.21.15.16:0/801933171 shutdown_connections 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac48890 0x563cfac48cd0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac4e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/801933171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac48350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 -- 172.21.15.16:0/801933171 >> 172.21.15.16:0/801933171 conn(0x563cfaba6210 msgr2=0x563cfac42df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 -- 172.21.15.16:0/801933171 shutdown_connections 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 -- 172.21.15.16:0/801933171 wait complete. 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.135+0000 7f87e51f2b80 1 -- start start 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 Processor -- start 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 0x563cfac52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfab34560 con 0x563cfac4e720 2024-05-08T23:30:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfab34200 con 0x563cfac48890 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfaa1b6a0 con 0x563cfac4de00 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87e51f2b80 10 client.0 _ll_register_callbacks cb 0x563cfab3db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 0x563cfac52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 0x563cfac52530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39428/0 (socket says 172.21.15.16:39428) 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 -- 172.21.15.16:0/492784126 learned_addr learned my addr 172.21.15.16:0/492784126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87ca59c700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 -- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 msgr2=0x563cfac51ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac51ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 -- 172.21.15.16:0/492784126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 msgr2=0x563cfac51ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac51ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.136+0000 7f87d359e700 1 -- 172.21.15.16:0/492784126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d410d550 con 0x563cfac4e720 2024-05-08T23:30:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87d359e700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 0x563cfac52530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009850 tx=0x7f87cc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a364927fb57266 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87ba7fc700 1 -- 172.21.15.16:0/492784126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc003c00 con 0x563cfac4e720 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87ba7fc700 1 -- 172.21.15.16:0/492784126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f87cc003d60 con 0x563cfac4e720 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cfaaf1270 con 0x563cfac4e720 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87ba7fc700 1 -- 172.21.15.16:0/492784126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87cc007a20 con 0x563cfac4e720 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 10 client.17814 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563cfa9c5b60 con 0x563cfac4e720 2024-05-08T23:30:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87ba7fc700 1 -- 172.21.15.16:0/492784126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563cfaaf1270 con 0x563cfac4e720 2024-05-08T23:30:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87ba7fc700 1 -- 172.21.15.16:0/492784126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87cc023020 con 0x563cfac4e720 2024-05-08T23:30:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 10 client.17814 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 10 client.17814 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:30:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 20 client.17814 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfac61c90 0x563cfac64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 4 client.17814 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.137+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cfac17810 con 0x563cfac61c90 2024-05-08T23:30:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.138+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfac61c90 0x563cfac64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.138+0000 7f87ba7fc700 10 client.17814 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.138+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfac61c90 0x563cfac64080 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=389a3dc5ca32ea98 server_cookie=624ed45ec1776be6 in_seq=0 out_seq=0 2024-05-08T23:30:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87ba7fc700 1 -- 172.21.15.16:0/492784126 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cfac17810 con 0x563cfac61c90 2024-05-08T23:30:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87ba7fc700 10 client.17814 handle_command_reply: tid=0 2024-05-08T23:30:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 client.17814 shutdown 2024-05-08T23:30:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 msgr2=0x563cfac52530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 0x563cfac52530 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009850 tx=0x7f87cc00be80 comp rx=0 tx=0).stop 2024-05-08T23:30:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 shutdown_connections 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4e720 0x563cfac52530 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfac61c90 0x563cfac64080 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac51ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/492784126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4de00 0x563cfac51ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 >> 172.21.15.16:0/492784126 conn(0x563cfaba6210 msgr2=0x563cfac425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87ba7fc700 0 client.17814 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 shutdown_connections 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 1 -- 172.21.15.16:0/492784126 wait complete. 2024-05-08T23:30:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.139+0000 7f87e51f2b80 20 client.17814 trim_cache size 0 max 16384 2024-05-08T23:30:07.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 Processor -- start 2024-05-08T23:30:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 -- start start 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac48350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac42d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4e720 0x563cfac52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfac53120 con 0x563cfac4de00 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfac69250 con 0x563cfac48890 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.153+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfac646e0 con 0x563cfac4e720 2024-05-08T23:30:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac42d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac42d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39430/0 (socket says 172.21.15.16:39430) 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 -- 172.21.15.16:0/3790729369 learned_addr learned my addr 172.21.15.16:0/3790729369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87d359e700 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4e720 0x563cfac52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac48350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 -- 172.21.15.16:0/3790729369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4e720 msgr2=0x563cfac52350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4e720 0x563cfac52350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 -- 172.21.15.16:0/3790729369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 msgr2=0x563cfac48350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac48350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 -- 172.21.15.16:0/3790729369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cfab5e560 con 0x563cfac4de00 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87ca59c700 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac42d20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c5b0 tx=0x7f87bc004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc7ea5488b4b2c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87c9d9b700 1 -- 172.21.15.16:0/3790729369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc09ded0 con 0x563cfac4de00 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87c9d9b700 1 -- 172.21.15.16:0/3790729369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87bc0a2520 con 0x563cfac4de00 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.154+0000 7f87c9d9b700 1 -- 172.21.15.16:0/3790729369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc01f340 con 0x563cfac4de00 2024-05-08T23:30:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 -- 172.21.15.16:0/3790729369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 msgr2=0x563cfac42d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac42d20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00c5b0 tx=0x7f87bc004620 comp rx=0 tx=0).stop 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 -- 172.21.15.16:0/3790729369 shutdown_connections 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac4e720 0x563cfac52350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac48890 0x563cfac48350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/3790729369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac42d20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 -- 172.21.15.16:0/3790729369 >> 172.21.15.16:0/3790729369 conn(0x563cfaba6210 msgr2=0x563cfac48c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 -- 172.21.15.16:0/3790729369 shutdown_connections 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 -- 172.21.15.16:0/3790729369 wait complete. 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.155+0000 7f87e51f2b80 1 -- start start 2024-05-08T23:30:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 Processor -- start 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac48890 0x563cfac595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac569a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac56fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfac576e0 con 0x563cfac4de00 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfac57850 con 0x563cfac4e720 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cfacb71a0 con 0x563cfac48890 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87e51f2b80 10 client.0 _ll_register_callbacks cb 0x563cfab3db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39442/0 (socket says 172.21.15.16:39442) 2024-05-08T23:30:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 -- 172.21.15.16:0/4188890275 learned_addr learned my addr 172.21.15.16:0/4188890275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac48890 0x563cfac595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87d359e700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac56fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 -- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac48890 msgr2=0x563cfac595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac48890 0x563cfac595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 -- 172.21.15.16:0/4188890275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 msgr2=0x563cfac56fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac56fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87ca59c700 1 -- 172.21.15.16:0/4188890275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d41a43d0 con 0x563cfac4de00 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.156+0000 7f87d359e700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ca59c700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac569a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f87bc09a030 tx=0x7f87bc09a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd0c63b5fcb247a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ba7fc700 1 -- 172.21.15.16:0/4188890275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc01f340 con 0x563cfac4de00 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ba7fc700 1 -- 172.21.15.16:0/4188890275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f87bc01f4a0 con 0x563cfac4de00 2024-05-08T23:30:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87cc0076e0 con 0x563cfac4de00 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ba7fc700 1 -- 172.21.15.16:0/4188890275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87bc09a300 con 0x563cfac4de00 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 10 client.17820 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563cfaa1dcd0 con 0x563cfac4de00 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ba7fc700 1 -- 172.21.15.16:0/4188890275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87bc0a6020 con 0x563cfac4de00 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87ba7fc700 1 -- 172.21.15.16:0/4188890275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87bc099590 con 0x563cfac4de00 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 10 client.17820 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 10 client.17820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 20 client.17820 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfacb8470 0x563cfacba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 4 client.17820 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.157+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563cfab90ec0 con 0x563cfacb8470 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.158+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfacb8470 0x563cfacba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.158+0000 7f87ba7fc700 10 client.17820 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.158+0000 7f87d2d9d700 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfacb8470 0x563cfacba860 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=542a7ef31340682c server_cookie=472262f42894c618 in_seq=0 out_seq=0 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.158+0000 7f87ba7fc700 1 -- 172.21.15.16:0/4188890275 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cfab90ec0 con 0x563cfacb8470 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.158+0000 7f87ba7fc700 10 client.17820 handle_command_reply: tid=0 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.158+0000 7f87e51f2b80 1 client.17820 shutdown 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 msgr2=0x563cfac569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac569a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f87bc09a030 tx=0x7f87bc09a210 comp rx=0 tx=0).stop 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 shutdown_connections 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x563cfacb8470 0x563cfacba860 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563cfac4e720 0x563cfac56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cfac48890 0x563cfac595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 --2- 172.21.15.16:0/4188890275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563cfac4de00 0x563cfac569a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87ba7fc700 0 client.17820 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 >> 172.21.15.16:0/4188890275 conn(0x563cfaba6210 msgr2=0x563cfac523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 shutdown_connections 2024-05-08T23:30:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 1 -- 172.21.15.16:0/4188890275 wait complete. 2024-05-08T23:30:07.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87e51f2b80 20 client.17820 trim_cache size 0 max 16384 2024-05-08T23:30:07.165 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87c00a1290 msgr2=0x7f87c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87c00a1290 0x7f87c00a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009c60 tx=0x7f87c4009260 comp rx=0 tx=0).stop 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 msgr2=0x7f87d41979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d41979d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00ea30 tx=0x7f87bc00ed40 comp rx=0 tx=0).stop 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 shutdown_connections 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f87c00a1290 0x7f87c00a3750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d419bd70 0x7f87d419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87d4100060 0x7f87d4197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 --2- 172.21.15.16:0/1007346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d4105160 0x7f87d41979d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.159+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 >> 172.21.15.16:0/1007346363 conn(0x7f87d4076f40 msgr2=0x7f87d4068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.160+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 shutdown_connections 2024-05-08T23:30:07.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.160+0000 7f87d9038700 1 -- 172.21.15.16:0/1007346363 wait complete. 2024-05-08T23:30:07.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:30:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 -- 172.21.15.16:0/14476747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 msgr2=0x7f52fc0686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 --2- 172.21.15.16:0/14476747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc0686c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f52f80099c0 tx=0x7f52f8009cd0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 -- 172.21.15.16:0/14476747 shutdown_connections 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 --2- 172.21.15.16:0/14476747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52fc106290 0x7f52fc10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 --2- 172.21.15.16:0/14476747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc0686c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 --2- 172.21.15.16:0/14476747 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52fc068c90 0x7f52fc105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 -- 172.21.15.16:0/14476747 >> 172.21.15.16:0/14476747 conn(0x7f52fc076f40 msgr2=0x7f52fc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 -- 172.21.15.16:0/14476747 shutdown_connections 2024-05-08T23:30:07.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.312+0000 7f5303d0d700 1 -- 172.21.15.16:0/14476747 wait complete. 2024-05-08T23:30:07.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.313+0000 7f5303d0d700 1 Processor -- start 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.313+0000 7f5303d0d700 1 -- start start 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5303d0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5303d0d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52fc106290 0x7f52fc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5303d0d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52fc19dd70 0x7f52fc19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5303d0d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc19e7f0 con 0x7f52fc0682e0 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5303d0d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc19e960 con 0x7f52fc19dd70 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5303d0d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc19eb20 con 0x7f52fc106290 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39452/0 (socket says 172.21.15.16:39452) 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 -- 172.21.15.16:0/651212340 learned_addr learned my addr 172.21.15.16:0/651212340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f53012a8700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52fc106290 0x7f52fc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f53022aa700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52fc19dd70 0x7f52fc19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 -- 172.21.15.16:0/651212340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52fc106290 msgr2=0x7f52fc199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52fc106290 0x7f52fc199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 -- 172.21.15.16:0/651212340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52fc19dd70 msgr2=0x7f52fc19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52fc19dd70 0x7f52fc19e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 -- 172.21.15.16:0/651212340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80095f0 con 0x7f52fc0682e0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.314+0000 7f5301aa9700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc1994e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005910 tx=0x7f52f800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.315+0000 7f52f2ffd700 1 -- 172.21.15.16:0/651212340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f801c070 con 0x7f52fc0682e0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.315+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc1a6020 con 0x7f52fc0682e0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.315+0000 7f52f2ffd700 1 -- 172.21.15.16:0/651212340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52f8005030 con 0x7f52fc0682e0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.315+0000 7f52f2ffd700 1 -- 172.21.15.16:0/651212340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8004930 con 0x7f52fc0682e0 2024-05-08T23:30:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.315+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc1a63e0 con 0x7f52fc0682e0 2024-05-08T23:30:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.317+0000 7f52f2ffd700 1 -- 172.21.15.16:0/651212340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f52f8020070 con 0x7f52fc0682e0 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.317+0000 7f52f2ffd700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f52e80a1290 0x7f52e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.318+0000 7f52f2ffd700 1 -- 172.21.15.16:0/651212340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f52f80cdcb0 con 0x7f52fc0682e0 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.318+0000 7f53012a8700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f52e80a1290 0x7f52e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.318+0000 7f53012a8700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f52e80a1290 0x7f52e80a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005ea0 tx=0x7f52ec005e30 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.318+0000 7f530fec7b80 1 Processor -- start 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 -- start start 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad0f8150 con 0x555dad1a1890 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad0f5b10 con 0x555dad1a6e00 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f530fec7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad0f1400 con 0x555dad1a7720 2024-05-08T23:30:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f53022aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f53022aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39462/0 (socket says 172.21.15.16:39462) 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f53022aa700 1 -- 172.21.15.16:0/2903278665 learned_addr learned my addr 172.21.15.16:0/2903278665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f5301aa9700 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f53012a8700 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f53022aa700 1 -- 172.21.15.16:0/2903278665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 msgr2=0x555dad1a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.319+0000 7f53022aa700 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f53022aa700 1 -- 172.21.15.16:0/2903278665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 msgr2=0x555dad1a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f53022aa700 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f53022aa700 1 -- 172.21.15.16:0/2903278665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dacf723b0 con 0x555dad1a1890 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f53012a8700 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f53022aa700 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1a1cd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009440 tx=0x7f52f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5b3a8ced55c281 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f52e7fff700 1 -- 172.21.15.16:0/2903278665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400e070 con 0x555dad1a1890 2024-05-08T23:30:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f52e7fff700 1 -- 172.21.15.16:0/2903278665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52f400f040 con 0x555dad1a1890 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.320+0000 7f52e7fff700 1 -- 172.21.15.16:0/2903278665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4015a40 con 0x555dad1a1890 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 -- 172.21.15.16:0/2903278665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 msgr2=0x555dad1a1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1a1cd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009440 tx=0x7f52f4009920 comp rx=0 tx=0).stop 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 -- 172.21.15.16:0/2903278665 shutdown_connections 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1a1cd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 --2- 172.21.15.16:0/2903278665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 -- 172.21.15.16:0/2903278665 >> 172.21.15.16:0/2903278665 conn(0x555dad0ff210 msgr2=0x555dad19bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 -- 172.21.15.16:0/2903278665 shutdown_connections 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 -- 172.21.15.16:0/2903278665 wait complete. 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 -- start start 2024-05-08T23:30:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.321+0000 7f530fec7b80 1 Processor -- start 2024-05-08T23:30:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a1890 0x555dad1ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 0x555dad1aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad08d560 con 0x555dad1a6e00 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad08d200 con 0x555dad1a1890 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dacf746a0 con 0x555dad1a7720 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f530fec7b80 10 client.0 _ll_register_callbacks cb 0x555dad096b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 0x555dad1aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 0x555dad1aaa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39476/0 (socket says 172.21.15.16:39476) 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 -- 172.21.15.16:0/1718102210 learned_addr learned my addr 172.21.15.16:0/1718102210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f5301aa9700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a1890 0x555dad1ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53022aa700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 -- 172.21.15.16:0/1718102210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 msgr2=0x555dad1ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 -- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a1890 msgr2=0x555dad1ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a1890 0x555dad1ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 -- 172.21.15.16:0/1718102210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f40090f0 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f53012a8700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 0x555dad1aaa30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00a9e0 tx=0x7f52ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8d5565e98e9175 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.322+0000 7f52e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f52e57fa700 1 -- 172.21.15.16:0/1718102210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00f8d0 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f52e57fa700 1 -- 172.21.15.16:0/1718102210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f52ec00fa30 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dad04a270 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f52e57fa700 1 -- 172.21.15.16:0/1718102210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52ec00fbf0 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 10 client.17832 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555dacf1eb60 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f52e57fa700 1 -- 172.21.15.16:0/1718102210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555dad04a270 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f52e57fa700 1 -- 172.21.15.16:0/1718102210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f52ec024020 con 0x555dad1a6e00 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 10 client.17832 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 20 client.17832 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad1bb1b0 0x555dad1bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 4 client.17832 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.323+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dad170810 con 0x555dad1bb1b0 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.324+0000 7f5301aa9700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad1bb1b0 0x555dad1bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.324+0000 7f5301aa9700 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad1bb1b0 0x555dad1bd5d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7a44da7897482 server_cookie=8de586aef846d8ae in_seq=0 out_seq=0 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.324+0000 7f52e57fa700 10 client.17832 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f52e57fa700 1 -- 172.21.15.16:0/1718102210 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555dad170810 con 0x555dad1bb1b0 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f52e57fa700 10 client.17832 handle_command_reply: tid=0 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 client.17832 shutdown 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 msgr2=0x555dad1aaa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 0x555dad1aaa30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00a9e0 tx=0x7f52ec0047c0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 shutdown_connections 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad1bb1b0 0x555dad1bd5d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a1890 0x555dad1ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1718102210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a6e00 0x555dad1aaa30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f52e57fa700 0 client.17832 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 >> 172.21.15.16:0/1718102210 conn(0x555dad0ff210 msgr2=0x555dad19b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 shutdown_connections 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 1 -- 172.21.15.16:0/1718102210 wait complete. 2024-05-08T23:30:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.325+0000 7f530fec7b80 20 client.17832 trim_cache size 0 max 16384 2024-05-08T23:30:07.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.339+0000 7f530fec7b80 1 Processor -- start 2024-05-08T23:30:07.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.339+0000 7f530fec7b80 1 -- start start 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f530fec7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad19cac0 con 0x555dad1a1890 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f530fec7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad1a8f80 con 0x555dad1a6e00 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f530fec7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad1b7b50 con 0x555dad1a7720 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39478/0 (socket says 172.21.15.16:39478) 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 -- 172.21.15.16:0/3520563434 learned_addr learned my addr 172.21.15.16:0/3520563434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f53022aa700 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 -- 172.21.15.16:0/3520563434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 msgr2=0x555dad1a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f53012a8700 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 -- 172.21.15.16:0/3520563434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 msgr2=0x555dad1b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 -- 172.21.15.16:0/3520563434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dad0b7560 con 0x555dad1a1890 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f53022aa700 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f5301aa9700 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1aa9d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f52f8000c00 tx=0x7f52f80ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14414a23440338f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.340+0000 7f52f0ff9700 1 -- 172.21.15.16:0/3520563434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8098e00 con 0x555dad1a1890 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f52f0ff9700 1 -- 172.21.15.16:0/3520563434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52f80051a0 con 0x555dad1a1890 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f52f0ff9700 1 -- 172.21.15.16:0/3520563434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f809acc0 con 0x555dad1a1890 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 -- 172.21.15.16:0/3520563434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 msgr2=0x555dad1aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1aa9d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f52f8000c00 tx=0x7f52f80ba040 comp rx=0 tx=0).stop 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 -- 172.21.15.16:0/3520563434 shutdown_connections 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1aa9d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 --2- 172.21.15.16:0/3520563434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 -- 172.21.15.16:0/3520563434 >> 172.21.15.16:0/3520563434 conn(0x555dad0ff210 msgr2=0x555dad1aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 -- 172.21.15.16:0/3520563434 shutdown_connections 2024-05-08T23:30:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.341+0000 7f530fec7b80 1 -- 172.21.15.16:0/3520563434 wait complete. 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 -- start start 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 Processor -- start 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad2106c0 con 0x555dad1a1890 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad210800 con 0x555dad1a6e00 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dad210990 con 0x555dad1a7720 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.342+0000 7f530fec7b80 10 client.0 _ll_register_callbacks cb 0x555dad096b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39480/0 (socket says 172.21.15.16:39480) 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 -- 172.21.15.16:0/1878986313 learned_addr learned my addr 172.21.15.16:0/1878986313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f53012a8700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f53022aa700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 -- 172.21.15.16:0/1878986313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 msgr2=0x555dad1aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 -- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 msgr2=0x555dad1ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 -- 172.21.15.16:0/1878986313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc1a63e0 con 0x555dad1a1890 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f5301aa9700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1ae140 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f52f80a2920 tx=0x7f52f809b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21019682da7aee60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f52e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f52e57fa700 1 -- 172.21.15.16:0/1878986313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f80cb4a0 con 0x555dad1a1890 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f52e57fa700 1 -- 172.21.15.16:0/1878986313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f52f80b3120 con 0x555dad1a1890 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52f400f040 con 0x555dad1a1890 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f52e57fa700 1 -- 172.21.15.16:0/1878986313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52f807e020 con 0x555dad1a1890 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f530fec7b80 10 client.17838 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.343+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555dacf76cd0 con 0x555dad1a1890 2024-05-08T23:30:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f52e57fa700 1 -- 172.21.15.16:0/1878986313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52f400f040 con 0x555dad1a1890 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f52e57fa700 1 -- 172.21.15.16:0/1878986313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f52f809acc0 con 0x555dad1a1890 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f530fec7b80 10 client.17838 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f530fec7b80 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f530fec7b80 20 client.17838 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad211c20 0x555dad214010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f530fec7b80 4 client.17838 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555dad0e9ec0 con 0x555dad211c20 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f53012a8700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad211c20 0x555dad214010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f53012a8700 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad211c20 0x555dad214010 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef17ae303afeb142 server_cookie=e2e05e11e501e069 in_seq=0 out_seq=0 2024-05-08T23:30:07.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.344+0000 7f52e57fa700 10 client.17838 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f52e57fa700 1 -- 172.21.15.16:0/1878986313 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555dad0e9ec0 con 0x555dad211c20 2024-05-08T23:30:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f52e57fa700 10 client.17838 handle_command_reply: tid=0 2024-05-08T23:30:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 client.17838 shutdown 2024-05-08T23:30:07.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 msgr2=0x555dad1ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1ae140 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f52f80a2920 tx=0x7f52f809b1d0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 shutdown_connections 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dad1a7720 0x555dad1aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dad1a1890 0x555dad1ae140 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x555dad211c20 0x555dad214010 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 --2- 172.21.15.16:0/1878986313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x555dad1a6e00 0x555dad1ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f52e57fa700 0 client.17838 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 >> 172.21.15.16:0/1878986313 conn(0x555dad0ff210 msgr2=0x555dad1a63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 shutdown_connections 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 1 -- 172.21.15.16:0/1878986313 wait complete. 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.345+0000 7f530fec7b80 20 client.17838 trim_cache size 0 max 16384 2024-05-08T23:30:07.351 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:30:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f52e80a1290 msgr2=0x7f52e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f52e80a1290 0x7f52e80a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005ea0 tx=0x7f52ec005e30 comp rx=0 tx=0).stop 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 msgr2=0x7f52fc1994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc1994e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005910 tx=0x7f52f800f720 comp rx=0 tx=0).stop 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 shutdown_connections 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f52e80a1290 0x7f52e80a3750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52fc19dd70 0x7f52fc19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52fc0682e0 0x7f52fc1994e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 --2- 172.21.15.16:0/651212340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52fc106290 0x7f52fc199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 >> 172.21.15.16:0/651212340 conn(0x7f52fc076f40 msgr2=0x7f52fc10c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 shutdown_connections 2024-05-08T23:30:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:07.346+0000 7f5303d0d700 1 -- 172.21.15.16:0/651212340 wait complete. 2024-05-08T23:30:07.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:07 smithi155 ceph-mon[47105]: pgmap v328: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 33 op/s 2024-05-08T23:30:07.465 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:07 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2666692163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:30:07.613 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:07 smithi016 ceph-mon[42541]: pgmap v328: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 33 op/s 2024-05-08T23:30:07.614 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:07 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2666692163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:30:07.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:07 smithi189 ceph-mon[46919]: pgmap v328: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 33 op/s 2024-05-08T23:30:07.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:07 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2666692163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:30:08.360 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:30:08.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:30:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 -- 172.21.15.16:0/676252753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0100050 msgr2=0x7f23b0100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 --2- 172.21.15.16:0/676252753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0100050 0x7f23b0100430 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009990 tx=0x7f23a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 -- 172.21.15.16:0/676252753 shutdown_connections 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 --2- 172.21.15.16:0/676252753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23b0105170 0x7f23b010ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 --2- 172.21.15.16:0/676252753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0100050 0x7f23b0100430 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 --2- 172.21.15.16:0/676252753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b0100a00 0x7f23b0104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 -- 172.21.15.16:0/676252753 >> 172.21.15.16:0/676252753 conn(0x7f23b0076fa0 msgr2=0x7f23b00773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 -- 172.21.15.16:0/676252753 shutdown_connections 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.512+0000 7f23b4db0700 1 -- 172.21.15.16:0/676252753 wait complete. 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 Processor -- start 2024-05-08T23:30:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 -- start start 2024-05-08T23:30:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23b0100050 0x7f23b0197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 0x7f23b0197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b019bc30 0x7f23b019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019c6b0 con 0x7f23b0105170 2024-05-08T23:30:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019c820 con 0x7f23b019bc30 2024-05-08T23:30:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23b4db0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b019c9e0 con 0x7f23b0100050 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 0x7f23b0197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 0x7f23b0197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39490/0 (socket says 172.21.15.16:39490) 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23add9b700 1 -- 172.21.15.16:0/3206137326 learned_addr learned my addr 172.21.15.16:0/3206137326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23ae59c700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23b0100050 0x7f23b0197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.513+0000 7f23aed9d700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b019bc30 0x7f23b019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23add9b700 1 -- 172.21.15.16:0/3206137326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23b0100050 msgr2=0x7f23b0197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23add9b700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23b0100050 0x7f23b0197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23add9b700 1 -- 172.21.15.16:0/3206137326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b019bc30 msgr2=0x7f23b019c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23add9b700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b019bc30 0x7f23b019c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23add9b700 1 -- 172.21.15.16:0/3206137326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00095f0 con 0x7f23b0105170 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23aed9d700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b019bc30 0x7f23b019c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23add9b700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 0x7f23b0197890 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f23a400d950 tx=0x7f23a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f239f7fe700 1 -- 172.21.15.16:0/3206137326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4004220 con 0x7f23b0105170 2024-05-08T23:30:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b01a3f40 con 0x7f23b0105170 2024-05-08T23:30:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f239f7fe700 1 -- 172.21.15.16:0/3206137326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a4007620 con 0x7f23b0105170 2024-05-08T23:30:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f239f7fe700 1 -- 172.21.15.16:0/3206137326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4005530 con 0x7f23b0105170 2024-05-08T23:30:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.514+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b01a4410 con 0x7f23b0105170 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.516+0000 7f239f7fe700 1 -- 172.21.15.16:0/3206137326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f23a4004750 con 0x7f23b0105170 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.517+0000 7f239f7fe700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f23980a1290 0x7f23980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.517+0000 7f239f7fe700 1 -- 172.21.15.16:0/3206137326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f23a40cced0 con 0x7f23b0105170 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.517+0000 7f23ae59c700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f23980a1290 0x7f23980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23ae59c700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f23980a1290 0x7f23980a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23a0006010 tx=0x7f23a000b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 Processor -- start 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 -- start start 2024-05-08T23:30:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc03410e80 0x55dc03411260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 0x55dc0340b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc033621d0 con 0x55dc034117a0 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0335fb90 con 0x55dc03410e80 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.518+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0335b480 con 0x55dc0340b910 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 0x55dc0340b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 0x55dc0340b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39506/0 (socket says 172.21.15.16:39506) 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 -- 172.21.15.16:0/3402495258 learned_addr learned my addr 172.21.15.16:0/3402495258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23aed9d700 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23ae59c700 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc03410e80 0x55dc03411260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 -- 172.21.15.16:0/3402495258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 msgr2=0x55dc0340bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 -- 172.21.15.16:0/3402495258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc03410e80 msgr2=0x55dc03411260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc03410e80 0x55dc03411260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 -- 172.21.15.16:0/3402495258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc031dc430 con 0x55dc034117a0 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f23add9b700 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 0x55dc0340b3d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f23a40a1b40 tx=0x7f23a4099860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17429d68cfce94dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f239cff9700 1 -- 172.21.15.16:0/3402495258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a409da20 con 0x55dc034117a0 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f239cff9700 1 -- 172.21.15.16:0/3402495258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a40a2070 con 0x55dc034117a0 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.519+0000 7f239cff9700 1 -- 172.21.15.16:0/3402495258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4004a00 con 0x55dc034117a0 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/3402495258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 msgr2=0x55dc0340b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 0x55dc0340b3d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f23a40a1b40 tx=0x7f23a4099860 comp rx=0 tx=0).stop 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/3402495258 shutdown_connections 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc03410e80 0x55dc03411260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/3402495258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc034117a0 0x55dc0340b3d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/3402495258 >> 172.21.15.16:0/3402495258 conn(0x55dc03369290 msgr2=0x55dc03405e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/3402495258 shutdown_connections 2024-05-08T23:30:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.520+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/3402495258 wait complete. 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 -- start start 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 Processor -- start 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc0340b910 0x55dc03417800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03414af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc034117a0 0x55dc03415230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc032f75e0 con 0x55dc03410e80 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc032f7280 con 0x55dc0340b910 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc031de720 con 0x55dc034117a0 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23c0f6ab80 10 client.0 _ll_register_callbacks cb 0x55dc03300bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03414af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03414af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39512/0 (socket says 172.21.15.16:39512) 2024-05-08T23:30:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23add9b700 1 -- 172.21.15.16:0/2907870831 learned_addr learned my addr 172.21.15.16:0/2907870831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23ae59c700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc0340b910 0x55dc03417800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.521+0000 7f23aed9d700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc034117a0 0x55dc03415230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23add9b700 1 -- 172.21.15.16:0/2907870831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc034117a0 msgr2=0x55dc03415230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23add9b700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc034117a0 0x55dc03415230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23add9b700 1 -- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc0340b910 msgr2=0x55dc03417800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23add9b700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc0340b910 0x55dc03417800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23add9b700 1 -- 172.21.15.16:0/2907870831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b01a4410 con 0x55dc03410e80 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23add9b700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03414af0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ca7b0 tx=0x7f23a40ca250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a7a2b72abb6502 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f2395ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f2395ffb700 1 -- 172.21.15.16:0/2907870831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4097f90 con 0x55dc03410e80 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f2395ffb700 1 -- 172.21.15.16:0/2907870831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f23a4004a00 con 0x55dc03410e80 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc032b42f0 con 0x55dc03410e80 2024-05-08T23:30:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f2395ffb700 1 -- 172.21.15.16:0/2907870831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23a4004bc0 con 0x55dc03410e80 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23c0f6ab80 10 client.17850 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dc03188be0 con 0x55dc03410e80 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f2395ffb700 1 -- 172.21.15.16:0/2907870831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a407c080 con 0x55dc03410e80 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f2395ffb700 1 -- 172.21.15.16:0/2907870831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23a40a1be0 con 0x55dc03410e80 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23c0f6ab80 10 client.17850 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23c0f6ab80 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.522+0000 7f23c0f6ab80 20 client.17850 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.523+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc03424c60 0x55dc03427050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.523+0000 7f23c0f6ab80 4 client.17850 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.523+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc033da890 con 0x55dc03424c60 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.523+0000 7f23ae59c700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc03424c60 0x55dc03427050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.523+0000 7f23ae59c700 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc03424c60 0x55dc03427050 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f77609a0ef31938 server_cookie=9d460b314dfbfdad in_seq=0 out_seq=0 2024-05-08T23:30:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.523+0000 7f2395ffb700 10 client.17850 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f2395ffb700 1 -- 172.21.15.16:0/2907870831 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc033da890 con 0x55dc03424c60 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f2395ffb700 10 client.17850 handle_command_reply: tid=0 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 client.17850 shutdown 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 msgr2=0x55dc03414af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03414af0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ca7b0 tx=0x7f23a40ca250 comp rx=0 tx=0).stop 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 shutdown_connections 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc034117a0 0x55dc03415230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc03424c60 0x55dc03427050 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc0340b910 0x55dc03417800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/2907870831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03414af0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 >> 172.21.15.16:0/2907870831 conn(0x55dc03369290 msgr2=0x55dc03405620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f2395ffb700 0 client.17850 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 shutdown_connections 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/2907870831 wait complete. 2024-05-08T23:30:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.524+0000 7f23c0f6ab80 20 client.17850 trim_cache size 0 max 16384 2024-05-08T23:30:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 Processor -- start 2024-05-08T23:30:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 -- start start 2024-05-08T23:30:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03405da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc0340fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc03402e40 con 0x55dc03410e80 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc03400b10 con 0x55dc034117a0 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc03415c00 con 0x55dc0340b910 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03405da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.543+0000 7f23add9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03405da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39526/0 (socket says 172.21.15.16:39526) 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 -- 172.21.15.16:0/1916582023 learned_addr learned my addr 172.21.15.16:0/1916582023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23ae59c700 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23aed9d700 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc0340fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 -- 172.21.15.16:0/1916582023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 msgr2=0x55dc0340b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 -- 172.21.15.16:0/1916582023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 msgr2=0x55dc0340fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc0340fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 -- 172.21.15.16:0/1916582023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc0340bfc0 con 0x55dc03410e80 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23ae59c700 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f23add9b700 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03405da0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f23a4097f40 tx=0x7f23a400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c814b564703a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f239d7fa700 1 -- 172.21.15.16:0/1916582023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a409a2f0 con 0x55dc03410e80 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f239d7fa700 1 -- 172.21.15.16:0/1916582023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dc032b42f0 con 0x55dc03410e80 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.544+0000 7f239d7fa700 1 -- 172.21.15.16:0/1916582023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40a1d00 con 0x55dc03410e80 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/1916582023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 msgr2=0x55dc03405da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03405da0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f23a4097f40 tx=0x7f23a400df90 comp rx=0 tx=0).stop 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/1916582023 shutdown_connections 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc0340fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc0340b910 0x55dc0340b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/1916582023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc03410e80 0x55dc03405da0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/1916582023 >> 172.21.15.16:0/1916582023 conn(0x55dc03369290 msgr2=0x55dc034133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/1916582023 shutdown_connections 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/1916582023 wait complete. 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 -- start start 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.545+0000 7f23c0f6ab80 1 Processor -- start 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 0x55dc03418ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc03410e80 0x55dc034191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc03419720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0347b180 con 0x55dc0340b910 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0347b2f0 con 0x55dc034117a0 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc0347b4b0 con 0x55dc03410e80 2024-05-08T23:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 10 client.0 _ll_register_callbacks cb 0x55dc03300bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 0x55dc03418ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 0x55dc03418ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39534/0 (socket says 172.21.15.16:39534) 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 -- 172.21.15.16:0/729852816 learned_addr learned my addr 172.21.15.16:0/729852816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23aed9d700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc03419720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23add9b700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc03410e80 0x55dc034191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 -- 172.21.15.16:0/729852816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc03410e80 msgr2=0x55dc034191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc03410e80 0x55dc034191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 -- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 msgr2=0x55dc03419720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc03419720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 -- 172.21.15.16:0/729852816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc03188be0 con 0x55dc0340b910 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23add9b700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc03410e80 0x55dc034191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23aed9d700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc03419720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23ae59c700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 0x55dc03418ca0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009990 tx=0x7f23a0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa465fbd3b6d26b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f2395ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f2395ffb700 1 -- 172.21.15.16:0/729852816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000bde0 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f2395ffb700 1 -- 172.21.15.16:0/729852816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f23a0004960 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.546+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23a40a2070 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f2395ffb700 1 -- 172.21.15.16:0/729852816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23a0004b60 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 10 client.17856 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55dc033215e0 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f2395ffb700 1 -- 172.21.15.16:0/729852816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23a40a2070 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f2395ffb700 1 -- 172.21.15.16:0/729852816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f23a0035670 con 0x55dc0340b910 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 10 client.17856 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 20 client.17856 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc0347cd50 0x55dc0347f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 4 client.17856 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:30:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dc03353f40 con 0x55dc0347cd50 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.547+0000 7f23add9b700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc0347cd50 0x55dc0347f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.548+0000 7f23add9b700 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc0347cd50 0x55dc0347f170 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b46acda2d1d718 server_cookie=f9a052d2b2bdbc2f in_seq=0 out_seq=0 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.548+0000 7f2395ffb700 10 client.17856 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f2395ffb700 1 -- 172.21.15.16:0/729852816 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dc03353f40 con 0x55dc0347cd50 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f2395ffb700 10 client.17856 handle_command_reply: tid=0 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 client.17856 shutdown 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 msgr2=0x55dc03418ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 0x55dc03418ca0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009990 tx=0x7f23a0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 shutdown_connections 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55dc0347cd50 0x55dc0347f170 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55dc034117a0 0x55dc03419720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc0340b910 0x55dc03418ca0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 --2- 172.21.15.16:0/729852816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc03410e80 0x55dc034191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 >> 172.21.15.16:0/729852816 conn(0x55dc03369290 msgr2=0x55dc03414030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f2395ffb700 0 client.17856 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 shutdown_connections 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 1 -- 172.21.15.16:0/729852816 wait complete. 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.549+0000 7f23c0f6ab80 20 client.17856 trim_cache size 0 max 16384 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:30:08.556 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:30:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "5f0dacfd-b43e-4a74-8156-60e0181be270", 2024-05-08T23:30:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:30:08.557 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:30:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f23980a1290 msgr2=0x7f23980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f23980a1290 0x7f23980a3750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f23a0006010 tx=0x7f23a000b260 comp rx=0 tx=0).stop 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 msgr2=0x7f23b0197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 0x7f23b0197890 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f23a400d950 tx=0x7f23a400dc60 comp rx=0 tx=0).stop 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 shutdown_connections 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f23980a1290 0x7f23980a3750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23b019bc30 0x7f23b019c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23b0100050 0x7f23b0197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 --2- 172.21.15.16:0/3206137326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23b0105170 0x7f23b0197890 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 >> 172.21.15.16:0/3206137326 conn(0x7f23b0076fa0 msgr2=0x7f23b00698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 shutdown_connections 2024-05-08T23:30:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.550+0000 7f23b4db0700 1 -- 172.21.15.16:0/3206137326 wait complete. 2024-05-08T23:30:08.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:30:08.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 -- 172.21.15.16:0/909283478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 msgr2=0x7fdf68100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 --2- 172.21.15.16:0/909283478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68100440 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009990 tx=0x7fdf58009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 -- 172.21.15.16:0/909283478 shutdown_connections 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 --2- 172.21.15.16:0/909283478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 0x7fdf6810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 --2- 172.21.15.16:0/909283478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68100440 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 --2- 172.21.15.16:0/909283478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf68100a10 0x7fdf68104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 -- 172.21.15.16:0/909283478 >> 172.21.15.16:0/909283478 conn(0x7fdf68076f40 msgr2=0x7fdf68077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 -- 172.21.15.16:0/909283478 shutdown_connections 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.703+0000 7fdf70241700 1 -- 172.21.15.16:0/909283478 wait complete. 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 Processor -- start 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 -- start start 2024-05-08T23:30:08.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 0x7fdf68196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf6819ab20 0x7fdf6819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6819b5a0 con 0x7fdf68100060 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6819b710 con 0x7fdf68105160 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf70241700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6819b8d0 con 0x7fdf6819ab20 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf6dfdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf6dfdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68196240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39544/0 (socket says 172.21.15.16:39544) 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3923835900 learned_addr learned my addr 172.21.15.16:0/3923835900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf6819ab20 0x7fdf6819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.704+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 0x7fdf68196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3923835900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf6819ab20 msgr2=0x7fdf6819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf6819ab20 0x7fdf6819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3923835900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 msgr2=0x7fdf68196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 0x7fdf68196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3923835900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580095f0 con 0x7fdf68100060 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 0x7fdf68196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68196240 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fdf58006010 tx=0x7fdf58005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf5effd700 1 -- 172.21.15.16:0/3923835900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5801d070 con 0x7fdf68100060 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf681a2dd0 con 0x7fdf68100060 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf5effd700 1 -- 172.21.15.16:0/3923835900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf5800fb00 con 0x7fdf68100060 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a31f0 con 0x7fdf68100060 2024-05-08T23:30:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.705+0000 7fdf5effd700 1 -- 172.21.15.16:0/3923835900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58022b60 con 0x7fdf68100060 2024-05-08T23:30:08.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.707+0000 7fdf5effd700 1 -- 172.21.15.16:0/3923835900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdf5800fc70 con 0x7fdf68100060 2024-05-08T23:30:08.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.708+0000 7fdf5effd700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdf540a1290 0x7fdf540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.708+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdf540a1290 0x7fdf540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.708+0000 7fdf5effd700 1 -- 172.21.15.16:0/3923835900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fdf580ce290 con 0x7fdf68100060 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.709+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdf540a1290 0x7fdf540a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf68197790 tx=0x7fdf6400b5f0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.709+0000 7fdf7c3fbb80 1 Processor -- start 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.709+0000 7fdf7c3fbb80 1 -- start start 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af63681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6368720 0x563af6362350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6362890 0x563af6362cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af62b9150 con 0x563af6367e00 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af62b6b10 con 0x563af6362890 2024-05-08T23:30:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af62b2400 con 0x563af6368720 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af63681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af63681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39550/0 (socket says 172.21.15.16:39550) 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/2138610561 learned_addr learned my addr 172.21.15.16:0/2138610561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6368720 0x563af6362350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6362890 0x563af6362cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/2138610561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6368720 msgr2=0x563af6362350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6368720 0x563af6362350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/2138610561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6362890 msgr2=0x563af6362cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6362890 0x563af6362cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/2138610561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563af61333b0 con 0x563af6367e00 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6362890 0x563af6362cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.710+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af63681e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdf58097540 tx=0x7fdf5809aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70e69a04a3b7973 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf53fff700 1 -- 172.21.15.16:0/2138610561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5809ee70 con 0x563af6367e00 2024-05-08T23:30:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf53fff700 1 -- 172.21.15.16:0/2138610561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf580a3430 con 0x563af6367e00 2024-05-08T23:30:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf53fff700 1 -- 172.21.15.16:0/2138610561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5809b4e0 con 0x563af6367e00 2024-05-08T23:30:08.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2138610561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 msgr2=0x563af63681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af63681e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdf58097540 tx=0x7fdf5809aa90 comp rx=0 tx=0).stop 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2138610561 shutdown_connections 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6362890 0x563af6362cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af63681e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2138610561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6368720 0x563af6362350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.711+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2138610561 >> 172.21.15.16:0/2138610561 conn(0x563af62c0210 msgr2=0x563af635cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2138610561 shutdown_connections 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2138610561 wait complete. 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 -- start start 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 Processor -- start 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af636ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af636bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6368720 0x563af636c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af624e560 con 0x563af6367e00 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af624e200 con 0x563af6368720 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.712+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af61356a0 con 0x563af6362890 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf7c3fbb80 10 client.0 _ll_register_callbacks cb 0x563af6257b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af636bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af636bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39566/0 (socket says 172.21.15.16:39566) 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 -- 172.21.15.16:0/2202712809 learned_addr learned my addr 172.21.15.16:0/2202712809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af636ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6368720 0x563af636c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 -- 172.21.15.16:0/2202712809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 msgr2=0x563af636ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af636ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 -- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6368720 msgr2=0x563af636c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6368720 0x563af636c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 -- 172.21.15.16:0/2202712809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a31f0 con 0x563af6367e00 2024-05-08T23:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af636bfb0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400ff60 tx=0x7fdf64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a080dd44d67cb520 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.713+0000 7fdf517fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf517fa700 1 -- 172.21.15.16:0/2202712809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64028470 con 0x563af6367e00 2024-05-08T23:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563af620b270 con 0x563af6367e00 2024-05-08T23:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf517fa700 1 -- 172.21.15.16:0/2202712809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fdf6401f430 con 0x563af6367e00 2024-05-08T23:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf517fa700 1 -- 172.21.15.16:0/2202712809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf6401f630 con 0x563af6367e00 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf7c3fbb80 10 client.17868 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563af60dfb60 con 0x563af6367e00 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf517fa700 1 -- 172.21.15.16:0/2202712809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563af620b270 con 0x563af6367e00 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf517fa700 1 -- 172.21.15.16:0/2202712809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf6401e8f0 con 0x563af6367e00 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.716+0000 7fdf7c3fbb80 10 client.17868 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf7c3fbb80 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf7c3fbb80 20 client.17868 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af637c010 0x563af637e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf7c3fbb80 4 client.17868 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563af6331810 con 0x563af637c010 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af637c010 0x563af637e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf517fa700 10 client.17868 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.717+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af637c010 0x563af637e430 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=613647bd60681f97 server_cookie=11176b53ecb3211 in_seq=0 out_seq=0 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf517fa700 1 -- 172.21.15.16:0/2202712809 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563af6331810 con 0x563af637c010 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf517fa700 10 client.17868 handle_command_reply: tid=0 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 client.17868 shutdown 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 msgr2=0x563af636bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af636bfb0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400ff60 tx=0x7fdf64004750 comp rx=0 tx=0).stop 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 shutdown_connections 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af637c010 0x563af637e430 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6368720 0x563af636c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af636ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/2202712809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6367e00 0x563af636bfb0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf517fa700 0 client.17868 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.718+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 >> 172.21.15.16:0/2202712809 conn(0x563af62c0210 msgr2=0x563af635c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.719+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 shutdown_connections 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.719+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/2202712809 wait complete. 2024-05-08T23:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.719+0000 7fdf7c3fbb80 20 client.17868 trim_cache size 0 max 16384 2024-05-08T23:30:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 Processor -- start 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 -- start start 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af6367410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af635cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af636c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af63632d0 con 0x563af6368720 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af6359dc0 con 0x563af6367e00 2024-05-08T23:30:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.739+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af635e670 con 0x563af6362890 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af636c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af636c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39582/0 (socket says 172.21.15.16:39582) 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 -- 172.21.15.16:0/998638842 learned_addr learned my addr 172.21.15.16:0/998638842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af635cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af6367410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 -- 172.21.15.16:0/998638842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 msgr2=0x563af6367410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af6367410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 -- 172.21.15.16:0/998638842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 msgr2=0x563af635cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af635cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 -- 172.21.15.16:0/998638842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563af6278560 con 0x563af6368720 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af636c310 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdf600099c0 tx=0x7fdf60003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71200fdc1b63e5e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf5cff9700 1 -- 172.21.15.16:0/998638842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60003c10 con 0x563af6368720 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf5cff9700 1 -- 172.21.15.16:0/998638842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf6000f040 con 0x563af6368720 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.740+0000 7fdf5cff9700 1 -- 172.21.15.16:0/998638842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60016430 con 0x563af6368720 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/998638842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 msgr2=0x563af636c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af636c310 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdf600099c0 tx=0x7fdf60003f90 comp rx=0 tx=0).stop 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/998638842 shutdown_connections 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af636c310 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af6367410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/998638842 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af635cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/998638842 >> 172.21.15.16:0/998638842 conn(0x563af62c0210 msgr2=0x563af636a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/998638842 shutdown_connections 2024-05-08T23:30:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.741+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/998638842 wait complete. 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 -- start start 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 Processor -- start 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af63c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af63c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af63d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af63d3a70 con 0x563af6368720 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af63d3bb0 con 0x563af6367e00 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563af63d3d40 con 0x563af6362890 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf7c3fbb80 10 client.0 _ll_register_callbacks cb 0x563af6257b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf6e7de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af63d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf6dfdd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af63c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf6dfdd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af63c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:53938/0 (socket says 172.21.15.16:53938) 2024-05-08T23:30:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3363144992 learned_addr learned my addr 172.21.15.16:0/3363144992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af63d3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39594/0 (socket says 172.21.15.16:39594) 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.742+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af63c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 msgr2=0x563af63c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af63c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3363144992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 msgr2=0x563af63d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af63d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6dfdd700 1 -- 172.21.15.16:0/3363144992 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60003d70 con 0x563af6362890 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6e7de700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af63d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf6dfdd700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af63c86a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf5809a9e0 tx=0x7fdf5800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea113e5c7be91e78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf517fa700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf517fa700 1 -- 172.21.15.16:0/3363144992 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58022470 con 0x563af6362890 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf517fa700 1 -- 172.21.15.16:0/3363144992 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fdf5809a530 con 0x563af6362890 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf580a3430 con 0x563af6362890 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf517fa700 1 -- 172.21.15.16:0/3363144992 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf5809a6f0 con 0x563af6362890 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf7c3fbb80 10 client.27505 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.743+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563af6137cd0 con 0x563af6362890 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf517fa700 1 -- 172.21.15.16:0/3363144992 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf580a3430 con 0x563af6362890 2024-05-08T23:30:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf517fa700 1 -- 172.21.15.16:0/3363144992 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdf5809b2a0 con 0x563af6362890 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf7c3fbb80 10 client.27505 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf7c3fbb80 10 client.27505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf7c3fbb80 20 client.27505 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af63d5110 0x563af63d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf7c3fbb80 4 client.27505 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563af62aaec0 con 0x563af63d5110 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af63d5110 0x563af63d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf6d7dc700 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af63d5110 0x563af63d7530 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e333e0cb39159dc server_cookie=101cb349d80b0392 in_seq=0 out_seq=0 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.744+0000 7fdf517fa700 10 client.27505 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf517fa700 1 -- 172.21.15.16:0/3363144992 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563af62aaec0 con 0x563af63d5110 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf517fa700 10 client.27505 handle_command_reply: tid=0 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 client.27505 shutdown 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 msgr2=0x563af63c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af63c86a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf5809a9e0 tx=0x7fdf5800ff70 comp rx=0 tx=0).stop 2024-05-08T23:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 shutdown_connections 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563af6368720 0x563af63d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563af6362890 0x563af63c86a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x563af63d5110 0x563af63d7530 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 --2- 172.21.15.16:0/3363144992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x563af6367e00 0x563af63c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf517fa700 0 client.27505 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 >> 172.21.15.16:0/3363144992 conn(0x563af62c0210 msgr2=0x563af636a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 shutdown_connections 2024-05-08T23:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 1 -- 172.21.15.16:0/3363144992 wait complete. 2024-05-08T23:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf7c3fbb80 20 client.27505 trim_cache size 0 max 16384 2024-05-08T23:30:08.759 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:30:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "5f0dacfd-b43e-4a74-8156-60e0181be270": { 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "5f0dacfd-b43e-4a74-8156-60e0181be270", 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:30:08.760 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:30:08.761 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdf540a1290 msgr2=0x7fdf540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.745+0000 7fdf70241700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdf540a1290 0x7fdf540a3750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf68197790 tx=0x7fdf6400b5f0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 msgr2=0x7fdf68196240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68196240 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fdf58006010 tx=0x7fdf58005740 comp rx=0 tx=0).stop 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 shutdown_connections 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdf540a1290 0x7fdf540a3750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf6819ab20 0x7fdf6819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68100060 0x7fdf68196240 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 --2- 172.21.15.16:0/3923835900 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf68105160 0x7fdf68196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 >> 172.21.15.16:0/3923835900 conn(0x7fdf68076f40 msgr2=0x7fdf680fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 shutdown_connections 2024-05-08T23:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:08.746+0000 7fdf70241700 1 -- 172.21.15.16:0/3923835900 wait complete. 2024-05-08T23:30:08.763 INFO:tasks.cephfs.filesystem:scrub status for tag:5f0dacfd-b43e-4a74-8156-60e0181be270 - {'path': '/', 'tag': '5f0dacfd-b43e-4a74-8156-60e0181be270', 'options': 'recursive,force'} 2024-05-08T23:30:10.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:09 smithi016 ceph-mon[42541]: pgmap v329: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 33 op/s 2024-05-08T23:30:10.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:09 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:30:10.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:09 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:10.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:09 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:30:10.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:09 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:09 smithi189 ceph-mon[46919]: pgmap v329: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 33 op/s 2024-05-08T23:30:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:09 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:30:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:09 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:09 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:30:10.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:09 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:10.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:09 smithi155 ceph-mon[47105]: pgmap v329: 129 pgs: 129 active+clean; 18 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 112 KiB/s wr, 33 op/s 2024-05-08T23:30:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:09 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:30:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:09 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:09 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:30:10.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:09 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:11.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:10 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:30:11.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:10 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:30:11.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:10 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:30:12.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:11 smithi016 ceph-mon[42541]: pgmap v330: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:12.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:11 smithi189 ceph-mon[46919]: pgmap v330: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:12.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:11 smithi155 ceph-mon[47105]: pgmap v330: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:13 smithi016 ceph-mon[42541]: pgmap v331: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:13 smithi016 ceph-mon[42541]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:30:14.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:13 smithi189 ceph-mon[46919]: pgmap v331: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:13 smithi189 ceph-mon[46919]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:30:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:13 smithi155 ceph-mon[47105]: pgmap v331: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 116 KiB/s wr, 33 op/s 2024-05-08T23:30:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:13 smithi155 ceph-mon[47105]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T23:30:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:15 smithi016 ceph-mon[42541]: pgmap v333: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-05-08T23:30:16.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:15 smithi189 ceph-mon[46919]: pgmap v333: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-05-08T23:30:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:15 smithi155 ceph-mon[47105]: pgmap v333: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-05-08T23:30:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:17 smithi016 ceph-mon[42541]: pgmap v334: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 12 op/s 2024-05-08T23:30:18.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:17 smithi189 ceph-mon[46919]: pgmap v334: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 12 op/s 2024-05-08T23:30:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:17 smithi155 ceph-mon[47105]: pgmap v334: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 12 op/s 2024-05-08T23:30:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:19 smithi016 ceph-mon[42541]: pgmap v335: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 12 op/s 2024-05-08T23:30:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:19 smithi189 ceph-mon[46919]: pgmap v335: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 12 op/s 2024-05-08T23:30:20.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:19 smithi155 ceph-mon[47105]: pgmap v335: 129 pgs: 129 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 62 KiB/s wr, 12 op/s 2024-05-08T23:30:22.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:21 smithi016 ceph-mon[42541]: pgmap v336: 129 pgs: 129 active+clean; 19 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-08T23:30:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:21 smithi189 ceph-mon[46919]: pgmap v336: 129 pgs: 129 active+clean; 19 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-08T23:30:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:21 smithi155 ceph-mon[47105]: pgmap v336: 129 pgs: 129 active+clean; 19 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-08T23:30:24.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:23 smithi016 ceph-mon[42541]: pgmap v337: 129 pgs: 129 active+clean; 19 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-08T23:30:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:23 smithi189 ceph-mon[46919]: pgmap v337: 129 pgs: 129 active+clean; 19 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-08T23:30:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:23 smithi155 ceph-mon[47105]: pgmap v337: 129 pgs: 129 active+clean; 19 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-05-08T23:30:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:25 smithi016 ceph-mon[42541]: pgmap v338: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 86 KiB/s wr, 17 op/s 2024-05-08T23:30:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:25 smithi189 ceph-mon[46919]: pgmap v338: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 86 KiB/s wr, 17 op/s 2024-05-08T23:30:26.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:25 smithi155 ceph-mon[47105]: pgmap v338: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 86 KiB/s wr, 17 op/s 2024-05-08T23:30:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:27 smithi016 ceph-mon[42541]: pgmap v339: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:27 smithi189 ceph-mon[46919]: pgmap v339: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:28.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:27 smithi155 ceph-mon[47105]: pgmap v339: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:30.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:29 smithi016 ceph-mon[42541]: pgmap v340: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:30.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:29 smithi189 ceph-mon[46919]: pgmap v340: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:30.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:29 smithi155 ceph-mon[47105]: pgmap v340: 129 pgs: 129 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:32.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:31 smithi016 ceph-mon[42541]: pgmap v341: 129 pgs: 129 active+clean; 20 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-05-08T23:30:32.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:31 smithi189 ceph-mon[46919]: pgmap v341: 129 pgs: 129 active+clean; 20 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-05-08T23:30:32.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:31 smithi155 ceph-mon[47105]: pgmap v341: 129 pgs: 129 active+clean; 20 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-05-08T23:30:34.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:33 smithi016 ceph-mon[42541]: pgmap v342: 129 pgs: 129 active+clean; 20 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:33 smithi189 ceph-mon[46919]: pgmap v342: 129 pgs: 129 active+clean; 20 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:33 smithi155 ceph-mon[47105]: pgmap v342: 129 pgs: 129 active+clean; 20 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:36.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:35 smithi016 ceph-mon[42541]: pgmap v343: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:36.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:35 smithi189 ceph-mon[46919]: pgmap v343: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:36.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:35 smithi155 ceph-mon[47105]: pgmap v343: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:38.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:37 smithi016 ceph-mon[42541]: pgmap v344: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:38.348 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:37 smithi155 ceph-mon[47105]: pgmap v344: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:38.348 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:37 smithi189 ceph-mon[46919]: pgmap v344: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:38.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:30:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 -- 172.21.15.16:0/4002771943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc0696f0 msgr2=0x7f6bcc069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/4002771943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc069b70 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009990 tx=0x7f6bbc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 -- 172.21.15.16:0/4002771943 shutdown_connections 2024-05-08T23:30:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/4002771943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bcc10d7f0 0x7f6bcc10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/4002771943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc101e20 0x7f6bcc0691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/4002771943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc069b70 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 -- 172.21.15.16:0/4002771943 >> 172.21.15.16:0/4002771943 conn(0x7f6bcc076ba0 msgr2=0x7f6bcc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 -- 172.21.15.16:0/4002771943 shutdown_connections 2024-05-08T23:30:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.912+0000 7f6bd3db4700 1 -- 172.21.15.16:0/4002771943 wait complete. 2024-05-08T23:30:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 Processor -- start 2024-05-08T23:30:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 -- start start 2024-05-08T23:30:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 0x7f6bcc199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bcc10d7f0 0x7f6bcc19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc19e4a0 con 0x7f6bcc101e20 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc19e610 con 0x7f6bcc10d7f0 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd3db4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc19e7d0 con 0x7f6bcc0696f0 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd134f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 0x7f6bcc199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd134f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 0x7f6bcc199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42380/0 (socket says 172.21.15.16:42380) 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd134f700 1 -- 172.21.15.16:0/756119613 learned_addr learned my addr 172.21.15.16:0/756119613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.913+0000 7f6bd2351700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bcc10d7f0 0x7f6bcc19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd134f700 1 -- 172.21.15.16:0/756119613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc0696f0 msgr2=0x7f6bcc199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd134f700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd134f700 1 -- 172.21.15.16:0/756119613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bcc10d7f0 msgr2=0x7f6bcc19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd134f700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bcc10d7f0 0x7f6bcc19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd134f700 1 -- 172.21.15.16:0/756119613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0095f0 con 0x7f6bcc101e20 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd134f700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 0x7f6bcc199ac0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc004730 tx=0x7f6bbc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bc2ffd700 1 -- 172.21.15.16:0/756119613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc01d070 con 0x7f6bcc101e20 2024-05-08T23:30:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bc2ffd700 1 -- 172.21.15.16:0/756119613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bbc022470 con 0x7f6bcc101e20 2024-05-08T23:30:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc19ea50 con 0x7f6bcc101e20 2024-05-08T23:30:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bc2ffd700 1 -- 172.21.15.16:0/756119613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc017650 con 0x7f6bcc101e20 2024-05-08T23:30:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.914+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc1a62f0 con 0x7f6bcc101e20 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.917+0000 7f6bc2ffd700 1 -- 172.21.15.16:0/756119613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6bbc0225e0 con 0x7f6bcc101e20 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.917+0000 7f6bc2ffd700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6bb80a1310 0x7f6bb80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.917+0000 7f6bc2ffd700 1 -- 172.21.15.16:0/756119613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6bbc0ce340 con 0x7f6bcc101e20 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.917+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6bb80a1310 0x7f6bb80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 Processor -- start 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6bb80a1310 0x7f6bb80a37d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8005950 tx=0x7f6bc80058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 -- start start 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e00cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713d57150 con 0x558713e06720 2024-05-08T23:30:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713d54b10 con 0x558713e05e00 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.918+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713d50400 con 0x558713e00890 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e00350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42392/0 (socket says 172.21.15.16:42392) 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 -- 172.21.15.16:0/2576118497 learned_addr learned my addr 172.21.15.16:0/2576118497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd2351700 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e00cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 -- 172.21.15.16:0/2576118497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 msgr2=0x558713e00cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e00cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 -- 172.21.15.16:0/2576118497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 msgr2=0x558713e061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 -- 172.21.15.16:0/2576118497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558713bd13b0 con 0x558713e06720 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bd134f700 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e00350 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0a3110 tx=0x7f6bbc09b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5055bc5f189ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bb7fff700 1 -- 172.21.15.16:0/2576118497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc09b250 con 0x558713e06720 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bb7fff700 1 -- 172.21.15.16:0/2576118497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bbc09f080 con 0x558713e06720 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.919+0000 7f6bb7fff700 1 -- 172.21.15.16:0/2576118497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc09b6f0 con 0x558713e06720 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2576118497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 msgr2=0x558713e00350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e00350 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0a3110 tx=0x7f6bbc09b130 comp rx=0 tx=0).stop 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2576118497 shutdown_connections 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e00cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2576118497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e00350 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2576118497 >> 172.21.15.16:0/2576118497 conn(0x558713d5e210 msgr2=0x558713dfadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2576118497 shutdown_connections 2024-05-08T23:30:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.920+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2576118497 wait complete. 2024-05-08T23:30:38.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 -- start start 2024-05-08T23:30:38.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 Processor -- start 2024-05-08T23:30:38.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e0c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e0a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713cec560 con 0x558713e06720 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713cec200 con 0x558713e05e00 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713bd36a0 con 0x558713e00890 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bdff6eb80 10 client.0 _ll_register_callbacks cb 0x558713cf5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bd2351700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e0a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bd2351700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e0a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42402/0 (socket says 172.21.15.16:42402) 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.921+0000 7f6bd2351700 1 -- 172.21.15.16:0/3541270298 learned_addr learned my addr 172.21.15.16:0/3541270298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd134f700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e0c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd2351700 1 -- 172.21.15.16:0/3541270298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 msgr2=0x558713e0c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd2351700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e0c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd2351700 1 -- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 msgr2=0x558713e099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd2351700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd2351700 1 -- 172.21.15.16:0/3541270298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc1a62f0 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bd2351700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e0a0e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4012e20 tx=0x7f6bc4014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2c385a85a167de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bb57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bb57fa700 1 -- 172.21.15.16:0/3541270298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400e040 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bb57fa700 1 -- 172.21.15.16:0/3541270298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6bc4014b90 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558713ca9270 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bb57fa700 1 -- 172.21.15.16:0/3541270298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc4014d50 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bdff6eb80 10 client.17886 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.922+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558713b7db60 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bb57fa700 1 -- 172.21.15.16:0/3541270298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558713ca9270 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bb57fa700 1 -- 172.21.15.16:0/3541270298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bc401c020 con 0x558713e06720 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bdff6eb80 10 client.17886 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bdff6eb80 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bdff6eb80 20 client.17886 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e19f10 0x558713e1c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bdff6eb80 4 client.17886 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558713dcf810 con 0x558713e19f10 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.923+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e19f10 0x558713e1c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.924+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e19f10 0x558713e1c330 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf36df11ba42fdc server_cookie=d6954ea3ee6a1401 in_seq=0 out_seq=0 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.924+0000 7f6bb57fa700 10 client.17886 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.924+0000 7f6bb57fa700 1 -- 172.21.15.16:0/3541270298 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558713dcf810 con 0x558713e19f10 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.924+0000 7f6bb57fa700 10 client.17886 handle_command_reply: tid=0 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.924+0000 7f6bdff6eb80 1 client.17886 shutdown 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.924+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 msgr2=0x558713e0a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e0a0e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4012e20 tx=0x7f6bc4014600 comp rx=0 tx=0).stop 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 shutdown_connections 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e0a0e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e0c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e19f10 0x558713e1c330 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/3541270298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 >> 172.21.15.16:0/3541270298 conn(0x558713d5e210 msgr2=0x558713dfa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bb57fa700 0 client.17886 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:38.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 shutdown_connections 2024-05-08T23:30:38.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/3541270298 wait complete. 2024-05-08T23:30:38.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.925+0000 7f6bdff6eb80 20 client.17886 trim_cache size 0 max 16384 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.943+0000 7f6bdff6eb80 1 Processor -- start 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.943+0000 7f6bdff6eb80 1 -- start start 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 0x558713e00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713dfad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e06720 0x558713e03b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713df5a90 con 0x558713e00890 2024-05-08T23:30:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713df5800 con 0x558713e05e00 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713e21520 con 0x558713e06720 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 0x558713e00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 0x558713e00350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42412/0 (socket says 172.21.15.16:42412) 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 -- 172.21.15.16:0/977424514 learned_addr learned my addr 172.21.15.16:0/977424514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd2351700 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e06720 0x558713e03b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 -- 172.21.15.16:0/977424514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e06720 msgr2=0x558713e03b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e06720 0x558713e03b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 -- 172.21.15.16:0/977424514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 msgr2=0x558713dfad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713dfad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 -- 172.21.15.16:0/977424514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558713e00f40 con 0x558713e00890 2024-05-08T23:30:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 0x558713e00350 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8004270 tx=0x7f6bc8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88eef58ddce74d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bc0ff9700 1 -- 172.21.15.16:0/977424514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8017040 con 0x558713e00890 2024-05-08T23:30:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bc0ff9700 1 -- 172.21.15.16:0/977424514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc8028d70 con 0x558713e00890 2024-05-08T23:30:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.944+0000 7f6bc0ff9700 1 -- 172.21.15.16:0/977424514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8003aa0 con 0x558713e00890 2024-05-08T23:30:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/977424514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 msgr2=0x558713e00350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 0x558713e00350 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8004270 tx=0x7f6bc8015000 comp rx=0 tx=0).stop 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/977424514 shutdown_connections 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e06720 0x558713e03b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e00890 0x558713e00350 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/977424514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713dfad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/977424514 >> 172.21.15.16:0/977424514 conn(0x558713d5e210 msgr2=0x558713e15990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/977424514 shutdown_connections 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/977424514 wait complete. 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 -- start start 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.945+0000 7f6bdff6eb80 1 Processor -- start 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e01e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713e117b0 con 0x558713e06720 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713e02900 con 0x558713e05e00 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558713e02a70 con 0x558713e00890 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bdff6eb80 10 client.0 _ll_register_callbacks cb 0x7f6bc4012750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bd2351700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bd2351700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42428/0 (socket says 172.21.15.16:42428) 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bd2351700 1 -- 172.21.15.16:0/2719230876 learned_addr learned my addr 172.21.15.16:0/2719230876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bd134f700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e01e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.946+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd2351700 1 -- 172.21.15.16:0/2719230876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 msgr2=0x558713e110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd2351700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd2351700 1 -- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 msgr2=0x558713e01e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd2351700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e01e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd2351700 1 -- 172.21.15.16:0/2719230876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc8004030 con 0x558713e06720 2024-05-08T23:30:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd134f700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bd2351700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e023c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4012df0 tx=0x7f6bc4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf742d55cb066ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bb57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bb57fa700 1 -- 172.21.15.16:0/2719230876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400e040 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bb57fa700 1 -- 172.21.15.16:0/2719230876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6bc40049b0 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bbc09f080 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bb57fa700 1 -- 172.21.15.16:0/2719230876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bc4004b90 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 10 client.17892 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558713df74e0 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bb57fa700 1 -- 172.21.15.16:0/2719230876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bc4037080 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bb57fa700 1 -- 172.21.15.16:0/2719230876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6bc4014320 con 0x558713e06720 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 10 client.17892 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 20 client.17892 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.947+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e71780 0x558713e73b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bdff6eb80 4 client.17892 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558713d48ec0 con 0x558713e71780 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e71780 0x558713e73b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bb57fa700 10 client.17892 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bd1b50700 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e71780 0x558713e73b70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94967dcdf5e32b81 server_cookie=325c1f7ab2bd7fdd in_seq=0 out_seq=0 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bb57fa700 1 -- 172.21.15.16:0/2719230876 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558713d48ec0 con 0x558713e71780 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bb57fa700 10 client.17892 handle_command_reply: tid=0 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.948+0000 7f6bdff6eb80 1 client.17892 shutdown 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 msgr2=0x558713e023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e023c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4012df0 tx=0x7f6bc4009e80 comp rx=0 tx=0).stop 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 shutdown_connections 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558713e06720 0x558713e023c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558713e00890 0x558713e110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x558713e71780 0x558713e73b70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 --2- 172.21.15.16:0/2719230876 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558713e05e00 0x558713e01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 >> 172.21.15.16:0/2719230876 conn(0x558713d5e210 msgr2=0x558713e03b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bb57fa700 0 client.17892 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 shutdown_connections 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 1 -- 172.21.15.16:0/2719230876 wait complete. 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bdff6eb80 20 client.17892 trim_cache size 0 max 16384 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:30:38.956 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6bb80a1310 msgr2=0x7f6bb80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6bb80a1310 0x7f6bb80a37d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8005950 tx=0x7f6bc80058e0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 msgr2=0x7f6bcc199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 0x7f6bcc199ac0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc004730 tx=0x7f6bbc004760 comp rx=0 tx=0).stop 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 shutdown_connections 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.949+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6bb80a1310 0x7f6bb80a37d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.950+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6bcc10d7f0 0x7f6bcc19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.950+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bcc0696f0 0x7f6bcc199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.950+0000 7f6bd3db4700 1 --2- 172.21.15.16:0/756119613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc101e20 0x7f6bcc199ac0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.950+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 >> 172.21.15.16:0/756119613 conn(0x7f6bcc076ba0 msgr2=0x7f6bcc10e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.950+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 shutdown_connections 2024-05-08T23:30:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:38.950+0000 7f6bd3db4700 1 -- 172.21.15.16:0/756119613 wait complete. 2024-05-08T23:30:38.963 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:30:38.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:30:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 -- 172.21.15.16:0/2945218426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f84100930 msgr2=0x7f8f84104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 --2- 172.21.15.16:0/2945218426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f84100930 0x7f8f84104970 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009990 tx=0x7f8f78009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 -- 172.21.15.16:0/2945218426 shutdown_connections 2024-05-08T23:30:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 --2- 172.21.15.16:0/2945218426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f841050a0 0x7f8f8410cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 --2- 172.21.15.16:0/2945218426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f840fff80 0x7f8f84100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 --2- 172.21.15.16:0/2945218426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f84100930 0x7f8f84104970 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 -- 172.21.15.16:0/2945218426 >> 172.21.15.16:0/2945218426 conn(0x7f8f840fb840 msgr2=0x7f8f840fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 -- 172.21.15.16:0/2945218426 shutdown_connections 2024-05-08T23:30:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.100+0000 7f8f8900f700 1 -- 172.21.15.16:0/2945218426 wait complete. 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 Processor -- start 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 -- start start 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f840fff80 0x7f8f841971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f841050a0 0x7f8f841976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 0x7f8f8419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419c510 con 0x7f8f8419ba90 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419c680 con 0x7f8f840fff80 2024-05-08T23:30:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.101+0000 7f8f8900f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8419c840 con 0x7f8f841050a0 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 0x7f8f8419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 0x7f8f8419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42458/0 (socket says 172.21.15.16:42458) 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 -- 172.21.15.16:0/1531807947 learned_addr learned my addr 172.21.15.16:0/1531807947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f82d9d700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f840fff80 0x7f8f841971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8259c700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f841050a0 0x7f8f841976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 -- 172.21.15.16:0/1531807947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f841050a0 msgr2=0x7f8f841976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f841050a0 0x7f8f841976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 -- 172.21.15.16:0/1531807947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f840fff80 msgr2=0x7f8f841971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f840fff80 0x7f8f841971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 -- 172.21.15.16:0/1531807947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780095f0 con 0x7f8f8419ba90 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8359e700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 0x7f8f8419bf10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c007d70 tx=0x7f8f7c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00ac90 con 0x7f8f8419ba90 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f841a3da0 con 0x7f8f8419ba90 2024-05-08T23:30:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f7c00adf0 con 0x7f8f8419ba90 2024-05-08T23:30:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0115e0 con 0x7f8f8419ba90 2024-05-08T23:30:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.102+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f841a4230 con 0x7f8f8419ba90 2024-05-08T23:30:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.103+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f84109380 con 0x7f8f8419ba90 2024-05-08T23:30:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.105+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f7c004ad0 con 0x7f8f8419ba90 2024-05-08T23:30:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.105+0000 7f8f6bfff700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8f6c0a1290 0x7f8f6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.106+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f8f7c0cd460 con 0x7f8f8419ba90 2024-05-08T23:30:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.106+0000 7f8f82d9d700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8f6c0a1290 0x7f8f6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.106+0000 7f8f82d9d700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8f6c0a1290 0x7f8f6c0a3750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005950 tx=0x7f8f7400b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.109+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8f7c096620 con 0x7f8f8419ba90 2024-05-08T23:30:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.380+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f84198550 con 0x7f8f8419ba90 2024-05-08T23:30:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.380+0000 7f8f6bfff700 1 -- 172.21.15.16:0/1531807947 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f8f7c099c40 con 0x7f8f8419ba90 2024-05-08T23:30:39.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:30:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8f6c0a1290 msgr2=0x7f8f6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8f6c0a1290 0x7f8f6c0a3750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005950 tx=0x7f8f7400b410 comp rx=0 tx=0).stop 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 msgr2=0x7f8f8419bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 0x7f8f8419bf10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c007d70 tx=0x7f8f7c00a340 comp rx=0 tx=0).stop 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 shutdown_connections 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f8f6c0a1290 0x7f8f6c0a3750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f8419ba90 0x7f8f8419bf10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f840fff80 0x7f8f841971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 --2- 172.21.15.16:0/1531807947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f841050a0 0x7f8f841976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 >> 172.21.15.16:0/1531807947 conn(0x7f8f840fb840 msgr2=0x7f8f840696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 shutdown_connections 2024-05-08T23:30:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.383+0000 7f8f8900f700 1 -- 172.21.15.16:0/1531807947 wait complete. 2024-05-08T23:30:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:30:39.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:30:39.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 -- 172.21.15.16:0/2234753579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484069770 msgr2=0x7f7484069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 --2- 172.21.15.16:0/2234753579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484069770 0x7f7484069b50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7474009990 tx=0x7f7474009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 -- 172.21.15.16:0/2234753579 shutdown_connections 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 --2- 172.21.15.16:0/2234753579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7484106280 0x7f748410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 --2- 172.21.15.16:0/2234753579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484069770 0x7f7484069b50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 --2- 172.21.15.16:0/2234753579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748406a120 0x7f7484105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 -- 172.21.15.16:0/2234753579 >> 172.21.15.16:0/2234753579 conn(0x7f7484076fa0 msgr2=0x7f74840773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 -- 172.21.15.16:0/2234753579 shutdown_connections 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.535+0000 7f748a29f700 1 -- 172.21.15.16:0/2234753579 wait complete. 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 Processor -- start 2024-05-08T23:30:39.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 -- start start 2024-05-08T23:30:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7484069770 0x7f74840ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 0x7f7484100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7484100890 0x7f74841a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841a5b30 con 0x7f7484106280 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841a5c70 con 0x7f7484100890 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f748a29f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841a5e30 con 0x7f7484069770 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f74837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 0x7f7484100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f74837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 0x7f7484100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42474/0 (socket says 172.21.15.16:42474) 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f74837fe700 1 -- 172.21.15.16:0/3287250768 learned_addr learned my addr 172.21.15.16:0/3287250768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.536+0000 7f7483fff700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7484069770 0x7f74840ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f748883c700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7484100890 0x7f74841a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74837fe700 1 -- 172.21.15.16:0/3287250768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7484069770 msgr2=0x7f74840ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74837fe700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7484069770 0x7f74840ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74837fe700 1 -- 172.21.15.16:0/3287250768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7484100890 msgr2=0x7f74841a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74837fe700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7484100890 0x7f74841a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74837fe700 1 -- 172.21.15.16:0/3287250768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740095f0 con 0x7f7484106280 2024-05-08T23:30:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74837fe700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 0x7f7484100350 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f747800d950 tx=0x7f747800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74817fa700 1 -- 172.21.15.16:0/3287250768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478004220 con 0x7f7484106280 2024-05-08T23:30:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74841a6110 con 0x7f7484106280 2024-05-08T23:30:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74817fa700 1 -- 172.21.15.16:0/3287250768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7478007620 con 0x7f7484106280 2024-05-08T23:30:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f74817fa700 1 -- 172.21.15.16:0/3287250768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478005560 con 0x7f7484106280 2024-05-08T23:30:39.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.537+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841a6620 con 0x7f7484106280 2024-05-08T23:30:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.539+0000 7f74817fa700 1 -- 172.21.15.16:0/3287250768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7478004750 con 0x7f7484106280 2024-05-08T23:30:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.540+0000 7f74817fa700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f746c0a1290 0x7f746c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.541+0000 7f7483fff700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f746c0a1290 0x7f746c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.541+0000 7f74817fa700 1 -- 172.21.15.16:0/3287250768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f74780ccde0 con 0x7f7484106280 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.541+0000 7f7483fff700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f746c0a1290 0x7f746c0a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f747400b2e0 tx=0x7f7474005040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.541+0000 7f7496459b80 1 Processor -- start 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.541+0000 7f7496459b80 1 -- start start 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7496459b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7496459b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dbc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7496459b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dbccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7496459b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64d13150 con 0x55ed64dc2720 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7496459b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64d10b10 con 0x55ed64dbc890 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7496459b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64d0c400 con 0x55ed64dc1e00 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dbc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dbc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42482/0 (socket says 172.21.15.16:42482) 2024-05-08T23:30:39.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 -- 172.21.15.16:0/1682797373 learned_addr learned my addr 172.21.15.16:0/1682797373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f7483fff700 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f748883c700 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dbccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 -- 172.21.15.16:0/1682797373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 msgr2=0x55ed64dc21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 -- 172.21.15.16:0/1682797373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 msgr2=0x55ed64dbccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dbccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 -- 172.21.15.16:0/1682797373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed64b8d3b0 con 0x55ed64dc2720 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f74837fe700 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dbc350 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7478005cd0 tx=0x7f747809a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919a702b1e0cce53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f746a7fc700 1 -- 172.21.15.16:0/1682797373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747809d980 con 0x55ed64dc2720 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f746a7fc700 1 -- 172.21.15.16:0/1682797373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74780b9070 con 0x55ed64dc2720 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.542+0000 7f746a7fc700 1 -- 172.21.15.16:0/1682797373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747809a060 con 0x55ed64dc2720 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 -- 172.21.15.16:0/1682797373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 msgr2=0x55ed64dbc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dbc350 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7478005cd0 tx=0x7f747809a2d0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 -- 172.21.15.16:0/1682797373 shutdown_connections 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dbccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 --2- 172.21.15.16:0/1682797373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dbc350 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 -- 172.21.15.16:0/1682797373 >> 172.21.15.16:0/1682797373 conn(0x55ed64d1a210 msgr2=0x55ed64db6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 -- 172.21.15.16:0/1682797373 shutdown_connections 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.543+0000 7f7496459b80 1 -- 172.21.15.16:0/1682797373 wait complete. 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 -- start start 2024-05-08T23:30:39.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 Processor -- start 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dc5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dc65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64ca8560 con 0x55ed64dc2720 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64ca8200 con 0x55ed64dbc890 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64b8f6a0 con 0x55ed64dc1e00 2024-05-08T23:30:39.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7496459b80 10 client.0 _ll_register_callbacks cb 0x55ed64cb1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f748883c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dc65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f748883c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dc65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42486/0 (socket says 172.21.15.16:42486) 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f748883c700 1 -- 172.21.15.16:0/3609895487 learned_addr learned my addr 172.21.15.16:0/3609895487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.544+0000 7f7483fff700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dc5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f74837fe700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f748883c700 1 -- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 msgr2=0x55ed64dc6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f748883c700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f748883c700 1 -- 172.21.15.16:0/3609895487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 msgr2=0x55ed64dc5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f748883c700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dc5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f748883c700 1 -- 172.21.15.16:0/3609895487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841a6620 con 0x55ed64dc2720 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f748883c700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dc65c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f747c007b10 tx=0x7f747c007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1386bb4c96ca9f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f74637fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f74637fe700 1 -- 172.21.15.16:0/3609895487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e040 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f74637fe700 1 -- 172.21.15.16:0/3609895487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f747c016900 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed64c65270 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f74637fe700 1 -- 172.21.15.16:0/3609895487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747c016ac0 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f7496459b80 10 client.17910 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.545+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ed64b39b60 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f74637fe700 1 -- 172.21.15.16:0/3609895487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ed64c65270 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f74637fe700 1 -- 172.21.15.16:0/3609895487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f747c01ecc0 con 0x55ed64dc2720 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7496459b80 10 client.17910 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7496459b80 10 client.17910 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7496459b80 20 client.17910 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7496459b80 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64dd62a0 0x55ed64dd86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7496459b80 4 client.17910 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed64d8b810 con 0x55ed64dd62a0 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.546+0000 7f7483fff700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64dd62a0 0x55ed64dd86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7483fff700 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64dd62a0 0x55ed64dd86c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff7c6cfc05c08eb1 server_cookie=5c55d81f1e69fceb in_seq=0 out_seq=0 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f74637fe700 10 client.17910 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f74637fe700 1 -- 172.21.15.16:0/3609895487 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed64d8b810 con 0x55ed64dd62a0 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f74637fe700 10 client.17910 handle_command_reply: tid=0 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7496459b80 1 client.17910 shutdown 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 msgr2=0x55ed64dc65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7496459b80 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dc65c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f747c007b10 tx=0x7f747c007b40 comp rx=0 tx=0).stop 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 shutdown_connections 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7496459b80 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc2720 0x55ed64dc65c0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.547+0000 7f7496459b80 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64dd62a0 0x55ed64dd86c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f7496459b80 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dc5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f7496459b80 1 --2- 172.21.15.16:0/3609895487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dc6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f74637fe700 0 client.17910 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 >> 172.21.15.16:0/3609895487 conn(0x55ed64d1a210 msgr2=0x55ed64db65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 shutdown_connections 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f7496459b80 1 -- 172.21.15.16:0/3609895487 wait complete. 2024-05-08T23:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.548+0000 7f7496459b80 20 client.17910 trim_cache size 0 max 16384 2024-05-08T23:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.561+0000 7f7496459b80 1 Processor -- start 2024-05-08T23:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 -- start start 2024-05-08T23:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 0x55ed64dbc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dd1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dc2720 0x55ed64dc1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64db8890 con 0x55ed64dbc890 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64db3dc0 con 0x55ed64dc2720 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7496459b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64ddd890 con 0x55ed64dc1e00 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 0x55ed64dbc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 0x55ed64dbc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42494/0 (socket says 172.21.15.16:42494) 2024-05-08T23:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 -- 172.21.15.16:0/2648648012 learned_addr learned my addr 172.21.15.16:0/2648648012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f74837fe700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dd1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 -- 172.21.15.16:0/2648648012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 msgr2=0x55ed64dd1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f748883c700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dc2720 0x55ed64dc1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dd1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 -- 172.21.15.16:0/2648648012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dc2720 msgr2=0x55ed64dc1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dc2720 0x55ed64dc1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 -- 172.21.15.16:0/2648648012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed64db34e0 con 0x55ed64dbc890 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f74837fe700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dd1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f748883c700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dc2720 0x55ed64dc1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.562+0000 7f7483fff700 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 0x55ed64dbc350 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7474009990 tx=0x7f7474023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c05c3e86cecf5e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f746affd700 1 -- 172.21.15.16:0/2648648012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740056e0 con 0x55ed64dbc890 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f746affd700 1 -- 172.21.15.16:0/2648648012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f747401e040 con 0x55ed64dbc890 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f746affd700 1 -- 172.21.15.16:0/2648648012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740367a0 con 0x55ed64dbc890 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 -- 172.21.15.16:0/2648648012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 msgr2=0x55ed64dbc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 0x55ed64dbc350 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7474009990 tx=0x7f7474023000 comp rx=0 tx=0).stop 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 -- 172.21.15.16:0/2648648012 shutdown_connections 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dc2720 0x55ed64dc1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dbc890 0x55ed64dbc350 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 --2- 172.21.15.16:0/2648648012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc1e00 0x55ed64dd1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 -- 172.21.15.16:0/2648648012 >> 172.21.15.16:0/2648648012 conn(0x55ed64d1a210 msgr2=0x55ed64dc0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 -- 172.21.15.16:0/2648648012 shutdown_connections 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.563+0000 7f7496459b80 1 -- 172.21.15.16:0/2648648012 wait complete. 2024-05-08T23:30:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 -- start start 2024-05-08T23:30:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 Processor -- start 2024-05-08T23:30:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dcc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 0x55ed64dccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc2720 0x55ed64e2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64dcd550 con 0x55ed64dc1e00 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64e2bca0 con 0x55ed64dbc890 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed64e2bde0 con 0x55ed64dc2720 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f7496459b80 10 client.0 _ll_register_callbacks cb 0x55ed64cb1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f74837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 0x55ed64dccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f74837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 0x55ed64dccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42504/0 (socket says 172.21.15.16:42504) 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.564+0000 7f74837fe700 1 -- 172.21.15.16:0/2479086049 learned_addr learned my addr 172.21.15.16:0/2479086049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f7483fff700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dcc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f748883c700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc2720 0x55ed64e2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74837fe700 1 -- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc2720 msgr2=0x55ed64e2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74837fe700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc2720 0x55ed64e2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74837fe700 1 -- 172.21.15.16:0/2479086049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 msgr2=0x55ed64dcc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74837fe700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dcc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74837fe700 1 -- 172.21.15.16:0/2479086049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748410e670 con 0x55ed64dc1e00 2024-05-08T23:30:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74837fe700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 0x55ed64dccc70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f7478096050 tx=0x7f7478099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73ae29c98ff6678 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74637fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74637fe700 1 -- 172.21.15.16:0/2479086049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780b2070 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74637fe700 1 -- 172.21.15.16:0/2479086049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f74780a1fd0 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74780b9070 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f74637fe700 1 -- 172.21.15.16:0/2479086049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7478099240 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f7496459b80 10 client.17916 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.565+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ed64cd2560 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f74637fe700 1 -- 172.21.15.16:0/2479086049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74780b9070 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f74637fe700 1 -- 172.21.15.16:0/2479086049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74780a1b40 con 0x55ed64dc1e00 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7496459b80 10 client.17916 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7496459b80 10 client.17916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7496459b80 20 client.17916 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7496459b80 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64e2d0f0 0x55ed64e2f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7496459b80 4 client.17916 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed64d04ec0 con 0x55ed64e2d0f0 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7483fff700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64e2d0f0 0x55ed64e2f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f74637fe700 10 client.17916 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.566+0000 7f7483fff700 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64e2d0f0 0x55ed64e2f510 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6edc663654052937 server_cookie=c7b217e3dbb7af96 in_seq=0 out_seq=0 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f74637fe700 1 -- 172.21.15.16:0/2479086049 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed64d04ec0 con 0x55ed64e2d0f0 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f74637fe700 10 client.17916 handle_command_reply: tid=0 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 client.17916 shutdown 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 msgr2=0x55ed64dccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 0x55ed64dccc70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f7478096050 tx=0x7f7478099a50 comp rx=0 tx=0).stop 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 shutdown_connections 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed64dc2720 0x55ed64e2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x55ed64e2d0f0 0x55ed64e2f510 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55ed64dbc890 0x55ed64dcc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 --2- 172.21.15.16:0/2479086049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed64dc1e00 0x55ed64dccc70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f74637fe700 0 client.17916 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 >> 172.21.15.16:0/2479086049 conn(0x55ed64d1a210 msgr2=0x55ed64dc2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 shutdown_connections 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 1 -- 172.21.15.16:0/2479086049 wait complete. 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.567+0000 7f7496459b80 20 client.17916 trim_cache size 0 max 16384 2024-05-08T23:30:39.575 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f746c0a1290 msgr2=0x7f746c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f746c0a1290 0x7f746c0a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f747400b2e0 tx=0x7f7474005040 comp rx=0 tx=0).stop 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 msgr2=0x7f7484100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 0x7f7484100350 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f747800d950 tx=0x7f747800dc60 comp rx=0 tx=0).stop 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 shutdown_connections 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f746c0a1290 0x7f746c0a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7484100890 0x7f74841a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7484069770 0x7f74840ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 --2- 172.21.15.16:0/3287250768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7484106280 0x7f7484100350 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 >> 172.21.15.16:0/3287250768 conn(0x7f7484076fa0 msgr2=0x7f74840fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 shutdown_connections 2024-05-08T23:30:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.568+0000 7f748a29f700 1 -- 172.21.15.16:0/3287250768 wait complete. 2024-05-08T23:30:39.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 -- 172.21.15.16:0/1209255152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c069d90 msgr2=0x7f0c9c105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 --2- 172.21.15.16:0/1209255152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c069d90 0x7f0c9c105b70 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009990 tx=0x7f0c90009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 -- 172.21.15.16:0/1209255152 shutdown_connections 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 --2- 172.21.15.16:0/1209255152 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 --2- 172.21.15.16:0/1209255152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c0693e0 0x7f0c9c0697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 --2- 172.21.15.16:0/1209255152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c069d90 0x7f0c9c105b70 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 -- 172.21.15.16:0/1209255152 >> 172.21.15.16:0/1209255152 conn(0x7f0c9c076ba0 msgr2=0x7f0c9c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 -- 172.21.15.16:0/1209255152 shutdown_connections 2024-05-08T23:30:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 -- 172.21.15.16:0/1209255152 wait complete. 2024-05-08T23:30:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.724+0000 7f0ca293e700 1 Processor -- start 2024-05-08T23:30:39.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 -- start start 2024-05-08T23:30:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c9c0693e0 0x7f0c9c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c19de90 0x7f0c9c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c19e910 con 0x7f0c9c1062a0 2024-05-08T23:30:39.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c19ea80 con 0x7f0c9c0693e0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca293e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c19ec40 con 0x7f0c9c19de90 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42524/0 (socket says 172.21.15.16:42524) 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/3121854153 learned_addr learned my addr 172.21.15.16:0/3121854153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c9c0693e0 0x7f0c9c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c19de90 0x7f0c9c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/3121854153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c19de90 msgr2=0x7f0c9c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c19de90 0x7f0c9c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/3121854153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c9c0693e0 msgr2=0x7f0c9c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c9c0693e0 0x7f0c9c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/3121854153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c900095f0 con 0x7f0c9c1062a0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.725+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c19de90 0x7f0c9c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.726+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c199b40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f0c90004730 tx=0x7f0c90009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.726+0000 7f0c997fa700 1 -- 172.21.15.16:0/3121854153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9001d070 con 0x7f0c9c1062a0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.726+0000 7f0c997fa700 1 -- 172.21.15.16:0/3121854153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c90004ca0 con 0x7f0c9c1062a0 2024-05-08T23:30:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.726+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c1a6140 con 0x7f0c9c1062a0 2024-05-08T23:30:39.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.726+0000 7f0c997fa700 1 -- 172.21.15.16:0/3121854153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000f730 con 0x7f0c9c1062a0 2024-05-08T23:30:39.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.726+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c1a65c0 con 0x7f0c9c1062a0 2024-05-08T23:30:39.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.728+0000 7f0c997fa700 1 -- 172.21.15.16:0/3121854153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0c90004810 con 0x7f0c9c1062a0 2024-05-08T23:30:39.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.729+0000 7f0c997fa700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0c840a1310 0x7f0c840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.729+0000 7f0c997fa700 1 -- 172.21.15.16:0/3121854153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0c900cdef0 con 0x7f0c9c1062a0 2024-05-08T23:30:39.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.729+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0c840a1310 0x7f0c840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0c840a1310 0x7f0c840a37d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009a10 tx=0x7f0c8c009190 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 Processor -- start 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 -- start start 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a800501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a8004a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a8004acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ffa1150 con 0x558a8004fe00 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ff9eb10 con 0x558a8004a890 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ff9a400 con 0x558a80050720 2024-05-08T23:30:39.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0c9bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a800501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0c9bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a800501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42528/0 (socket says 172.21.15.16:42528) 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.730+0000 7f0c9bfff700 1 -- 172.21.15.16:0/955056608 learned_addr learned my addr 172.21.15.16:0/955056608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a8004a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a8004acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9bfff700 1 -- 172.21.15.16:0/955056608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 msgr2=0x558a8004a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a8004a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9bfff700 1 -- 172.21.15.16:0/955056608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 msgr2=0x558a8004acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a8004acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9bfff700 1 -- 172.21.15.16:0/955056608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7fe1b3b0 con 0x558a8004fe00 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a8004acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a800501e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00ea00 tx=0x7f0c8c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b8b4518bada23e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c827fc700 1 -- 172.21.15.16:0/955056608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00eec0 con 0x558a8004fe00 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c827fc700 1 -- 172.21.15.16:0/955056608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c8c015910 con 0x558a8004fe00 2024-05-08T23:30:39.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.731+0000 7f0c827fc700 1 -- 172.21.15.16:0/955056608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0138b0 con 0x558a8004fe00 2024-05-08T23:30:39.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/955056608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 msgr2=0x558a800501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a800501e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00ea00 tx=0x7f0c8c011000 comp rx=0 tx=0).stop 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/955056608 shutdown_connections 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a8004acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a800501e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/955056608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a8004a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/955056608 >> 172.21.15.16:0/955056608 conn(0x558a7ffa8210 msgr2=0x558a80044df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/955056608 shutdown_connections 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/955056608 wait complete. 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.732+0000 7f0caeaf8b80 1 -- start start 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 Processor -- start 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80053b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a800540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80054610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ff36560 con 0x558a8004a890 2024-05-08T23:30:39.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7ff36200 con 0x558a80050720 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7fe1d6a0 con 0x558a8004fe00 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0caeaf8b80 10 client.0 _ll_register_callbacks cb 0x558a7ff3fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0c9b7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a800540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0c9b7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a800540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:53374/0 (socket says 172.21.15.16:53374) 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.733+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/958384013 learned_addr learned my addr 172.21.15.16:0/958384013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80053b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/958384013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 msgr2=0x558a80054610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80054610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80054610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/958384013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 msgr2=0x558a80053b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80053b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/958384013 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c00e6e0 con 0x558a8004fe00 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80054610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:39.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a800540d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c9009ac80 tx=0x7f0c9009a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd703bc1b13ae06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/958384013 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9009b3f0 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a7fef3270 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/958384013 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0c9009acf0 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.734+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/958384013 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c9009eb20 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 10 client.27556 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558a7fdc7b60 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/958384013 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c900b7080 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/958384013 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c90096ae0 con 0x558a8004fe00 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 10 client.27556 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 10 client.27556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 20 client.27556 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a80063f50 0x558a80066340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 4 client.27556 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a80019810 con 0x558a80063f50 2024-05-08T23:30:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.735+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a80063f50 0x558a80066340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.736+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a80063f50 0x558a80066340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be3cc61916f32033 server_cookie=9a3d07f250b7cd5a in_seq=0 out_seq=0 2024-05-08T23:30:39.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.736+0000 7f0c7b7fe700 10 client.27556 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.736+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/958384013 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a80019810 con 0x558a80063f50 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0c7b7fe700 10 client.27556 handle_command_reply: tid=0 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 client.27556 shutdown 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 msgr2=0x558a800540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a800540d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c9009ac80 tx=0x7f0c9009a8b0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 shutdown_connections 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a80063f50 0x558a80066340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80054610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0c7b7fe700 0 client.27556 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80053b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/958384013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a800540d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 >> 172.21.15.16:0/958384013 conn(0x558a7ffa8210 msgr2=0x558a800445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 shutdown_connections 2024-05-08T23:30:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/958384013 wait complete. 2024-05-08T23:30:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.737+0000 7f0caeaf8b80 20 client.27556 trim_cache size 0 max 16384 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 Processor -- start 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 -- start start 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80046a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a8005fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80044d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a80045c90 con 0x558a8004a890 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a80063d30 con 0x558a80050720 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.751+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a80056b60 con 0x558a8004fe00 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80046a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80046a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42536/0 (socket says 172.21.15.16:42536) 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 -- 172.21.15.16:0/1278513158 learned_addr learned my addr 172.21.15.16:0/1278513158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a8005fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80044d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 -- 172.21.15.16:0/1278513158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 msgr2=0x558a8005fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a8005fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 -- 172.21.15.16:0/1278513158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 msgr2=0x558a80044d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80044d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 -- 172.21.15.16:0/1278513158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7ff60560 con 0x558a8004a890 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a8005fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80046a20 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0134c0 tx=0x7f0c8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cc5773bdd2b014 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c82ffd700 1 -- 172.21.15.16:0/1278513158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c030d70 con 0x558a8004a890 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c82ffd700 1 -- 172.21.15.16:0/1278513158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c8c030330 con 0x558a8004a890 2024-05-08T23:30:39.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.752+0000 7f0c82ffd700 1 -- 172.21.15.16:0/1278513158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0137c0 con 0x558a8004a890 2024-05-08T23:30:39.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/1278513158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 msgr2=0x558a80046a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80046a20 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0134c0 tx=0x7f0c8c015000 comp rx=0 tx=0).stop 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/1278513158 shutdown_connections 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a80050720 0x558a80044d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004a890 0x558a80046a20 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/1278513158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a8004fe00 0x558a8005fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/1278513158 >> 172.21.15.16:0/1278513158 conn(0x558a7ffa8210 msgr2=0x558a80052460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/1278513158 shutdown_connections 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/1278513158 wait complete. 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.753+0000 7f0caeaf8b80 1 -- start start 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 Processor -- start 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a80059150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a80059690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a80059bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a800b94b0 con 0x558a8004fe00 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a800b9620 con 0x558a8004a890 2024-05-08T23:30:39.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a800b97e0 con 0x558a80050720 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0caeaf8b80 10 client.0 _ll_register_callbacks cb 0x558a7ff3fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a80059690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a80059690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42544/0 (socket says 172.21.15.16:42544) 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/2989173046 learned_addr learned my addr 172.21.15.16:0/2989173046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a80059150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0ca0edb700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a80059bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 msgr2=0x558a80059bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a80059bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/2989173046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 msgr2=0x558a80059150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a80059150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.754+0000 7f0c9b7fe700 1 -- 172.21.15.16:0/2989173046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a7fe1b3b0 con 0x558a8004fe00 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c9b7fe700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a80059690 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0c90096f00 tx=0x7f0c900cb370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93572e6a342db11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/2989173046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9009b3f0 con 0x558a8004fe00 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/2989173046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0c9009acf0 con 0x558a8004fe00 2024-05-08T23:30:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c8c015910 con 0x558a8004fe00 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/2989173046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c9009eb20 con 0x558a8004fe00 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 10 client.17934 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558a7fe1fcd0 con 0x558a8004fe00 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/2989173046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c8c015910 con 0x558a8004fe00 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/2989173046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0c90099c50 con 0x558a8004fe00 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 10 client.17934 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 10 client.17934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 20 client.17934 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a800baab0 0x558a800bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 4 client.17934 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.755+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a7ff92ec0 con 0x558a800baab0 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a800baab0 0x558a800bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0c9bfff700 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a800baab0 0x558a800bcea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d83aff26b1c60b6 server_cookie=a47a21030a34c9c5 in_seq=0 out_seq=0 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0c7b7fe700 10 client.17934 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/2989173046 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a7ff92ec0 con 0x558a800baab0 2024-05-08T23:30:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0c7b7fe700 10 client.17934 handle_command_reply: tid=0 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 client.17934 shutdown 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 msgr2=0x558a80059690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a80059690 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0c90096f00 tx=0x7f0c900cb370 comp rx=0 tx=0).stop 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 shutdown_connections 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558a80050720 0x558a80059bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x558a800baab0 0x558a800bcea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558a8004a890 0x558a80059150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 --2- 172.21.15.16:0/2989173046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a8004fe00 0x558a80059690 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0c7b7fe700 0 client.17934 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.756+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 >> 172.21.15.16:0/2989173046 conn(0x558a7ffa8210 msgr2=0x558a8004ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 shutdown_connections 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0caeaf8b80 1 -- 172.21.15.16:0/2989173046 wait complete. 2024-05-08T23:30:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0caeaf8b80 20 client.17934 trim_cache size 0 max 16384 2024-05-08T23:30:39.763 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:30:39.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0c840a1310 msgr2=0x7f0c840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0c840a1310 0x7f0c840a37d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009a10 tx=0x7f0c8c009190 comp rx=0 tx=0).stop 2024-05-08T23:30:39.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 msgr2=0x7f0c9c199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c199b40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f0c90004730 tx=0x7f0c90009580 comp rx=0 tx=0).stop 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 shutdown_connections 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f0c840a1310 0x7f0c840a37d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c9c19de90 0x7f0c9c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c9c0693e0 0x7f0c9c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 --2- 172.21.15.16:0/3121854153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c9c1062a0 0x7f0c9c199b40 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 >> 172.21.15.16:0/3121854153 conn(0x7f0c9c076ba0 msgr2=0x7f0c9c10c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 shutdown_connections 2024-05-08T23:30:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.757+0000 7f0ca293e700 1 -- 172.21.15.16:0/3121854153 wait complete. 2024-05-08T23:30:39.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 -- 172.21.15.16:0/1313416984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 msgr2=0x7f5ad8100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 --2- 172.21.15.16:0/1313416984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad8100440 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009990 tx=0x7f5ad4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 -- 172.21.15.16:0/1313416984 shutdown_connections 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 --2- 172.21.15.16:0/1313416984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad8105180 0x7f5ad810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 --2- 172.21.15.16:0/1313416984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad8100440 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 --2- 172.21.15.16:0/1313416984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8100a10 0x7f5ad8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 -- 172.21.15.16:0/1313416984 >> 172.21.15.16:0/1313416984 conn(0x7f5ad8076fa0 msgr2=0x7f5ad80773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 -- 172.21.15.16:0/1313416984 shutdown_connections 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 -- 172.21.15.16:0/1313416984 wait complete. 2024-05-08T23:30:39.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 Processor -- start 2024-05-08T23:30:39.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.911+0000 7f5adf617700 1 -- start start 2024-05-08T23:30:39.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adf617700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adf617700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8105180 0x7f5ad8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adf617700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad819ddd0 0x7f5ad819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adf617700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad819e850 con 0x7f5ad8100060 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adf617700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad819e9c0 con 0x7f5ad8105180 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adf617700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad819eb80 con 0x7f5ad819ddd0 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad81994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42560/0 (socket says 172.21.15.16:42560) 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 -- 172.21.15.16:0/225341556 learned_addr learned my addr 172.21.15.16:0/225341556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8105180 0x7f5ad8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5addbb4700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad819ddd0 0x7f5ad819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 -- 172.21.15.16:0/225341556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad819ddd0 msgr2=0x7f5ad819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad819ddd0 0x7f5ad819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 -- 172.21.15.16:0/225341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8105180 msgr2=0x7f5ad8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8105180 0x7f5ad8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 -- 172.21.15.16:0/225341556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40095f0 con 0x7f5ad8100060 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8105180 0x7f5ad8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.912+0000 7f5add3b3700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad81994f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c000 tx=0x7f5ad4004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.913+0000 7f5ace7fc700 1 -- 172.21.15.16:0/225341556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401d070 con 0x7f5ad8100060 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.913+0000 7f5ace7fc700 1 -- 172.21.15.16:0/225341556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ad400fdc0 con 0x7f5ad8100060 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.913+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad81a6080 con 0x7f5ad8100060 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.913+0000 7f5ace7fc700 1 -- 172.21.15.16:0/225341556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40173f0 con 0x7f5ad8100060 2024-05-08T23:30:39.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.913+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad81a64a0 con 0x7f5ad8100060 2024-05-08T23:30:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.916+0000 7f5ace7fc700 1 -- 172.21.15.16:0/225341556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5ad4022670 con 0x7f5ad8100060 2024-05-08T23:30:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.916+0000 7f5ace7fc700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5ac40a1290 0x7f5ac40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.916+0000 7f5ace7fc700 1 -- 172.21.15.16:0/225341556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5ad40ce4f0 con 0x7f5ad8100060 2024-05-08T23:30:39.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.916+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5ac40a1290 0x7f5ac40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.917+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5ac40a1290 0x7f5ac40a3750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ad819aa40 tx=0x7f5ac8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.917+0000 7f5aeb7d1b80 1 Processor -- start 2024-05-08T23:30:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.917+0000 7f5aeb7d1b80 1 -- start start 2024-05-08T23:30:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.917+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0ed1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.917+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.917+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0e7890 0x564e2d0e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d03e150 con 0x564e2d0ece00 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d03bb10 con 0x564e2d0e7890 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d037400 con 0x564e2d0ed720 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0ed1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0ed1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42562/0 (socket says 172.21.15.16:42562) 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5adcbb2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 -- 172.21.15.16:0/2978142493 learned_addr learned my addr 172.21.15.16:0/2978142493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5addbb4700 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0e7890 0x564e2d0e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 -- 172.21.15.16:0/2978142493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 msgr2=0x564e2d0e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 -- 172.21.15.16:0/2978142493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0e7890 msgr2=0x564e2d0e7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0e7890 0x564e2d0e7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 -- 172.21.15.16:0/2978142493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2ceb83b0 con 0x564e2d0ece00 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5add3b3700 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0ed1e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40a3150 tx=0x7f5ad409ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922dcfbb328d1e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5ac37fe700 1 -- 172.21.15.16:0/2978142493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4095ea0 con 0x564e2d0ece00 2024-05-08T23:30:39.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5ac37fe700 1 -- 172.21.15.16:0/2978142493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ad409b290 con 0x564e2d0ece00 2024-05-08T23:30:39.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.918+0000 7f5ac37fe700 1 -- 172.21.15.16:0/2978142493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad409b750 con 0x564e2d0ece00 2024-05-08T23:30:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/2978142493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 msgr2=0x564e2d0ed1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0ed1e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40a3150 tx=0x7f5ad409ac70 comp rx=0 tx=0).stop 2024-05-08T23:30:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/2978142493 shutdown_connections 2024-05-08T23:30:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0e7890 0x564e2d0e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0ed1e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/2978142493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/2978142493 >> 172.21.15.16:0/2978142493 conn(0x564e2d045210 msgr2=0x564e2d0e1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/2978142493 shutdown_connections 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.919+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/2978142493 wait complete. 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 -- start start 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 Processor -- start 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 0x564e2d0f3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ece00 0x564e2d0f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2cfd3560 con 0x564e2d0e7890 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2cfd3200 con 0x564e2d0ece00 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2ceba6a0 con 0x564e2d0ed720 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5aeb7d1b80 10 client.0 _ll_register_callbacks cb 0x564e2cfdcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5add3b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 0x564e2d0f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5add3b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 0x564e2d0f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42574/0 (socket says 172.21.15.16:42574) 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5add3b3700 1 -- 172.21.15.16:0/1881838496 learned_addr learned my addr 172.21.15.16:0/1881838496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5addbb4700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.920+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ece00 0x564e2d0f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5add3b3700 1 -- 172.21.15.16:0/1881838496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 msgr2=0x564e2d0f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5add3b3700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5add3b3700 1 -- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ece00 msgr2=0x564e2d0f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5add3b3700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ece00 0x564e2d0f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5add3b3700 1 -- 172.21.15.16:0/1881838496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad81a64a0 con 0x564e2d0e7890 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ece00 0x564e2d0f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5add3b3700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 0x564e2d0f3710 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4097790 tx=0x7f5ad409a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2b5f416c4a67bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5ac0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/1881838496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40b7070 con 0x564e2d0e7890 2024-05-08T23:30:39.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/1881838496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5ad40a3680 con 0x564e2d0e7890 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e2cf90270 con 0x564e2d0e7890 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/1881838496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad4095ea0 con 0x564e2d0e7890 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5aeb7d1b80 10 client.17946 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e2ce64b60 con 0x564e2d0e7890 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/1881838496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564e2cf90270 con 0x564e2d0e7890 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.921+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/1881838496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5ad40b92b0 con 0x564e2d0e7890 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5aeb7d1b80 10 client.17946 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5aeb7d1b80 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5aeb7d1b80 20 client.17946 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d100910 0x564e2d102d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5aeb7d1b80 4 client.17946 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e2d0b6810 con 0x564e2d100910 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d100910 0x564e2d102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d100910 0x564e2d102d00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7622fab68c699dad server_cookie=4c9385001fb4c725 in_seq=0 out_seq=0 2024-05-08T23:30:39.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.922+0000 7f5ac0ff9700 10 client.17946 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/1881838496 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e2d0b6810 con 0x564e2d100910 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5ac0ff9700 10 client.17946 handle_command_reply: tid=0 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 client.17946 shutdown 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 msgr2=0x564e2d0f3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 0x564e2d0f3710 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4097790 tx=0x7f5ad409a9b0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 shutdown_connections 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0ed720 0x564e2d0f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0e7890 0x564e2d0f3710 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d100910 0x564e2d102d00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/1881838496 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ece00 0x564e2d0f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5ac0ff9700 0 client.17946 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 >> 172.21.15.16:0/1881838496 conn(0x564e2d045210 msgr2=0x564e2d0e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 shutdown_connections 2024-05-08T23:30:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/1881838496 wait complete. 2024-05-08T23:30:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.923+0000 7f5aeb7d1b80 20 client.17946 trim_cache size 0 max 16384 2024-05-08T23:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.937+0000 7f5aeb7d1b80 1 Processor -- start 2024-05-08T23:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.937+0000 7f5aeb7d1b80 1 -- start start 2024-05-08T23:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d107f20 con 0x564e2d0ece00 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d0e2910 con 0x564e2d0ed720 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d0dc830 con 0x564e2d0e7890 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42576/0 (socket says 172.21.15.16:42576) 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3292185911 learned_addr learned my addr 172.21.15.16:0/3292185911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5addbb4700 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5add3b3700 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3292185911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 msgr2=0x564e2d0f09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3292185911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 msgr2=0x564e2d0f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3292185911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2d0e7f40 con 0x564e2d0ece00 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5addbb4700 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.938+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0e1d20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x564e2d0edb80 tx=0x7f5ac8014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0964ebff5e4a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5ac3fff700 1 -- 172.21.15.16:0/3292185911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8026040 con 0x564e2d0ece00 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5ac3fff700 1 -- 172.21.15.16:0/3292185911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ac802c070 con 0x564e2d0ece00 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5ac3fff700 1 -- 172.21.15.16:0/3292185911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8003b00 con 0x564e2d0ece00 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3292185911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 msgr2=0x564e2d0e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0e1d20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x564e2d0edb80 tx=0x7f5ac8014070 comp rx=0 tx=0).stop 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3292185911 shutdown_connections 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3292185911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0e1d20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3292185911 >> 172.21.15.16:0/3292185911 conn(0x564e2d045210 msgr2=0x564e2d0f0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3292185911 shutdown_connections 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.939+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3292185911 wait complete. 2024-05-08T23:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 -- start start 2024-05-08T23:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 Processor -- start 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d0f4ec0 con 0x564e2d0ece00 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d0f5030 con 0x564e2d0ed720 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2d0f51f0 con 0x564e2d0e7890 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5aeb7d1b80 10 client.0 _ll_register_callbacks cb 0x564e2cfdcb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5adcbb2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5adcbb2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42580/0 (socket says 172.21.15.16:42580) 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3640941361 learned_addr learned my addr 172.21.15.16:0/3640941361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5addbb4700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.940+0000 7f5add3b3700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3640941361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 msgr2=0x564e2d0f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 msgr2=0x564e2d0f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5adcbb2700 1 -- 172.21.15.16:0/3640941361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac8028b70 con 0x564e2d0ece00 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5adcbb2700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0f4400 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80036c0 tx=0x7f5ac8028820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d594e460edec194 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5ac0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/3640941361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8026050 con 0x564e2d0ece00 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/3640941361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f5ac8018040 con 0x564e2d0ece00 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ad409b290 con 0x564e2d0ece00 2024-05-08T23:30:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/3640941361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ac80099e0 con 0x564e2d0ece00 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5aeb7d1b80 10 client.17952 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e2cffd560 con 0x564e2d0ece00 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.941+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/3640941361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ad409b290 con 0x564e2d0ece00 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/3640941361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5ac8012680 con 0x564e2d0ece00 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5aeb7d1b80 10 client.17952 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5aeb7d1b80 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5aeb7d1b80 20 client.17952 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d1582c0 0x564e2d15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5aeb7d1b80 4 client.17952 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e2d02fec0 con 0x564e2d1582c0 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5add3b3700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d1582c0 0x564e2d15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5add3b3700 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d1582c0 0x564e2d15a6e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a20a6bc42f4c5 server_cookie=94b39cbf6523b1ee in_seq=0 out_seq=0 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.942+0000 7f5ac0ff9700 10 client.17952 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5ac0ff9700 1 -- 172.21.15.16:0/3640941361 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e2d02fec0 con 0x564e2d1582c0 2024-05-08T23:30:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5ac0ff9700 10 client.17952 handle_command_reply: tid=0 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 client.17952 shutdown 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 msgr2=0x564e2d0f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0f4400 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80036c0 tx=0x7f5ac8028820 comp rx=0 tx=0).stop 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 shutdown_connections 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564e2d1582c0 0x564e2d15a6e0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564e2d0ed720 0x564e2d0f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e2d0e7890 0x564e2d0f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5ac0ff9700 0 client.17952 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 --2- 172.21.15.16:0/3640941361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e2d0ece00 0x564e2d0f4400 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 >> 172.21.15.16:0/3640941361 conn(0x564e2d045210 msgr2=0x564e2d0ef2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 shutdown_connections 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 1 -- 172.21.15.16:0/3640941361 wait complete. 2024-05-08T23:30:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5aeb7d1b80 20 client.17952 trim_cache size 0 max 16384 2024-05-08T23:30:39.950 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:30:39.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5ac40a1290 msgr2=0x7f5ac40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5adf617700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5ac40a1290 0x7f5ac40a3750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ad819aa40 tx=0x7f5ac8009260 comp rx=0 tx=0).stop 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 msgr2=0x7f5ad81994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.943+0000 7f5adf617700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad81994f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c000 tx=0x7f5ad4004500 comp rx=0 tx=0).stop 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 shutdown_connections 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f5ac40a1290 0x7f5ac40a3750 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad819ddd0 0x7f5ad819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ad8100060 0x7f5ad81994f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 --2- 172.21.15.16:0/225341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ad8105180 0x7f5ad8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 >> 172.21.15.16:0/225341556 conn(0x7f5ad8076fa0 msgr2=0x7f5ad80ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 shutdown_connections 2024-05-08T23:30:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:39.944+0000 7f5adf617700 1 -- 172.21.15.16:0/225341556 wait complete. 2024-05-08T23:30:40.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:39 smithi016 ceph-mon[42541]: pgmap v345: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:40.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:39 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1531807947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:30:40.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:39 smithi189 ceph-mon[46919]: pgmap v345: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:40.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:39 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1531807947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:30:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:39 smithi155 ceph-mon[47105]: pgmap v345: 129 pgs: 129 active+clean; 20 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:40.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:39 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1531807947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:30:40.958 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:30:40.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 -- 172.21.15.16:0/2490822806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc1062a0 msgr2=0x7f72bc10de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 --2- 172.21.15.16:0/2490822806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc1062a0 0x7f72bc10de20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f72b400b0a0 tx=0x7f72b400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 -- 172.21.15.16:0/2490822806 shutdown_connections 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 --2- 172.21.15.16:0/2490822806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc1062a0 0x7f72bc10de20 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 --2- 172.21.15.16:0/2490822806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 0x7f72bc069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 --2- 172.21.15.16:0/2490822806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc069dd0 0x7f72bc105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 -- 172.21.15.16:0/2490822806 >> 172.21.15.16:0/2490822806 conn(0x7f72bc076ba0 msgr2=0x7f72bc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 -- 172.21.15.16:0/2490822806 shutdown_connections 2024-05-08T23:30:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.108+0000 7f72c4004700 1 -- 172.21.15.16:0/2490822806 wait complete. 2024-05-08T23:30:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 Processor -- start 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 -- start start 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 0x7f72bc1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc1062a0 0x7f72bc1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 0x7f72bc19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc19c800 con 0x7f72bc19bd80 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc19c970 con 0x7f72bc1062a0 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c4004700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc19cb30 con 0x7f72bc069420 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 0x7f72bc19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 0x7f72bc19c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42596/0 (socket says 172.21.15.16:42596) 2024-05-08T23:30:41.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 -- 172.21.15.16:0/1222196268 learned_addr learned my addr 172.21.15.16:0/1222196268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c159f700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc1062a0 0x7f72bc1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c1da0700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 0x7f72bc1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 -- 172.21.15.16:0/1222196268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 msgr2=0x7f72bc1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 0x7f72bc1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 -- 172.21.15.16:0/1222196268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc1062a0 msgr2=0x7f72bc1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc1062a0 0x7f72bc1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c25a1700 1 -- 172.21.15.16:0/1222196268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4009d00 con 0x7f72bc19bd80 2024-05-08T23:30:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c159f700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc1062a0 0x7f72bc1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.109+0000 7f72c1da0700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 0x7f72bc1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.110+0000 7f72c25a1700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 0x7f72bc19c200 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f72b40094e0 tx=0x7f72b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.110+0000 7f72b2ffd700 1 -- 172.21.15.16:0/1222196268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b400e040 con 0x7f72bc19bd80 2024-05-08T23:30:41.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.110+0000 7f72b2ffd700 1 -- 172.21.15.16:0/1222196268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72b400bc50 con 0x7f72bc19bd80 2024-05-08T23:30:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.110+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc1a4030 con 0x7f72bc19bd80 2024-05-08T23:30:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.110+0000 7f72b2ffd700 1 -- 172.21.15.16:0/1222196268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b40052a0 con 0x7f72bc19bd80 2024-05-08T23:30:41.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.110+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc1a44b0 con 0x7f72bc19bd80 2024-05-08T23:30:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.112+0000 7f72b2ffd700 1 -- 172.21.15.16:0/1222196268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f72b401b450 con 0x7f72bc19bd80 2024-05-08T23:30:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.113+0000 7f72b2ffd700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72a80a1290 0x7f72a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.113+0000 7f72c1da0700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72a80a1290 0x7f72a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.113+0000 7f72b2ffd700 1 -- 172.21.15.16:0/1222196268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f72b40ce530 con 0x7f72bc19bd80 2024-05-08T23:30:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.113+0000 7f72c1da0700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72a80a1290 0x7f72a80a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009cb0 tx=0x7f72b8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 Processor -- start 2024-05-08T23:30:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 -- start start 2024-05-08T23:30:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d888670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d8827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d883160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d7d51d0 con 0x55b64d888bb0 2024-05-08T23:30:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d7d2b90 con 0x55b64d882d20 2024-05-08T23:30:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.114+0000 7f72d01beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d7ce480 con 0x55b64d888290 2024-05-08T23:30:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d8827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d8827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42598/0 (socket says 172.21.15.16:42598) 2024-05-08T23:30:41.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 -- 172.21.15.16:0/216800512 learned_addr learned my addr 172.21.15.16:0/216800512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c1da0700 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d888670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c25a1700 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d883160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 -- 172.21.15.16:0/216800512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 msgr2=0x55b64d888670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d888670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 -- 172.21.15.16:0/216800512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 msgr2=0x55b64d883160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d883160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 -- 172.21.15.16:0/216800512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b64d64f430 con 0x55b64d888bb0 2024-05-08T23:30:41.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c25a1700 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d883160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:41.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72c159f700 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d8827e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00dcb0 tx=0x7f72ac00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dd61b482fe5a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72a7fff700 1 -- 172.21.15.16:0/216800512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00fc90 con 0x55b64d888bb0 2024-05-08T23:30:41.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72a7fff700 1 -- 172.21.15.16:0/216800512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72ac00fdf0 con 0x55b64d888bb0 2024-05-08T23:30:41.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.115+0000 7f72a7fff700 1 -- 172.21.15.16:0/216800512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac010630 con 0x55b64d888bb0 2024-05-08T23:30:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 -- 172.21.15.16:0/216800512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 msgr2=0x55b64d8827e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d8827e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00dcb0 tx=0x7f72ac00f360 comp rx=0 tx=0).stop 2024-05-08T23:30:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 -- 172.21.15.16:0/216800512 shutdown_connections 2024-05-08T23:30:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d883160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d888670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 --2- 172.21.15.16:0/216800512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d8827e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 -- 172.21.15.16:0/216800512 >> 172.21.15.16:0/216800512 conn(0x55b64d7dc290 msgr2=0x55b64d878e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 -- 172.21.15.16:0/216800512 shutdown_connections 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.116+0000 7f72d01beb80 1 -- 172.21.15.16:0/216800512 wait complete. 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 -- start start 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 Processor -- start 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d88bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888290 0x55b64d88c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888bb0 0x55b64d88c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d76a5e0 con 0x55b64d882d20 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d76a280 con 0x55b64d888290 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d651720 con 0x55b64d888bb0 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72d01beb80 10 client.0 _ll_register_callbacks cb 0x55b64d773bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72c1da0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d88bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72c1da0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d88bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42604/0 (socket says 172.21.15.16:42604) 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72c1da0700 1 -- 172.21.15.16:0/4174239114 learned_addr learned my addr 172.21.15.16:0/4174239114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72c25a1700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888bb0 0x55b64d88c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.117+0000 7f72c159f700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888290 0x55b64d88c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c25a1700 1 -- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888290 msgr2=0x55b64d88c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c25a1700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888290 0x55b64d88c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c25a1700 1 -- 172.21.15.16:0/4174239114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 msgr2=0x55b64d88bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c25a1700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d88bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c25a1700 1 -- 172.21.15.16:0/4174239114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac00d8e0 con 0x55b64d888bb0 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c1da0700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d88bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:41.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72c25a1700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888bb0 0x55b64d88c960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f72b40cb880 tx=0x7f72b409acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb05b131d96b5dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72a57fa700 1 -- 172.21.15.16:0/4174239114 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b409b730 con 0x55b64d888bb0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b64d7272f0 con 0x55b64d888bb0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72a57fa700 1 -- 172.21.15.16:0/4174239114 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f72b401be90 con 0x55b64d888bb0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72a57fa700 1 -- 172.21.15.16:0/4174239114 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72b409b2d0 con 0x55b64d888bb0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72d01beb80 10 client.27592 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.118+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b64d5fbbe0 con 0x55b64d888bb0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72a57fa700 1 -- 172.21.15.16:0/4174239114 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b64d7272f0 con 0x55b64d888bb0 2024-05-08T23:30:41.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72a57fa700 1 -- 172.21.15.16:0/4174239114 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72b40a3230 con 0x55b64d888bb0 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72d01beb80 10 client.27592 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72d01beb80 10 client.27592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72d01beb80 20 client.27592 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72d01beb80 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d89c2e0 0x55b64d89e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72d01beb80 4 client.27592 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b64d84d890 con 0x55b64d89c2e0 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.119+0000 7f72c1da0700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d89c2e0 0x55b64d89e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72a57fa700 10 client.27592 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72c1da0700 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d89c2e0 0x55b64d89e700 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba1bf7ca1bf63aa server_cookie=44d12d3d6e9ed977 in_seq=0 out_seq=0 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72a57fa700 1 -- 172.21.15.16:0/4174239114 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b64d84d890 con 0x55b64d89c2e0 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72a57fa700 10 client.27592 handle_command_reply: tid=0 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 client.27592 shutdown 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888bb0 msgr2=0x55b64d88c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888bb0 0x55b64d88c960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f72b40cb880 tx=0x7f72b409acb0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 shutdown_connections 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888bb0 0x55b64d88c960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d88bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d89c2e0 0x55b64d89e700 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.120+0000 7f72d01beb80 1 --2- 172.21.15.16:0/4174239114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888290 0x55b64d88c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.121+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 >> 172.21.15.16:0/4174239114 conn(0x55b64d7dc290 msgr2=0x55b64d878620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.121+0000 7f72a57fa700 0 client.27592 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.121+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 shutdown_connections 2024-05-08T23:30:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.121+0000 7f72d01beb80 1 -- 172.21.15.16:0/4174239114 wait complete. 2024-05-08T23:30:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.121+0000 7f72d01beb80 20 client.27592 trim_cache size 0 max 16384 2024-05-08T23:30:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 Processor -- start 2024-05-08T23:30:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 -- start start 2024-05-08T23:30:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d878da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d88c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d886fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d8a3910 con 0x55b64d888bb0 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d88a430 con 0x55b64d882d20 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.139+0000 7f72d01beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d879f00 con 0x55b64d888290 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d886fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d886fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42608/0 (socket says 172.21.15.16:42608) 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 -- 172.21.15.16:0/1306180000 learned_addr learned my addr 172.21.15.16:0/1306180000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c159f700 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d88c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c1da0700 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d878da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 -- 172.21.15.16:0/1306180000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 msgr2=0x55b64d88c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d88c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 -- 172.21.15.16:0/1306180000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 msgr2=0x55b64d878da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d878da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 -- 172.21.15.16:0/1306180000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b64d7945e0 con 0x55b64d888bb0 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c1da0700 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d878da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72c25a1700 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d886fa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b2040 tx=0x7f72b4099880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b18cee48712473 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72b0ff9700 1 -- 172.21.15.16:0/1306180000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b401be90 con 0x55b64d888bb0 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72b0ff9700 1 -- 172.21.15.16:0/1306180000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72b409b2d0 con 0x55b64d888bb0 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.140+0000 7f72b0ff9700 1 -- 172.21.15.16:0/1306180000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b409b530 con 0x55b64d888bb0 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 -- 172.21.15.16:0/1306180000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 msgr2=0x55b64d886fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d886fa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b2040 tx=0x7f72b4099880 comp rx=0 tx=0).stop 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 -- 172.21.15.16:0/1306180000 shutdown_connections 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d888bb0 0x55b64d886fa0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d882d20 0x55b64d878da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 --2- 172.21.15.16:0/1306180000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d88c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 -- 172.21.15.16:0/1306180000 >> 172.21.15.16:0/1306180000 conn(0x55b64d7dc290 msgr2=0x55b64d88a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 -- 172.21.15.16:0/1306180000 shutdown_connections 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 -- 172.21.15.16:0/1306180000 wait complete. 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 -- start start 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.141+0000 7f72d01beb80 1 Processor -- start 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d8935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d893b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888bb0 0x55b64d891d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d892460 con 0x55b64d882d20 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d8925d0 con 0x55b64d888bb0 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b64d892790 con 0x55b64d888290 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72d01beb80 10 client.0 _ll_register_callbacks cb 0x55b64d773bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d8935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d8935d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42620/0 (socket says 172.21.15.16:42620) 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 -- 172.21.15.16:0/405134131 learned_addr learned my addr 172.21.15.16:0/405134131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c159f700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d893b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c25a1700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888bb0 0x55b64d891d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 -- 172.21.15.16:0/405134131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 msgr2=0x55b64d893b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d893b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 -- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888bb0 msgr2=0x55b64d891d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888bb0 0x55b64d891d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.142+0000 7f72c1da0700 1 -- 172.21.15.16:0/405134131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b64d5fbbe0 con 0x55b64d882d20 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72c1da0700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d8935d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x55b64d89efe0 tx=0x7f72b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f5c28aa59b2ca3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72a57fa700 1 -- 172.21.15.16:0/405134131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8028630 con 0x55b64d882d20 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72a57fa700 1 -- 172.21.15.16:0/405134131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f72b8018040 con 0x55b64d882d20 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72ac00fdf0 con 0x55b64d882d20 2024-05-08T23:30:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72a57fa700 1 -- 172.21.15.16:0/405134131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72b8028790 con 0x55b64d882d20 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 10 client.17970 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b64d653d50 con 0x55b64d882d20 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72a57fa700 1 -- 172.21.15.16:0/405134131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72ac00fdf0 con 0x55b64d882d20 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72a57fa700 1 -- 172.21.15.16:0/405134131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72b80125f0 con 0x55b64d882d20 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 10 client.17970 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 20 client.17970 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d8f3bc0 0x55b64d8f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 4 client.17970 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.143+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b64d7c6f40 con 0x55b64d8f3bc0 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.144+0000 7f72c159f700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d8f3bc0 0x55b64d8f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.144+0000 7f72c159f700 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d8f3bc0 0x55b64d8f5fe0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898a7f99a158517 server_cookie=3c1a26be41636f84 in_seq=0 out_seq=0 2024-05-08T23:30:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.144+0000 7f72a57fa700 10 client.17970 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72a57fa700 1 -- 172.21.15.16:0/405134131 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b64d7c6f40 con 0x55b64d8f3bc0 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72a57fa700 10 client.17970 handle_command_reply: tid=0 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 client.17970 shutdown 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 msgr2=0x55b64d8935d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d8935d0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x55b64d89efe0 tx=0x7f72b80047c0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 shutdown_connections 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b64d8f3bc0 0x55b64d8f5fe0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b64d888bb0 0x55b64d891d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b64d882d20 0x55b64d8935d0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72a57fa700 0 client.17970 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 --2- 172.21.15.16:0/405134131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b64d888290 0x55b64d893b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.145+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 >> 172.21.15.16:0/405134131 conn(0x55b64d7dc290 msgr2=0x55b64d888f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 shutdown_connections 2024-05-08T23:30:41.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72d01beb80 1 -- 172.21.15.16:0/405134131 wait complete. 2024-05-08T23:30:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72d01beb80 20 client.17970 trim_cache size 0 max 16384 2024-05-08T23:30:41.152 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:30:41.152 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:30:41.152 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e1e24dc0-28d8-4e39-b39e-e78bcb3f94b9", 2024-05-08T23:30:41.152 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:30:41.152 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72a80a1290 msgr2=0x7f72a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72a80a1290 0x7f72a80a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009cb0 tx=0x7f72b8009260 comp rx=0 tx=0).stop 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 msgr2=0x7f72bc19c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 0x7f72bc19c200 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f72b40094e0 tx=0x7f72b40095c0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 shutdown_connections 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72a80a1290 0x7f72a80a3750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72bc19bd80 0x7f72bc19c200 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72bc069420 0x7f72bc1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 --2- 172.21.15.16:0/1222196268 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72bc1062a0 0x7f72bc1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 >> 172.21.15.16:0/1222196268 conn(0x7f72bc076ba0 msgr2=0x7f72bc0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 shutdown_connections 2024-05-08T23:30:41.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.146+0000 7f72c4004700 1 -- 172.21.15.16:0/1222196268 wait complete. 2024-05-08T23:30:41.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.298+0000 7fc8b84ff700 1 -- 172.21.15.16:0/2880695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 msgr2=0x7fc8b010ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.298+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/2880695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b010ccc0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b0d0 tx=0x7fc8a800b3e0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 -- 172.21.15.16:0/2880695369 shutdown_connections 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/2880695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b010ccc0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/2880695369 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8b0105f90 0x7fc8b0106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/2880695369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 0x7fc8b0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 -- 172.21.15.16:0/2880695369 >> 172.21.15.16:0/2880695369 conn(0x7fc8b0076ba0 msgr2=0x7fc8b0076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 -- 172.21.15.16:0/2880695369 shutdown_connections 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 -- 172.21.15.16:0/2880695369 wait complete. 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 Processor -- start 2024-05-08T23:30:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.299+0000 7fc8b84ff700 1 -- start start 2024-05-08T23:30:41.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b84ff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 0x7fc8b0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b84ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b0199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b84ff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8b0105f90 0x7fc8b019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b84ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019e440 con 0x7fc8b0100ab0 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b84ff700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019e5b0 con 0x7fc8b0105f90 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b84ff700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019e770 con 0x7fc8b01000f0 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b0199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b0199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42626/0 (socket says 172.21.15.16:42626) 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 -- 172.21.15.16:0/4091714853 learned_addr learned my addr 172.21.15.16:0/4091714853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b629b700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 0x7fc8b0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8b0105f90 0x7fc8b019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 -- 172.21.15.16:0/4091714853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 msgr2=0x7fc8b0199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 0x7fc8b0199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 -- 172.21.15.16:0/4091714853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8b0105f90 msgr2=0x7fc8b019de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8b0105f90 0x7fc8b019de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 -- 172.21.15.16:0/4091714853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a8009d00 con 0x7fc8b0100ab0 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b629b700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 0x7fc8b0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b0199a60 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e9a0 tx=0x7fc8a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.300+0000 7fc8a37fe700 1 -- 172.21.15.16:0/4091714853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009790 con 0x7fc8b0100ab0 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.301+0000 7fc8a37fe700 1 -- 172.21.15.16:0/4091714853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a4010430 con 0x7fc8b0100ab0 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.301+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b019ea50 con 0x7fc8b0100ab0 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.301+0000 7fc8a37fe700 1 -- 172.21.15.16:0/4091714853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400cea0 con 0x7fc8b0100ab0 2024-05-08T23:30:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.301+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b01a6290 con 0x7fc8b0100ab0 2024-05-08T23:30:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.303+0000 7fc8a37fe700 1 -- 172.21.15.16:0/4091714853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc8a40105a0 con 0x7fc8b0100ab0 2024-05-08T23:30:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.304+0000 7fc8a37fe700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc89c0a1290 0x7fc89c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.304+0000 7fc8b629b700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc89c0a1290 0x7fc89c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.304+0000 7fc8a37fe700 1 -- 172.21.15.16:0/4091714853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc8a4014070 con 0x7fc8b0100ab0 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8b629b700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc89c0a1290 0x7fc89c0a3750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0067bb0 tx=0x7fc8ac009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 Processor -- start 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 -- start start 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f2e720 0x561fb4f28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f28890 0x561fb4f28cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4e7f150 con 0x561fb4f28890 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4e7cb10 con 0x561fb4f2e720 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.305+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4e78400 con 0x561fb4f2de00 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b629b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f28890 0x561fb4f28cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b629b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:53534/0 (socket says 172.21.15.16:53534) 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b629b700 1 -- 172.21.15.16:0/2520393741 learned_addr learned my addr 172.21.15.16:0/2520393741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f2e720 0x561fb4f28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/2520393741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 msgr2=0x561fb4f2e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/2520393741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f2e720 msgr2=0x561fb4f28350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f2e720 0x561fb4f28350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/2520393741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb4cf93b0 con 0x561fb4f28890 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b629b700 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f28890 0x561fb4f28cd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b770 tx=0x7fc8a8009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d9ee7ea40d7541 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8a0ff9700 1 -- 172.21.15.16:0/2520393741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800e070 con 0x561fb4f28890 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8a0ff9700 1 -- 172.21.15.16:0/2520393741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a8009460 con 0x561fb4f28890 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.306+0000 7fc8a0ff9700 1 -- 172.21.15.16:0/2520393741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8003c60 con 0x561fb4f28890 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/2520393741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f28890 msgr2=0x561fb4f28cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f28890 0x561fb4f28cd0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800b770 tx=0x7fc8a8009c60 comp rx=0 tx=0).stop 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/2520393741 shutdown_connections 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f28890 0x561fb4f28cd0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/2520393741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f2e720 0x561fb4f28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/2520393741 >> 172.21.15.16:0/2520393741 conn(0x561fb4e86210 msgr2=0x561fb4f22df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/2520393741 shutdown_connections 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.307+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/2520393741 wait complete. 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 -- start start 2024-05-08T23:30:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 Processor -- start 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f31a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f32510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4e14560 con 0x561fb4f2e720 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4e14200 con 0x561fb4f28890 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4cfb6a0 con 0x561fb4f2de00 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8c46b9b80 10 client.0 _ll_register_callbacks cb 0x561fb4e1db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f32510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f32510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42654/0 (socket says 172.21.15.16:42654) 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3086820643 learned_addr learned my addr 172.21.15.16:0/3086820643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f31a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3086820643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 msgr2=0x561fb4f31fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f31fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 msgr2=0x561fb4f31a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f31a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3086820643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac017070 con 0x561fb4f2e720 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.308+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f31a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f32510 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007d40 tx=0x7fc8a8007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b33512c5f6102fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc899ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc899ffb700 1 -- 172.21.15.16:0/3086820643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800e070 con 0x561fb4f2e720 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fb4dd1270 con 0x561fb4f2e720 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc899ffb700 1 -- 172.21.15.16:0/3086820643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc8a800ce80 con 0x561fb4f2e720 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc899ffb700 1 -- 172.21.15.16:0/3086820643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a8003a00 con 0x561fb4f2e720 2024-05-08T23:30:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 10 client.17982 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7fc8b010d550 con 0x561fb4f2e720 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc899ffb700 1 -- 172.21.15.16:0/3086820643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561fb4dd1270 con 0x561fb4f2e720 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc899ffb700 1 -- 172.21.15.16:0/3086820643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8a8019050 con 0x561fb4f2e720 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 10 client.17982 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 20 client.17982 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f41df0 0x561fb4f44210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 4 client.17982 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.309+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fb4ef7810 con 0x561fb4f41df0 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.310+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f41df0 0x561fb4f44210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.310+0000 7fc899ffb700 10 client.17982 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.310+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f41df0 0x561fb4f44210 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=becc83abce81e49f server_cookie=b481ddeaa21beba1 in_seq=0 out_seq=0 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc899ffb700 1 -- 172.21.15.16:0/3086820643 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fb4ef7810 con 0x561fb4f41df0 2024-05-08T23:30:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc899ffb700 10 client.17982 handle_command_reply: tid=0 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 client.17982 shutdown 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 msgr2=0x561fb4f32510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f32510 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007d40 tx=0x7fc8a8007e20 comp rx=0 tx=0).stop 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 shutdown_connections 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f32510 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f41df0 0x561fb4f44210 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f31a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc899ffb700 0 client.17982 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3086820643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f31fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 >> 172.21.15.16:0/3086820643 conn(0x561fb4e86210 msgr2=0x561fb4f225a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 shutdown_connections 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3086820643 wait complete. 2024-05-08T23:30:41.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.311+0000 7fc8c46b9b80 20 client.17982 trim_cache size 0 max 16384 2024-05-08T23:30:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.329+0000 7fc8c46b9b80 1 Processor -- start 2024-05-08T23:30:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.329+0000 7fc8c46b9b80 1 -- start start 2024-05-08T23:30:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f22d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f2e560 con 0x561fb4f2e720 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f49420 con 0x561fb4f28890 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f2ffe0 con 0x561fb4f2de00 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f22d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f22d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42656/0 (socket says 172.21.15.16:42656) 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/479012095 learned_addr learned my addr 172.21.15.16:0/479012095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b629b700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/479012095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 msgr2=0x561fb4f2d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/479012095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 msgr2=0x561fb4f28350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f28350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/479012095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb4ca5b60 con 0x561fb4f2e720 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b629b700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f22d20 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007cc0 tx=0x7fc8a8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87732197117a5121 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8a17fa700 1 -- 172.21.15.16:0/479012095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800e070 con 0x561fb4f2e720 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8a17fa700 1 -- 172.21.15.16:0/479012095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a8025450 con 0x561fb4f2e720 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.330+0000 7fc8a17fa700 1 -- 172.21.15.16:0/479012095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8003e40 con 0x561fb4f2e720 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/479012095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 msgr2=0x561fb4f22d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f22d20 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007cc0 tx=0x7fc8a8007f40 comp rx=0 tx=0).stop 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/479012095 shutdown_connections 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f22d20 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/479012095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f2d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/479012095 >> 172.21.15.16:0/479012095 conn(0x561fb4e86210 msgr2=0x561fb4f2cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/479012095 shutdown_connections 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/479012095 wait complete. 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 -- start start 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.331+0000 7fc8c46b9b80 1 Processor -- start 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f37340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f37880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f37ff0 con 0x561fb4f2e720 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f97830 con 0x561fb4f28890 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb4f979f0 con 0x561fb4f2de00 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8c46b9b80 10 client.0 _ll_register_callbacks cb 0x561fb4e1db30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42658/0 (socket says 172.21.15.16:42658) 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3864006745 learned_addr learned my addr 172.21.15.16:0/3864006745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b5a9a700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f37880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f37340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3864006745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 msgr2=0x561fb4f37880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f37880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 msgr2=0x561fb4f37340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f37340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 -- 172.21.15.16:0/3864006745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b010d550 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc8b6a9c700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f972f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007d40 tx=0x7fc8a8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d741f43efaa157 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc899ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc899ffb700 1 -- 172.21.15.16:0/3864006745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800e040 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.332+0000 7fc899ffb700 1 -- 172.21.15.16:0/3864006745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fc8a800ce80 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8a8009460 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc899ffb700 1 -- 172.21.15.16:0/3864006745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a8003c10 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 10 client.17988 fetch_fsmap learned FSMap version 25 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561fb4e3e560 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc899ffb700 1 -- 172.21.15.16:0/3864006745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a8009460 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc899ffb700 1 -- 172.21.15.16:0/3864006745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8a800b100 con 0x561fb4f2e720 2024-05-08T23:30:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 10 client.17988 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 20 client.17988 populate_metadata read hostname 'smithi016' 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f98cb0 0x561fb4f9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 4 client.17988 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561fb4e70ec0 con 0x561fb4f98cb0 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.333+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f98cb0 0x561fb4f9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc899ffb700 10 client.17988 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8b629b700 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f98cb0 0x561fb4f9b0a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d10cfb121497cc server_cookie=a11349056df35cdb in_seq=0 out_seq=0 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc899ffb700 1 -- 172.21.15.16:0/3864006745 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561fb4e70ec0 con 0x561fb4f98cb0 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc899ffb700 10 client.17988 handle_command_reply: tid=0 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 client.17988 shutdown 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 msgr2=0x561fb4f972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f972f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8007d40 tx=0x7fc8a8009650 comp rx=0 tx=0).stop 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 shutdown_connections 2024-05-08T23:30:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fb4f2e720 0x561fb4f972f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x561fb4f98cb0 0x561fb4f9b0a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x561fb4f28890 0x561fb4f37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 --2- 172.21.15.16:0/3864006745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fb4f2de00 0x561fb4f37880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc899ffb700 0 client.17988 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 >> 172.21.15.16:0/3864006745 conn(0x561fb4e86210 msgr2=0x561fb4f2cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 shutdown_connections 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.334+0000 7fc8c46b9b80 1 -- 172.21.15.16:0/3864006745 wait complete. 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8c46b9b80 20 client.17988 trim_cache size 0 max 16384 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:30:41.341 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout: "e1e24dc0-28d8-4e39-b39e-e78bcb3f94b9": { 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e1e24dc0-28d8-4e39-b39e-e78bcb3f94b9", 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:30:41.342 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc89c0a1290 msgr2=0x7fc89c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc89c0a1290 0x7fc89c0a3750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0067bb0 tx=0x7fc8ac009260 comp rx=0 tx=0).stop 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 msgr2=0x7fc8b0199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b0199a60 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e9a0 tx=0x7fc8a400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 shutdown_connections 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fc89c0a1290 0x7fc89c0a3750 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8b0105f90 0x7fc8b019de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8b01000f0 0x7fc8b0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 --2- 172.21.15.16:0/4091714853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b0100ab0 0x7fc8b0199a60 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 >> 172.21.15.16:0/4091714853 conn(0x7fc8b0076ba0 msgr2=0x7fc8b0069290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 shutdown_connections 2024-05-08T23:30:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:30:41.335+0000 7fc8b84ff700 1 -- 172.21.15.16:0/4091714853 wait complete. 2024-05-08T23:30:41.349 INFO:tasks.cephfs.filesystem:scrub status for tag:e1e24dc0-28d8-4e39-b39e-e78bcb3f94b9 - {'path': '/', 'tag': 'e1e24dc0-28d8-4e39-b39e-e78bcb3f94b9', 'options': 'recursive,force'} 2024-05-08T23:30:42.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:41 smithi016 ceph-mon[42541]: pgmap v346: 129 pgs: 129 active+clean; 20 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-08T23:30:42.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:41 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:30:42.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:41 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:42.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:41 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:30:42.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:41 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:42.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:41 smithi189 ceph-mon[46919]: pgmap v346: 129 pgs: 129 active+clean; 20 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-08T23:30:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:41 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:30:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:41 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:41 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:30:42.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:41 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:41 smithi155 ceph-mon[47105]: pgmap v346: 129 pgs: 129 active+clean; 20 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-08T23:30:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:41 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:30:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:41 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:41 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:30:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:41 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:30:44.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:43 smithi016 ceph-mon[42541]: pgmap v347: 129 pgs: 129 active+clean; 20 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:44.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:43 smithi189 ceph-mon[46919]: pgmap v347: 129 pgs: 129 active+clean; 20 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:44.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:43 smithi155 ceph-mon[47105]: pgmap v347: 129 pgs: 129 active+clean; 20 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:45 smithi016 ceph-mon[42541]: pgmap v348: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:46.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:45 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:30:46.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:45 smithi189 ceph-mon[46919]: pgmap v348: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:46.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:45 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:30:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:45 smithi155 ceph-mon[47105]: pgmap v348: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:46.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:45 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:30:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:47 smithi016 ceph-mon[42541]: pgmap v349: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:47 smithi189 ceph-mon[46919]: pgmap v349: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:48.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:47 smithi155 ceph-mon[47105]: pgmap v349: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:50.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:49 smithi016 ceph-mon[42541]: pgmap v350: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:50.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:49 smithi189 ceph-mon[46919]: pgmap v350: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:50.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:49 smithi155 ceph-mon[47105]: pgmap v350: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:51.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:50 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:30:51.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:50 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:30:51.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:50 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:30:52.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:51 smithi016 ceph-mon[42541]: pgmap v351: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-08T23:30:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:51 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:30:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:51 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:30:52.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:51 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:30:52.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:51 smithi189 ceph-mon[46919]: pgmap v351: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-08T23:30:52.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:51 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:30:52.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:51 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:30:52.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:51 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:30:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:51 smithi155 ceph-mon[47105]: pgmap v351: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-05-08T23:30:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:51 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:30:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:51 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:30:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:51 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:30:54.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:53 smithi016 ceph-mon[42541]: pgmap v352: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:54.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:53 smithi189 ceph-mon[46919]: pgmap v352: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:54.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:53 smithi155 ceph-mon[47105]: pgmap v352: 129 pgs: 129 active+clean; 21 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-08T23:30:56.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:55 smithi016 ceph-mon[42541]: pgmap v353: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:56.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:55 smithi189 ceph-mon[46919]: pgmap v353: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:56.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:55 smithi155 ceph-mon[47105]: pgmap v353: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-05-08T23:30:58.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:57 smithi016 ceph-mon[42541]: pgmap v354: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:58.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:57 smithi189 ceph-mon[46919]: pgmap v354: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:30:58.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:57 smithi155 ceph-mon[47105]: pgmap v354: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:31:00.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:30:59 smithi016 ceph-mon[42541]: pgmap v355: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:31:00.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:30:59 smithi189 ceph-mon[46919]: pgmap v355: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:31:00.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:30:59 smithi155 ceph-mon[47105]: pgmap v355: 129 pgs: 129 active+clean; 21 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-08T23:31:01.864 INFO:tasks.workunit.client.0.smithi016.stdout:499/600: open fd = 3 2024-05-08T23:31:01.864 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:500/600: open fd = 3 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:501/600: open fd = 3 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:502/600: open fd = 3 2024-05-08T23:31:01.865 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:503/600: open fd = 3 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:504/600: open fd = 3 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:505/600: open fd = 3 2024-05-08T23:31:01.866 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:506/600: open fd = 3 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:507/600: open fd = 3 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:508/600: open fd = 3 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.867 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:509/600: open fd = 3 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:510/600: open fd = 3 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:511/600: open fd = 3 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.868 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:512/600: open fd = 3 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:513/600: open fd = 3 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:514/600: open fd = 3 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.869 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:515/600: open fd = 3 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:516/600: open fd = 3 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:517/600: open fd = 3 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.870 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:518/600: open fd = 3 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:519/600: open fd = 3 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:520/600: open fd = 3 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.871 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:521/600: open fd = 3 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:522/600: open fd = 3 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:523/600: open fd = 3 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.872 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:524/600: open fd = 3 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:525/600: open fd = 3 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:526/600: open fd = 3 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.873 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:527/600: open fd = 3 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:528/600: open fd = 3 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:529/600: open fd = 3 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.874 INFO:tasks.workunit.client.0.smithi016.stdout:530/600: open fd = 3 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:531/600: open fd = 3 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:532/600: open fd = 3 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.875 INFO:tasks.workunit.client.0.smithi016.stdout:533/600: open fd = 3 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:534/600: open fd = 3 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:535/600: open fd = 3 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:536/600: open fd = 3 2024-05-08T23:31:01.876 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:537/600: open fd = 3 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:538/600: open fd = 3 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:539/600: open fd = 3 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.877 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:540/600: open fd = 3 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:541/600: open fd = 3 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:542/600: open fd = 3 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.878 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:543/600: open fd = 3 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:544/600: open fd = 3 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:545/600: open fd = 3 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.879 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:546/600: open fd = 3 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:547/600: open fd = 3 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:548/600: open fd = 3 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.880 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:549/600: open fd = 3 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:550/600: open fd = 3 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:551/600: open fd = 3 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.881 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:552/600: open fd = 3 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:553/600: open fd = 3 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:554/600: open fd = 3 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.882 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:555/600: open fd = 3 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:556/600: open fd = 3 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:557/600: open fd = 3 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.883 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:558/600: open fd = 3 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:559/600: open fd = 3 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:560/600: open fd = 3 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:01.884 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:02.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:01 smithi016 ceph-mon[42541]: pgmap v356: 129 pgs: 129 active+clean; 21 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-05-08T23:31:02.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:01 smithi189 ceph-mon[46919]: pgmap v356: 129 pgs: 129 active+clean; 21 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-05-08T23:31:02.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:01 smithi155 ceph-mon[47105]: pgmap v356: 129 pgs: 129 active+clean; 21 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 79 KiB/s wr, 15 op/s 2024-05-08T23:31:04.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:03 smithi016 ceph-mon[42541]: pgmap v357: 129 pgs: 129 active+clean; 21 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-05-08T23:31:04.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:03 smithi189 ceph-mon[46919]: pgmap v357: 129 pgs: 129 active+clean; 21 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-05-08T23:31:04.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:03 smithi155 ceph-mon[47105]: pgmap v357: 129 pgs: 129 active+clean; 21 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 10 op/s 2024-05-08T23:31:06.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:05 smithi016 ceph-mon[42541]: pgmap v358: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 125 KiB/s wr, 32 op/s 2024-05-08T23:31:06.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:05 smithi189 ceph-mon[46919]: pgmap v358: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 125 KiB/s wr, 32 op/s 2024-05-08T23:31:06.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:05 smithi155 ceph-mon[47105]: pgmap v358: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 125 KiB/s wr, 32 op/s 2024-05-08T23:31:07.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:06 smithi016 ceph-mon[42541]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:31:07.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:06 smithi189 ceph-mon[46919]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:31:07.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:06 smithi155 ceph-mon[47105]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T23:31:08.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:07 smithi016 ceph-mon[42541]: pgmap v360: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-05-08T23:31:08.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:07 smithi189 ceph-mon[46919]: pgmap v360: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-05-08T23:31:08.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:07 smithi155 ceph-mon[47105]: pgmap v360: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-05-08T23:31:10.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:09 smithi016 ceph-mon[42541]: pgmap v361: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-05-08T23:31:10.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:09 smithi189 ceph-mon[46919]: pgmap v361: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-05-08T23:31:10.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:09 smithi155 ceph-mon[47105]: pgmap v361: 129 pgs: 129 active+clean; 22 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 121 KiB/s wr, 33 op/s 2024-05-08T23:31:11.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:31:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 -- 172.21.15.16:0/669092489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6190105160 msgr2=0x7f619010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 --2- 172.21.15.16:0/669092489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6190105160 0x7f619010cce0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f618800b0a0 tx=0x7f618800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 -- 172.21.15.16:0/669092489 shutdown_connections 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 --2- 172.21.15.16:0/669092489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6190105160 0x7f619010cce0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 --2- 172.21.15.16:0/669092489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6190100060 0x7f6190100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 --2- 172.21.15.16:0/669092489 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6190100a10 0x7f6190104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 -- 172.21.15.16:0/669092489 >> 172.21.15.16:0/669092489 conn(0x7f6190076f40 msgr2=0x7f6190077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 -- 172.21.15.16:0/669092489 shutdown_connections 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.501+0000 7f6198393700 1 -- 172.21.15.16:0/669092489 wait complete. 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 Processor -- start 2024-05-08T23:31:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 -- start start 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6190100060 0x7f61901973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6190105160 0x7f61901978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 0x7f619019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619019c710 con 0x7f619019bc90 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619019c880 con 0x7f6190105160 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6198393700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619019ca40 con 0x7f6190100060 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 0x7f619019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 0x7f619019c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47462/0 (socket says 172.21.15.16:47462) 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 -- 172.21.15.16:0/419811141 learned_addr learned my addr 172.21.15.16:0/419811141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f619592e700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6190105160 0x7f61901978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f619612f700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6190100060 0x7f61901973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 -- 172.21.15.16:0/419811141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6190100060 msgr2=0x7f61901973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6190100060 0x7f61901973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 -- 172.21.15.16:0/419811141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6190105160 msgr2=0x7f61901978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6190105160 0x7f61901978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.502+0000 7f6196930700 1 -- 172.21.15.16:0/419811141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188009d00 con 0x7f619019bc90 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.503+0000 7f6196930700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 0x7f619019c110 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f61880094c0 tx=0x7f61880094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.503+0000 7f61877fe700 1 -- 172.21.15.16:0/419811141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800e040 con 0x7f619019bc90 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.503+0000 7f61877fe700 1 -- 172.21.15.16:0/419811141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6188003c00 con 0x7f619019bc90 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.503+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61901a3f40 con 0x7f619019bc90 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.503+0000 7f61877fe700 1 -- 172.21.15.16:0/419811141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61880052a0 con 0x7f619019bc90 2024-05-08T23:31:11.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.503+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61901a4410 con 0x7f619019bc90 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.505+0000 7f61877fe700 1 -- 172.21.15.16:0/419811141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f618801d480 con 0x7f619019bc90 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.506+0000 7f61877fe700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f617c0a1310 0x7f617c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.506+0000 7f619612f700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f617c0a1310 0x7f617c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.506+0000 7f61877fe700 1 -- 172.21.15.16:0/419811141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f61880ce270 con 0x7f619019bc90 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.506+0000 7f619612f700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f617c0a1310 0x7f617c0a37d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f618c005950 tx=0x7f618c0058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 Processor -- start 2024-05-08T23:31:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 -- start start 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a5138165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513810760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a5138110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a513763150 con 0x55a513816b30 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a513760b10 con 0x55a513816210 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f61a454db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51375c400 con 0x55a513810ca0 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f619592e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513810760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f619612f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a5138165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f619612f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a5138165f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.16:54908/0 (socket says 172.21.15.16:54908) 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f619612f700 1 -- 172.21.15.16:0/2665870723 learned_addr learned my addr 172.21.15.16:0/2665870723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.507+0000 7f619592e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513810760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47478/0 (socket says 172.21.15.16:47478) 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f6196930700 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a5138110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619612f700 1 -- 172.21.15.16:0/2665870723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 msgr2=0x55a5138110e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619612f700 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a5138110e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619612f700 1 -- 172.21.15.16:0/2665870723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 msgr2=0x55a513810760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619612f700 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513810760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619612f700 1 -- 172.21.15.16:0/2665870723 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5135dd3b0 con 0x55a513816210 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619592e700 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513810760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f619612f700 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a5138165f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f618c00fd70 tx=0x7f618c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d4c7fd91dbe331 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f6184da4700 1 -- 172.21.15.16:0/2665870723 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c011bb0 con 0x55a513816210 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f6184da4700 1 -- 172.21.15.16:0/2665870723 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f618c011d10 con 0x55a513816210 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.508+0000 7f6184da4700 1 -- 172.21.15.16:0/2665870723 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c01d610 con 0x55a513816210 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 -- 172.21.15.16:0/2665870723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 msgr2=0x55a5138165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a5138165f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f618c00fd70 tx=0x7f618c012000 comp rx=0 tx=0).stop 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 -- 172.21.15.16:0/2665870723 shutdown_connections 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a5138110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a5138165f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 --2- 172.21.15.16:0/2665870723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513810760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 -- 172.21.15.16:0/2665870723 >> 172.21.15.16:0/2665870723 conn(0x55a51376a210 msgr2=0x55a513806df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 -- 172.21.15.16:0/2665870723 shutdown_connections 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.509+0000 7f61a454db80 1 -- 172.21.15.16:0/2665870723 wait complete. 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 -- start start 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 Processor -- start 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513819f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a51381a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816b30 0x55a51381a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5136f8560 con 0x55a513810ca0 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5136f8200 con 0x55a513816210 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5135df6a0 con 0x55a513816b30 2024-05-08T23:31:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f61a454db80 10 client.0 _ll_register_callbacks cb 0x55a513701b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.510+0000 7f619612f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513819f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513819f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47494/0 (socket says 172.21.15.16:47494) 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 -- 172.21.15.16:0/2825884921 learned_addr learned my addr 172.21.15.16:0/2825884921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f6196930700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816b30 0x55a51381a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619592e700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a51381a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 -- 172.21.15.16:0/2825884921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816b30 msgr2=0x55a51381a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816b30 0x55a51381a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 -- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 msgr2=0x55a51381a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a51381a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 -- 172.21.15.16:0/2825884921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618000b860 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f619612f700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513819f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f618c00ffd0 tx=0x7f618c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ee86caddf04feb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f6179ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f6179ffb700 1 -- 172.21.15.16:0/2825884921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c02d070 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5136b5270 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f6179ffb700 1 -- 172.21.15.16:0/2825884921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f618c005240 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.511+0000 7f6179ffb700 1 -- 172.21.15.16:0/2825884921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f618c005400 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 10 client.18000 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f618c00fa20 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f6179ffb700 1 -- 172.21.15.16:0/2825884921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a5136b5270 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f6179ffb700 1 -- 172.21.15.16:0/2825884921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f618c01e570 con 0x55a513810ca0 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 10 client.18000 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:11.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 20 client.18000 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a51382a610 0x55a51382ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 4 client.18000 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5137db810 con 0x55a51382a610 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.512+0000 7f619592e700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a51382a610 0x55a51382ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.513+0000 7f6179ffb700 10 client.18000 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.513+0000 7f619592e700 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a51382a610 0x55a51382ca30 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519b6ba59f0428a5 server_cookie=98a54ae0b51cfbc in_seq=0 out_seq=0 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.513+0000 7f6179ffb700 1 -- 172.21.15.16:0/2825884921 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5137db810 con 0x55a51382a610 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.513+0000 7f6179ffb700 10 client.18000 handle_command_reply: tid=0 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.513+0000 7f61a454db80 1 client.18000 shutdown 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 msgr2=0x55a513819f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513819f70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f618c00ffd0 tx=0x7f618c011bb0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 shutdown_connections 2024-05-08T23:31:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816b30 0x55a51381a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513819f70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a51382a610 0x55a51382ca30 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 --2- 172.21.15.16:0/2825884921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a51381a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f6179ffb700 0 client.18000 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 >> 172.21.15.16:0/2825884921 conn(0x55a51376a210 msgr2=0x55a5138065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 shutdown_connections 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 1 -- 172.21.15.16:0/2825884921 wait complete. 2024-05-08T23:31:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.514+0000 7f61a454db80 20 client.18000 trim_cache size 0 max 16384 2024-05-08T23:31:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 Processor -- start 2024-05-08T23:31:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 -- start start 2024-05-08T23:31:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513806d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816210 0x55a513821580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816b30 0x55a513821ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a513807ca0 con 0x55a513810ca0 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a513808e50 con 0x55a513816b30 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f61a454db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a513831c40 con 0x55a513816210 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619612f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513806d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619612f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513806d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47506/0 (socket says 172.21.15.16:47506) 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619612f700 1 -- 172.21.15.16:0/3665927197 learned_addr learned my addr 172.21.15.16:0/3665927197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619592e700 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816210 0x55a513821580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f6196930700 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816b30 0x55a513821ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619612f700 1 -- 172.21.15.16:0/3665927197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816210 msgr2=0x55a513821580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619612f700 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816210 0x55a513821580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.532+0000 7f619612f700 1 -- 172.21.15.16:0/3665927197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816b30 msgr2=0x55a513821ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f619612f700 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816b30 0x55a513821ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f619612f700 1 -- 172.21.15.16:0/3665927197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a513589b60 con 0x55a513810ca0 2024-05-08T23:31:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f6196930700 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816b30 0x55a513821ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f619612f700 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513806d20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f618c005810 tx=0x7f618c01eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7eb9560b79552f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f61855a5700 1 -- 172.21.15.16:0/3665927197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c02d070 con 0x55a513810ca0 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f61855a5700 1 -- 172.21.15.16:0/3665927197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f618c00b440 con 0x55a513810ca0 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f61855a5700 1 -- 172.21.15.16:0/3665927197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c004970 con 0x55a513810ca0 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f61a454db80 1 -- 172.21.15.16:0/3665927197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 msgr2=0x55a513806d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.533+0000 7f61a454db80 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513806d20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f618c005810 tx=0x7f618c01eaa0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- 172.21.15.16:0/3665927197 shutdown_connections 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816b30 0x55a513821ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513810ca0 0x55a513806d20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 --2- 172.21.15.16:0/3665927197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513816210 0x55a513821580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- 172.21.15.16:0/3665927197 >> 172.21.15.16:0/3665927197 conn(0x55a51376a210 msgr2=0x55a51381a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- 172.21.15.16:0/3665927197 shutdown_connections 2024-05-08T23:31:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- 172.21.15.16:0/3665927197 wait complete. 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- start start 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 Processor -- start 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a513874aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a513874fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513875520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51387fb20 con 0x55a513816b30 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51387fc60 con 0x55a513816210 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51387fe20 con 0x55a513810ca0 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f61a454db80 10 client.0 _ll_register_callbacks cb 0x55a513701b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f6196930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513875520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f6196930700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513875520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47518/0 (socket says 172.21.15.16:47518) 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.534+0000 7f6196930700 1 -- 172.21.15.16:0/1118971067 learned_addr learned my addr 172.21.15.16:0/1118971067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f619612f700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a513874aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f619592e700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a513874fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6196930700 1 -- 172.21.15.16:0/1118971067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 msgr2=0x55a513874aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6196930700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a513874aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6196930700 1 -- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 msgr2=0x55a513874fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6196930700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a513874fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6196930700 1 -- 172.21.15.16:0/1118971067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c00fa20 con 0x55a513816b30 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6196930700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513875520 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61880975b0 tx=0x7f618809b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbefd420ace5a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6179ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6179ffb700 1 -- 172.21.15.16:0/1118971067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618809eee0 con 0x55a513816b30 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6179ffb700 1 -- 172.21.15.16:0/1118971067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f61880a34a0 con 0x55a513816b30 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f618c011d10 con 0x55a513816b30 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6179ffb700 1 -- 172.21.15.16:0/1118971067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61880a10b0 con 0x55a513816b30 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f61a454db80 10 client.18006 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a513722560 con 0x55a513816b30 2024-05-08T23:31:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.535+0000 7f6179ffb700 1 -- 172.21.15.16:0/1118971067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f618c011d10 con 0x55a513816b30 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f6179ffb700 1 -- 172.21.15.16:0/1118971067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61880b9250 con 0x55a513816b30 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f61a454db80 10 client.18006 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f61a454db80 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f61a454db80 20 client.18006 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f61a454db80 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a513881220 0x55a513883610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f61a454db80 4 client.18006 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a513754ec0 con 0x55a513881220 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f619612f700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a513881220 0x55a513883610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f619612f700 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a513881220 0x55a513883610 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5ca241add2a50 server_cookie=719f614566da6b7 in_seq=0 out_seq=0 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.536+0000 7f6179ffb700 10 client.18006 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f6179ffb700 1 -- 172.21.15.16:0/1118971067 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a513754ec0 con 0x55a513881220 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f6179ffb700 10 client.18006 handle_command_reply: tid=0 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 client.18006 shutdown 2024-05-08T23:31:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 msgr2=0x55a513875520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513875520 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f61880975b0 tx=0x7f618809b7b0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 shutdown_connections 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a513816b30 0x55a513875520 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a513810ca0 0x55a513874aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55a513881220 0x55a513883610 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 --2- 172.21.15.16:0/1118971067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55a513816210 0x55a513874fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 >> 172.21.15.16:0/1118971067 conn(0x55a51376a210 msgr2=0x55a513815050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f6179ffb700 0 client.18006 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 shutdown_connections 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 1 -- 172.21.15.16:0/1118971067 wait complete. 2024-05-08T23:31:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.537+0000 7f61a454db80 20 client.18006 trim_cache size 0 max 16384 2024-05-08T23:31:11.544 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:31:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:31:11.544 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:11.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f617c0a1310 msgr2=0x7f617c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f617c0a1310 0x7f617c0a37d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f618c005950 tx=0x7f618c0058e0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 msgr2=0x7f619019c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 0x7f619019c110 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f61880094c0 tx=0x7f61880094f0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 shutdown_connections 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f617c0a1310 0x7f617c0a37d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619019bc90 0x7f619019c110 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6190100060 0x7f61901973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 --2- 172.21.15.16:0/419811141 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6190105160 0x7f61901978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 >> 172.21.15.16:0/419811141 conn(0x7f6190076f40 msgr2=0x7f6190068350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 shutdown_connections 2024-05-08T23:31:11.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.538+0000 7f6198393700 1 -- 172.21.15.16:0/419811141 wait complete. 2024-05-08T23:31:11.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:31:11.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 -- 172.21.15.16:0/18645460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6681008f0 msgr2=0x7fd668104930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 --2- 172.21.15.16:0/18645460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6681008f0 0x7fd668104930 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd658009990 tx=0x7fd658009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 -- 172.21.15.16:0/18645460 shutdown_connections 2024-05-08T23:31:11.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 --2- 172.21.15.16:0/18645460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd668105060 0x7fd66810cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 --2- 172.21.15.16:0/18645460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6680fff40 0x7fd668100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 --2- 172.21.15.16:0/18645460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6681008f0 0x7fd668104930 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 -- 172.21.15.16:0/18645460 >> 172.21.15.16:0/18645460 conn(0x7fd6680fb7e0 msgr2=0x7fd6680fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 -- 172.21.15.16:0/18645460 shutdown_connections 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.688+0000 7fd66ff32700 1 -- 172.21.15.16:0/18645460 wait complete. 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 Processor -- start 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 -- start start 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 0x7fd6681019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd668105060 0x7fd668101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd668102450 0x7fd6681a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681a5970 con 0x7fd6680fff40 2024-05-08T23:31:11.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681a5ae0 con 0x7fd668105060 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.689+0000 7fd66ff32700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6681a5ca0 con 0x7fd668102450 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 0x7fd6681019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 0x7fd6681019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47536/0 (socket says 172.21.15.16:47536) 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 -- 172.21.15.16:0/3611448745 learned_addr learned my addr 172.21.15.16:0/3611448745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66d4cd700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd668105060 0x7fd668101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66e4cf700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd668102450 0x7fd6681a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 -- 172.21.15.16:0/3611448745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd668102450 msgr2=0x7fd6681a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd668102450 0x7fd6681a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 -- 172.21.15.16:0/3611448745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd668105060 msgr2=0x7fd668101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd668105060 0x7fd668101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 -- 172.21.15.16:0/3611448745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6580095f0 con 0x7fd6680fff40 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66dcce700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 0x7fd6681019d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd66400d980 tx=0x7fd66400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664004220 con 0x7fd6680fff40 2024-05-08T23:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd664007640 con 0x7fd6680fff40 2024-05-08T23:31:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664005660 con 0x7fd6680fff40 2024-05-08T23:31:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6681a5f80 con 0x7fd6680fff40 2024-05-08T23:31:11.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.690+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6681a63b0 con 0x7fd6680fff40 2024-05-08T23:31:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.691+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd668109340 con 0x7fd6680fff40 2024-05-08T23:31:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.693+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd664004750 con 0x7fd6680fff40 2024-05-08T23:31:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.693+0000 7fd65effd700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd6540a1290 0x7fd6540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.693+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7fd6640cd290 con 0x7fd6680fff40 2024-05-08T23:31:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.693+0000 7fd66d4cd700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd6540a1290 0x7fd6540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:11.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.694+0000 7fd66d4cd700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd6540a1290 0x7fd6540a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd668103100 tx=0x7fd65800b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.697+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd6640962f0 con 0x7fd6680fff40 2024-05-08T23:31:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.970+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd668101410 con 0x7fd6680fff40 2024-05-08T23:31:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.970+0000 7fd65effd700 1 -- 172.21.15.16:0/3611448745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7fd664016090 con 0x7fd6680fff40 2024-05-08T23:31:11.970 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:31:11.970 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd6540a1290 msgr2=0x7fd6540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd6540a1290 0x7fd6540a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd668103100 tx=0x7fd65800b260 comp rx=0 tx=0).stop 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 msgr2=0x7fd6681019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 0x7fd6681019d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd66400d980 tx=0x7fd66400dc90 comp rx=0 tx=0).stop 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 shutdown_connections 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd6540a1290 0x7fd6540a3750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd668102450 0x7fd6681a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6680fff40 0x7fd6681019d0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 --2- 172.21.15.16:0/3611448745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd668105060 0x7fd668101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 >> 172.21.15.16:0/3611448745 conn(0x7fd6680fb7e0 msgr2=0x7fd668068460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 shutdown_connections 2024-05-08T23:31:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:11.973+0000 7fd66ff32700 1 -- 172.21.15.16:0/3611448745 wait complete. 2024-05-08T23:31:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:31:11.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:31:12.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:11 smithi016 ceph-mon[42541]: pgmap v362: 129 pgs: 129 active+clean; 22 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 32 op/s 2024-05-08T23:31:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.126+0000 7f7cc359e700 1 -- 172.21.15.16:0/1956632637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc100f90 msgr2=0x7f7cbc101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.126+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1956632637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc100f90 0x7f7cbc101370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009990 tx=0x7f7cb0009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 -- 172.21.15.16:0/1956632637 shutdown_connections 2024-05-08T23:31:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1956632637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc1060b0 0x7f7cbc10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1956632637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc100f90 0x7f7cbc101370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1956632637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc101940 0x7f7cbc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 -- 172.21.15.16:0/1956632637 >> 172.21.15.16:0/1956632637 conn(0x7f7cbc0fc870 msgr2=0x7f7cbc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 -- 172.21.15.16:0/1956632637 shutdown_connections 2024-05-08T23:31:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 -- 172.21.15.16:0/1956632637 wait complete. 2024-05-08T23:31:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 Processor -- start 2024-05-08T23:31:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.127+0000 7f7cc359e700 1 -- start start 2024-05-08T23:31:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 0x7f7cbc1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc359e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc1060b0 0x7f7cbc197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc359e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc19bac0 0x7f7cbc19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc359e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc19c540 con 0x7f7cbc100f90 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc359e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc19c6b0 con 0x7f7cbc19bac0 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc359e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc19c870 con 0x7f7cbc1060b0 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 0x7f7cbc1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 0x7f7cbc1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47542/0 (socket says 172.21.15.16:47542) 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 -- 172.21.15.16:0/1639867182 learned_addr learned my addr 172.21.15.16:0/1639867182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc1060b0 0x7f7cbc197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc19bac0 0x7f7cbc19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 -- 172.21.15.16:0/1639867182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc1060b0 msgr2=0x7f7cbc197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc1060b0 0x7f7cbc197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 -- 172.21.15.16:0/1639867182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc19bac0 msgr2=0x7f7cbc19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc19bac0 0x7f7cbc19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 -- 172.21.15.16:0/1639867182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00095f0 con 0x7f7cbc100f90 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc19bac0 0x7f7cbc19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.128+0000 7f7cc259c700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 0x7f7cbc1971e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000c000 tx=0x7f7cb0004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.129+0000 7f7caf7fe700 1 -- 172.21.15.16:0/1639867182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb001d070 con 0x7f7cbc100f90 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.129+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc1a3d70 con 0x7f7cbc100f90 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.129+0000 7f7caf7fe700 1 -- 172.21.15.16:0/1639867182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cb000fdc0 con 0x7f7cbc100f90 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.129+0000 7f7caf7fe700 1 -- 172.21.15.16:0/1639867182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb00173f0 con 0x7f7cbc100f90 2024-05-08T23:31:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.129+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc1a41a0 con 0x7f7cbc100f90 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.131+0000 7f7caf7fe700 1 -- 172.21.15.16:0/1639867182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7cb0022670 con 0x7f7cbc100f90 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.132+0000 7f7caf7fe700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7ca80a1290 0x7f7ca80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.132+0000 7f7caf7fe700 1 -- 172.21.15.16:0/1639867182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f7cb00ce4d0 con 0x7f7cbc100f90 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.132+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7ca80a1290 0x7f7ca80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.132+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7ca80a1290 0x7f7ca80a3750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc198730 tx=0x7f7cb4009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.132+0000 7f7cd09bab80 1 Processor -- start 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 -- start start 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a372b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 0x5600a3725350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3725cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a367c150 con 0x5600a372b970 2024-05-08T23:31:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3679b10 con 0x5600a372b050 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3675400 con 0x5600a3725890 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 0x5600a3725350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 0x5600a3725350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47548/0 (socket says 172.21.15.16:47548) 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/268372410 learned_addr learned my addr 172.21.15.16:0/268372410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc259c700 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a372b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3725cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/268372410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 msgr2=0x5600a3725cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3725cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/268372410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 msgr2=0x5600a372b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a372b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/268372410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a34f63b0 con 0x5600a372b970 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.133+0000 7f7cc259c700 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a372b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.134+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 0x5600a3725350 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011de0 tx=0x7f7cb4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82e73c202ac8631 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.134+0000 7f7cacff9700 1 -- 172.21.15.16:0/268372410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb401f420 con 0x5600a372b970 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.134+0000 7f7cacff9700 1 -- 172.21.15.16:0/268372410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cb4016070 con 0x5600a372b970 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.134+0000 7f7cacff9700 1 -- 172.21.15.16:0/268372410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb401d840 con 0x5600a372b970 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 -- 172.21.15.16:0/268372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 msgr2=0x5600a3725350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 0x5600a3725350 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4011de0 tx=0x7f7cb4014000 comp rx=0 tx=0).stop 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 -- 172.21.15.16:0/268372410 shutdown_connections 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3725cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a372b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/268372410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b970 0x5600a3725350 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 -- 172.21.15.16:0/268372410 >> 172.21.15.16:0/268372410 conn(0x5600a3683210 msgr2=0x5600a37202d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 -- 172.21.15.16:0/268372410 shutdown_connections 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 -- 172.21.15.16:0/268372410 wait complete. 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 -- start start 2024-05-08T23:31:12.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.135+0000 7f7cd09bab80 1 Processor -- start 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a37318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b050 0x5600a372ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a372f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3611560 con 0x5600a3725890 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3611200 con 0x5600a372b970 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a34f86a0 con 0x5600a372b050 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 10 client.0 _ll_register_callbacks cb 0x5600a361ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a37318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a37318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47554/0 (socket says 172.21.15.16:47554) 2024-05-08T23:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc1d9b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b050 0x5600a372ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 -- 172.21.15.16:0/1702257945 learned_addr learned my addr 172.21.15.16:0/1702257945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a372f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 -- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b050 msgr2=0x5600a372ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b050 0x5600a372ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 -- 172.21.15.16:0/1702257945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 msgr2=0x5600a372f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a372f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 -- 172.21.15.16:0/1702257945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb4011ac0 con 0x5600a3725890 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a372f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cc259c700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a37318a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f7cb009b210 tx=0x7f7cb00224f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380f1ada6f1c4f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7ca5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/1702257945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb009a970 con 0x5600a3725890 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/1702257945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f7cb009f0b0 con 0x5600a3725890 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.136+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a35ce270 con 0x5600a3725890 2024-05-08T23:31:12.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/1702257945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb00a3670 con 0x5600a3725890 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 10 client.18024 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5600a34a2b60 con 0x5600a3725890 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/1702257945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5600a35ce270 con 0x5600a3725890 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/1702257945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cb000f930 con 0x5600a3725890 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 10 client.18024 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 10 client.18024 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 20 client.18024 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a373ee10 0x5600a3741230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 4 client.18024 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a36f4810 con 0x5600a373ee10 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.137+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a373ee10 0x5600a3741230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.138+0000 7f7ca5ffb700 10 client.18024 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.138+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a373ee10 0x5600a3741230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9d914fa4e7f0bbc server_cookie=376c04bc4ba69023 in_seq=0 out_seq=0 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.138+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/1702257945 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600a36f4810 con 0x5600a373ee10 2024-05-08T23:31:12.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.138+0000 7f7ca5ffb700 10 client.18024 handle_command_reply: tid=0 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.138+0000 7f7cd09bab80 1 client.18024 shutdown 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 msgr2=0x5600a37318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a37318a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f7cb009b210 tx=0x7f7cb00224f0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 shutdown_connections 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a373ee10 0x5600a3741230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a372f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a37318a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/1702257945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b050 0x5600a372ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7ca5ffb700 0 client.18024 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 >> 172.21.15.16:0/1702257945 conn(0x5600a3683210 msgr2=0x5600a37200b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 shutdown_connections 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 1 -- 172.21.15.16:0/1702257945 wait complete. 2024-05-08T23:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.139+0000 7f7cd09bab80 20 client.18024 trim_cache size 0 max 16384 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 Processor -- start 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 -- start start 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a3725350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a37313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b970 0x5600a372d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a372b430 con 0x5600a3725890 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a372fad0 con 0x5600a372b050 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.153+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a372d130 con 0x5600a372b970 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a3725350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a3725350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47566/0 (socket says 172.21.15.16:47566) 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 -- 172.21.15.16:0/456884749 learned_addr learned my addr 172.21.15.16:0/456884749 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b970 0x5600a372d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a37313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 -- 172.21.15.16:0/456884749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b970 msgr2=0x5600a372d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b970 0x5600a372d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 -- 172.21.15.16:0/456884749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 msgr2=0x5600a37313d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a37313d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 -- 172.21.15.16:0/456884749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a363b560 con 0x5600a3725890 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b970 0x5600a372d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cc259c700 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a3725350 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00059d0 tx=0x7f7cb0005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581734dfd6c549b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cad7fa700 1 -- 172.21.15.16:0/456884749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb009b050 con 0x5600a3725890 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cad7fa700 1 -- 172.21.15.16:0/456884749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cb0095e90 con 0x5600a3725890 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.154+0000 7f7cad7fa700 1 -- 172.21.15.16:0/456884749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb00a31e0 con 0x5600a3725890 2024-05-08T23:31:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 -- 172.21.15.16:0/456884749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 msgr2=0x5600a3725350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a3725350 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00059d0 tx=0x7f7cb0005940 comp rx=0 tx=0).stop 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 -- 172.21.15.16:0/456884749 shutdown_connections 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a372b970 0x5600a372d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a3725890 0x5600a3725350 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/456884749 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b050 0x5600a37313d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 -- 172.21.15.16:0/456884749 >> 172.21.15.16:0/456884749 conn(0x5600a3683210 msgr2=0x5600a3728c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 -- 172.21.15.16:0/456884749 shutdown_connections 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 -- 172.21.15.16:0/456884749 wait complete. 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 -- start start 2024-05-08T23:31:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.155+0000 7f7cd09bab80 1 Processor -- start 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3736390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 0x5600a37368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a3726a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3726f40 con 0x5600a372b050 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a37270b0 con 0x5600a372b970 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3727270 con 0x5600a3725890 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cd09bab80 10 client.0 _ll_register_callbacks cb 0x5600a361ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 0x5600a37368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 0x5600a37368d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47576/0 (socket says 172.21.15.16:47576) 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/3167426432 learned_addr learned my addr 172.21.15.16:0/3167426432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc259c700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3736390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc2d9d700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a3726a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 msgr2=0x5600a3736390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3736390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/3167426432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 msgr2=0x5600a3726a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a3726a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.156+0000 7f7cc1d9b700 1 -- 172.21.15.16:0/3167426432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600a34a2b60 con 0x5600a372b050 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cc1d9b700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 0x5600a37368d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4012010 tx=0x7f7cb4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a21903e9a850034 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7ca5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/3167426432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb402e040 con 0x5600a372b050 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/3167426432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f7cb4018070 con 0x5600a372b050 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cb4016070 con 0x5600a372b050 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/3167426432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb40054d0 con 0x5600a372b050 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 10 client.18030 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5600a34facd0 con 0x5600a372b050 2024-05-08T23:31:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/3167426432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cb4016070 con 0x5600a372b050 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/3167426432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cb403c020 con 0x5600a372b050 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 10 client.18030 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 10 client.18030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 20 client.18030 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a379a400 0x5600a379c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 4 client.18030 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.157+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5600a366dec0 con 0x5600a379a400 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cc259c700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a379a400 0x5600a379c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7ca5ffb700 10 client.18030 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cc259c700 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a379a400 0x5600a379c820 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ddbf8fc21ff389ac server_cookie=380a480cf5d8378a in_seq=0 out_seq=0 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7ca5ffb700 1 -- 172.21.15.16:0/3167426432 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5600a366dec0 con 0x5600a379a400 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7ca5ffb700 10 client.18030 handle_command_reply: tid=0 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 client.18030 shutdown 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 msgr2=0x5600a37368d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 0x5600a37368d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4012010 tx=0x7f7cb4005db0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 shutdown_connections 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x5600a379a400 0x5600a379c820 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5600a372b970 0x5600a3726a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600a3725890 0x5600a3736390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 --2- 172.21.15.16:0/3167426432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600a372b050 0x5600a37368d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7ca5ffb700 0 client.18030 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 >> 172.21.15.16:0/3167426432 conn(0x5600a3683210 msgr2=0x5600a372d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.158+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 shutdown_connections 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cd09bab80 1 -- 172.21.15.16:0/3167426432 wait complete. 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cd09bab80 20 client.18030 trim_cache size 0 max 16384 2024-05-08T23:31:12.164 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:31:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7ca80a1290 msgr2=0x7f7ca80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7ca80a1290 0x7f7ca80a3750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc198730 tx=0x7f7cb4009260 comp rx=0 tx=0).stop 2024-05-08T23:31:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 msgr2=0x7f7cbc1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 0x7f7cbc1971e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000c000 tx=0x7f7cb0004500 comp rx=0 tx=0).stop 2024-05-08T23:31:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 shutdown_connections 2024-05-08T23:31:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f7ca80a1290 0x7f7ca80a3750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7cbc19bac0 0x7f7cbc19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cbc100f90 0x7f7cbc1971e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 --2- 172.21.15.16:0/1639867182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7cbc1060b0 0x7f7cbc197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 >> 172.21.15.16:0/1639867182 conn(0x7f7cbc0fc870 msgr2=0x7f7cbc0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 shutdown_connections 2024-05-08T23:31:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.159+0000 7f7cc359e700 1 -- 172.21.15.16:0/1639867182 wait complete. 2024-05-08T23:31:12.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:11 smithi189 ceph-mon[46919]: pgmap v362: 129 pgs: 129 active+clean; 22 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 32 op/s 2024-05-08T23:31:12.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:31:12.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:11 smithi155 ceph-mon[47105]: pgmap v362: 129 pgs: 129 active+clean; 22 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 32 op/s 2024-05-08T23:31:12.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 -- 172.21.15.16:0/2240680037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0100060 msgr2=0x7f30e0100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 --2- 172.21.15.16:0/2240680037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0100060 0x7f30e0100440 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009990 tx=0x7f30dc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 -- 172.21.15.16:0/2240680037 shutdown_connections 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 --2- 172.21.15.16:0/2240680037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e0105160 0x7f30e010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 --2- 172.21.15.16:0/2240680037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0100060 0x7f30e0100440 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 --2- 172.21.15.16:0/2240680037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30e0100a10 0x7f30e0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 -- 172.21.15.16:0/2240680037 >> 172.21.15.16:0/2240680037 conn(0x7f30e0076f40 msgr2=0x7f30e0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 -- 172.21.15.16:0/2240680037 shutdown_connections 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 -- 172.21.15.16:0/2240680037 wait complete. 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 Processor -- start 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.312+0000 7f30e8018700 1 -- start start 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e8018700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e0100060 0x7f30e01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e8018700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 0x7f30e01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e8018700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30e019bb90 0x7f30e019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e8018700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e019c610 con 0x7f30e0105160 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e8018700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e019c780 con 0x7f30e019bb90 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e8018700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e019c940 con 0x7f30e0100060 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 0x7f30e01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 0x7f30e01977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47592/0 (socket says 172.21.15.16:47592) 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 -- 172.21.15.16:0/1631512140 learned_addr learned my addr 172.21.15.16:0/1631512140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e65b5700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30e019bb90 0x7f30e019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e5db4700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e0100060 0x7f30e01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 -- 172.21.15.16:0/1631512140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e0100060 msgr2=0x7f30e01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e0100060 0x7f30e01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 -- 172.21.15.16:0/1631512140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30e019bb90 msgr2=0x7f30e019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30e019bb90 0x7f30e019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 -- 172.21.15.16:0/1631512140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0095f0 con 0x7f30e0105160 2024-05-08T23:31:12.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30e55b3700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 0x7f30e01977f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c9e0 tx=0x7f30d000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.313+0000 7f30d6ffd700 1 -- 172.21.15.16:0/1631512140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0007aa0 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.314+0000 7f30d6ffd700 1 -- 172.21.15.16:0/1631512140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30d0007c00 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.314+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e01a3ea0 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.314+0000 7f30d6ffd700 1 -- 172.21.15.16:0/1631512140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0018660 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.314+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e01a4370 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.316+0000 7f30d6ffd700 1 -- 172.21.15.16:0/1631512140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f30d000f450 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.317+0000 7f30d6ffd700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f30cc0a1310 0x7f30cc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.317+0000 7f30e5db4700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f30cc0a1310 0x7f30cc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.317+0000 7f30d6ffd700 1 -- 172.21.15.16:0/1631512140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f30d00cd030 con 0x7f30e0105160 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.318+0000 7f30e5db4700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f30cc0a1310 0x7f30cc0a37d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30dc006010 tx=0x7f30dc00b540 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.318+0000 7f30f41d2b80 1 Processor -- start 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.318+0000 7f30f41d2b80 1 -- start start 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba95350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba95cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b9ec150 con 0x55731ba9ae00 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b9e9b10 con 0x55731ba95890 2024-05-08T23:31:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b9e5400 con 0x55731ba9b720 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47600/0 (socket says 172.21.15.16:47600) 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 -- 172.21.15.16:0/3526310030 learned_addr learned my addr 172.21.15.16:0/3526310030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e65b5700 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba95cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e55b3700 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba95350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 -- 172.21.15.16:0/3526310030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 msgr2=0x55731ba95350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba95350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 -- 172.21.15.16:0/3526310030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 msgr2=0x55731ba95cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba95cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 -- 172.21.15.16:0/3526310030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55731b8663b0 con 0x55731ba9ae00 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e55b3700 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba95350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.319+0000 7f30e5db4700 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9b1e0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0053e0 tx=0x7f30dc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb1fb677319038e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30cbfff700 1 -- 172.21.15.16:0/3526310030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc02b070 con 0x55731ba9ae00 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30cbfff700 1 -- 172.21.15.16:0/3526310030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30dc01e040 con 0x55731ba9ae00 2024-05-08T23:31:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30cbfff700 1 -- 172.21.15.16:0/3526310030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc004aa0 con 0x55731ba9ae00 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 -- 172.21.15.16:0/3526310030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 msgr2=0x55731ba9b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9b1e0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0053e0 tx=0x7f30dc01f000 comp rx=0 tx=0).stop 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 -- 172.21.15.16:0/3526310030 shutdown_connections 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba95cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9b1e0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/3526310030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba95350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.320+0000 7f30f41d2b80 1 -- 172.21.15.16:0/3526310030 >> 172.21.15.16:0/3526310030 conn(0x55731b9f3210 msgr2=0x55731ba8fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.321+0000 7f30f41d2b80 1 -- 172.21.15.16:0/3526310030 shutdown_connections 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.321+0000 7f30f41d2b80 1 -- 172.21.15.16:0/3526310030 wait complete. 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.321+0000 7f30f41d2b80 1 -- start start 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.321+0000 7f30f41d2b80 1 Processor -- start 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.321+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9b720 0x55731ba9f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b981560 con 0x55731ba9ae00 2024-05-08T23:31:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b981200 con 0x55731ba9b720 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731b8686a0 con 0x55731ba95890 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 10 client.0 _ll_register_callbacks cb 0x55731b98ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e5db4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47602/0 (socket says 172.21.15.16:47602) 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 -- 172.21.15.16:0/1600562823 learned_addr learned my addr 172.21.15.16:0/1600562823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e65b5700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9b720 0x55731ba9f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 -- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 msgr2=0x55731baa1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 -- 172.21.15.16:0/1600562823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9b720 msgr2=0x55731ba9f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9b720 0x55731ba9f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 -- 172.21.15.16:0/1600562823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e0067ff0 con 0x55731ba9ae00 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e5db4700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30e55b3700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9e9a0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f30d00074a0 tx=0x7f30d001f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c712a1f714479316 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30c97fa700 1 -- 172.21.15.16:0/1600562823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00a21d0 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30c97fa700 1 -- 172.21.15.16:0/1600562823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f30d0099fb0 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.322+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55731b93e270 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30c97fa700 1 -- 172.21.15.16:0/1600562823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30d00b8080 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 10 client.18042 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55731b812b60 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30c97fa700 1 -- 172.21.15.16:0/1600562823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30d007c070 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30c97fa700 1 -- 172.21.15.16:0/1600562823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f30d00a1d40 con 0x55731ba9ae00 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 10 client.18042 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 10 client.18042 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 20 client.18042 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731baaef10 0x55731bab1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 4 client.18042 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55731ba64810 con 0x55731baaef10 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.323+0000 7f30e5db4700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731baaef10 0x55731bab1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.324+0000 7f30c97fa700 10 client.18042 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.324+0000 7f30e5db4700 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731baaef10 0x55731bab1300 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=964b8d091b04c54f server_cookie=b1ff506619f43341 in_seq=0 out_seq=0 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.324+0000 7f30c97fa700 1 -- 172.21.15.16:0/1600562823 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55731ba64810 con 0x55731baaef10 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.324+0000 7f30c97fa700 10 client.18042 handle_command_reply: tid=0 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 client.18042 shutdown 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 msgr2=0x55731ba9e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9e9a0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f30d00074a0 tx=0x7f30d001f430 comp rx=0 tx=0).stop 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 shutdown_connections 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731baaef10 0x55731bab1300 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9b720 0x55731ba9f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/1600562823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731ba9e9a0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30c97fa700 0 client.18042 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 >> 172.21.15.16:0/1600562823 conn(0x55731b9f3210 msgr2=0x55731ba8f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 shutdown_connections 2024-05-08T23:31:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 1 -- 172.21.15.16:0/1600562823 wait complete. 2024-05-08T23:31:12.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.325+0000 7f30f41d2b80 20 client.18042 trim_cache size 0 max 16384 2024-05-08T23:31:12.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 Processor -- start 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 -- start start 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731baa1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba9e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731baaba60 con 0x55731ba9ae00 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731ba99410 con 0x55731ba95890 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731ba9b1e0 con 0x55731ba9b720 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731baa1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731baa1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47608/0 (socket says 172.21.15.16:47608) 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 -- 172.21.15.16:0/70169053 learned_addr learned my addr 172.21.15.16:0/70169053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e65b5700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba9e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e5db4700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 -- 172.21.15.16:0/70169053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 msgr2=0x55731ba9e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba9e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 -- 172.21.15.16:0/70169053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 msgr2=0x55731ba8fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba8fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.339+0000 7f30e55b3700 1 -- 172.21.15.16:0/70169053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55731ba95f40 con 0x55731ba9ae00 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30e65b5700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30e5db4700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba8fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30e55b3700 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731baa1250 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f30d00ca990 tx=0x7f30d0098300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfde170a1c06c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30d4ff9700 1 -- 172.21.15.16:0/70169053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0099b20 con 0x55731ba9ae00 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30d4ff9700 1 -- 172.21.15.16:0/70169053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30d007c040 con 0x55731ba9ae00 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30d4ff9700 1 -- 172.21.15.16:0/70169053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0094960 con 0x55731ba9ae00 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 -- 172.21.15.16:0/70169053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 msgr2=0x55731baa1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731baa1250 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f30d00ca990 tx=0x7f30d0098300 comp rx=0 tx=0).stop 2024-05-08T23:31:12.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 -- 172.21.15.16:0/70169053 shutdown_connections 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba9b720 0x55731ba9e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba95890 0x55731ba8fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/70169053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9ae00 0x55731baa1250 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.340+0000 7f30f41d2b80 1 -- 172.21.15.16:0/70169053 >> 172.21.15.16:0/70169053 conn(0x55731b9f3210 msgr2=0x55731baaab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 -- 172.21.15.16:0/70169053 shutdown_connections 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 -- 172.21.15.16:0/70169053 wait complete. 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 -- start start 2024-05-08T23:31:12.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 Processor -- start 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9ae00 0x55731ba97a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 0x55731ba980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731ba969a0 con 0x55731ba9b720 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731ba96b10 con 0x55731ba9ae00 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.341+0000 7f30f41d2b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731ba96cd0 con 0x55731ba95890 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30f41d2b80 10 client.0 _ll_register_callbacks cb 0x55731b98ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 0x55731ba980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 0x55731ba980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47612/0 (socket says 172.21.15.16:47612) 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 -- 172.21.15.16:0/166035226 learned_addr learned my addr 172.21.15.16:0/166035226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e5db4700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e55b3700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9ae00 0x55731ba97a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 -- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 msgr2=0x55731baa6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 -- 172.21.15.16:0/166035226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9ae00 msgr2=0x55731ba97a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9ae00 0x55731ba97a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 -- 172.21.15.16:0/166035226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d00b7040 con 0x55731ba9b720 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e5db4700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30e65b5700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 0x55731ba980c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007ab0 tx=0x7f30d8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27026cda8467f1f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30c97fa700 1 -- 172.21.15.16:0/166035226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8015b50 con 0x55731ba9b720 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30c97fa700 1 -- 172.21.15.16:0/166035226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f30d8015cb0 con 0x55731ba9b720 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.342+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30dc01e040 con 0x55731ba9b720 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30c97fa700 1 -- 172.21.15.16:0/166035226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30d8015e70 con 0x55731ba9b720 2024-05-08T23:31:12.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 10 client.18048 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55731ba8c4e0 con 0x55731ba9b720 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30c97fa700 1 -- 172.21.15.16:0/166035226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30dc01e040 con 0x55731ba9b720 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30c97fa700 1 -- 172.21.15.16:0/166035226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f30d8021540 con 0x55731ba9b720 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 10 client.18048 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 10 client.18048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 20 client.18048 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731bb0a260 0x55731bb0c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 4 client.18048 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55731b9ddec0 con 0x55731bb0a260 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.343+0000 7f30e5db4700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731bb0a260 0x55731bb0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30c97fa700 10 client.18048 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30e5db4700 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731bb0a260 0x55731bb0c680 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e4eca559452ecca server_cookie=6db74989019e1563 in_seq=0 out_seq=0 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30c97fa700 1 -- 172.21.15.16:0/166035226 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55731b9ddec0 con 0x55731bb0a260 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30c97fa700 10 client.18048 handle_command_reply: tid=0 2024-05-08T23:31:12.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 client.18048 shutdown 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 msgr2=0x55731ba980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 0x55731ba980c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007ab0 tx=0x7f30d8007dc0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 shutdown_connections 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55731ba9b720 0x55731ba980c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55731ba95890 0x55731baa6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55731bb0a260 0x55731bb0c680 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 --2- 172.21.15.16:0/166035226 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55731ba9ae00 0x55731ba97a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 >> 172.21.15.16:0/166035226 conn(0x55731b9f3210 msgr2=0x55731ba9bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30c97fa700 0 client.18048 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 shutdown_connections 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 1 -- 172.21.15.16:0/166035226 wait complete. 2024-05-08T23:31:12.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.344+0000 7f30f41d2b80 20 client.18048 trim_cache size 0 max 16384 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f30cc0a1310 msgr2=0x7f30cc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f30cc0a1310 0x7f30cc0a37d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30dc006010 tx=0x7f30dc00b540 comp rx=0 tx=0).stop 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 msgr2=0x7f30e01977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 0x7f30e01977f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c9e0 tx=0x7f30d000cda0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 shutdown_connections 2024-05-08T23:31:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f30cc0a1310 0x7f30cc0a37d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30e019bb90 0x7f30e019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e0100060 0x7f30e01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 --2- 172.21.15.16:0/1631512140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30e0105160 0x7f30e01977f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 >> 172.21.15.16:0/1631512140 conn(0x7f30e0076f40 msgr2=0x7f30e00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 shutdown_connections 2024-05-08T23:31:12.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.345+0000 7f30e8018700 1 -- 172.21.15.16:0/1631512140 wait complete. 2024-05-08T23:31:12.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:31:12.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 -- 172.21.15.16:0/2742854213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c01062a0 msgr2=0x7ff1c010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 --2- 172.21.15.16:0/2742854213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c01062a0 0x7ff1c010de20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800b0a0 tx=0x7ff1b800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 -- 172.21.15.16:0/2742854213 shutdown_connections 2024-05-08T23:31:12.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 --2- 172.21.15.16:0/2742854213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c01062a0 0x7ff1c010de20 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 --2- 172.21.15.16:0/2742854213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 0x7ff1c00697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 --2- 172.21.15.16:0/2742854213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1c0069d90 0x7ff1c0105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 -- 172.21.15.16:0/2742854213 >> 172.21.15.16:0/2742854213 conn(0x7ff1c0076ba0 msgr2=0x7ff1c0076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 -- 172.21.15.16:0/2742854213 shutdown_connections 2024-05-08T23:31:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 -- 172.21.15.16:0/2742854213 wait complete. 2024-05-08T23:31:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 Processor -- start 2024-05-08T23:31:12.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.498+0000 7ff1c6605700 1 -- start start 2024-05-08T23:31:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c6605700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 0x7ff1c01001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c6605700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1c01062a0 0x7ff1c0100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c6605700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 0x7ff1c01a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c6605700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c01a5c10 con 0x7ff1c0100c40 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c6605700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c01a5d80 con 0x7ff1c00693e0 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c6605700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c01a5f40 con 0x7ff1c01062a0 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 0x7ff1c01a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 0x7ff1c01a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47626/0 (socket says 172.21.15.16:47626) 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/174060703 learned_addr learned my addr 172.21.15.16:0/174060703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1bffff700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 0x7ff1c01001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1c01062a0 0x7ff1c0100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/174060703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1c01062a0 msgr2=0x7ff1c0100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1c01062a0 0x7ff1c0100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/174060703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 msgr2=0x7ff1c01001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 0x7ff1c01001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/174060703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009d00 con 0x7ff1c0100c40 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.499+0000 7ff1bffff700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 0x7ff1c01001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.500+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 0x7ff1c01a5640 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80094c0 tx=0x7ff1b80075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.500+0000 7ff1bd7fa700 1 -- 172.21.15.16:0/174060703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800e040 con 0x7ff1c0100c40 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.500+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c01a61c0 con 0x7ff1c0100c40 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.500+0000 7ff1bd7fa700 1 -- 172.21.15.16:0/174060703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1b8003c00 con 0x7ff1c0100c40 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.500+0000 7ff1bd7fa700 1 -- 172.21.15.16:0/174060703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80052a0 con 0x7ff1c0100c40 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.500+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c01a6640 con 0x7ff1c0100c40 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.502+0000 7ff1bd7fa700 1 -- 172.21.15.16:0/174060703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff1b8005400 con 0x7ff1c0100c40 2024-05-08T23:31:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.503+0000 7ff1bd7fa700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1a80a1290 0x7ff1a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.503+0000 7ff1bffff700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1a80a1290 0x7ff1a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.503+0000 7ff1bd7fa700 1 -- 172.21.15.16:0/174060703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7ff1b80ce380 con 0x7ff1c0100c40 2024-05-08T23:31:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.503+0000 7ff1bffff700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1a80a1290 0x7ff1a80a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000bc10 tx=0x7ff1b000b5f0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 Processor -- start 2024-05-08T23:31:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 -- start start 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b113194350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b113194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1130eb150 con 0x55b11319a720 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1130e8b10 con 0x55b113199e00 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1130e4400 con 0x55b113194890 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b113194350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b113194350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47634/0 (socket says 172.21.15.16:47634) 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1bf7fe700 1 -- 172.21.15.16:0/790479035 learned_addr learned my addr 172.21.15.16:0/790479035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1bffff700 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.504+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b113194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1bf7fe700 1 -- 172.21.15.16:0/790479035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 msgr2=0x55b113194cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b113194cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1bf7fe700 1 -- 172.21.15.16:0/790479035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 msgr2=0x55b11319a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1bf7fe700 1 -- 172.21.15.16:0/790479035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b112f653b0 con 0x55b11319a720 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b113194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b113194350 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40046c0 tx=0x7ff1b400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911cf8ed45c3dd8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1a67fc700 1 -- 172.21.15.16:0/790479035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4004a30 con 0x55b11319a720 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1a67fc700 1 -- 172.21.15.16:0/790479035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1b400fce0 con 0x55b11319a720 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.505+0000 7ff1a67fc700 1 -- 172.21.15.16:0/790479035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4005480 con 0x55b11319a720 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/790479035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 msgr2=0x55b113194350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b113194350 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40046c0 tx=0x7ff1b400de50 comp rx=0 tx=0).stop 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/790479035 shutdown_connections 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b113194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/790479035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b113194350 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/790479035 >> 172.21.15.16:0/790479035 conn(0x55b1130f2210 msgr2=0x55b11318edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/790479035 shutdown_connections 2024-05-08T23:31:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/790479035 wait complete. 2024-05-08T23:31:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 -- start start 2024-05-08T23:31:12.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.506+0000 7ff1d27bfb80 1 Processor -- start 2024-05-08T23:31:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b11319db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11319e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b113080560 con 0x55b11319a720 2024-05-08T23:31:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b113080200 con 0x55b113199e00 2024-05-08T23:31:12.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b112f676a0 con 0x55b113194890 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1d27bfb80 10 client.0 _ll_register_callbacks cb 0x55b113089b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11319e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11319e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47650/0 (socket says 172.21.15.16:47650) 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1681937835 learned_addr learned my addr 172.21.15.16:0/1681937835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1bffff700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b11319db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1681937835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 msgr2=0x55b11319db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b11319db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 msgr2=0x55b11319e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1681937835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b400d7d0 con 0x55b11319a720 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11319e5c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80062a0 tx=0x7ff1b809b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e045ae9d7c828890 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.507+0000 7ff19f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff19f7fe700 1 -- 172.21.15.16:0/1681937835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b80a35b0 con 0x55b11319a720 2024-05-08T23:31:12.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b11303d270 con 0x55b11319a720 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff19f7fe700 1 -- 172.21.15.16:0/1681937835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7ff1b8003d70 con 0x55b11319a720 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff19f7fe700 1 -- 172.21.15.16:0/1681937835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1b8099560 con 0x55b11319a720 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 10 client.18060 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b112f11b60 con 0x55b11319a720 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff19f7fe700 1 -- 172.21.15.16:0/1681937835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1b8064020 con 0x55b11319a720 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff19f7fe700 1 -- 172.21.15.16:0/1681937835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1b80a10c0 con 0x55b11319a720 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 10 client.18060 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 20 client.18060 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1131ade30 0x55b1131b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 4 client.18060 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.508+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b113163810 con 0x55b1131ade30 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.509+0000 7ff1bffff700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1131ade30 0x55b1131b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.509+0000 7ff19f7fe700 10 client.18060 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.509+0000 7ff1bffff700 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1131ade30 0x55b1131b0250 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b975367716c36ed2 server_cookie=545c86a1bba52df6 in_seq=0 out_seq=0 2024-05-08T23:31:12.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff19f7fe700 1 -- 172.21.15.16:0/1681937835 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b113163810 con 0x55b1131ade30 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff19f7fe700 10 client.18060 handle_command_reply: tid=0 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 client.18060 shutdown 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 msgr2=0x55b11319e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11319e5c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80062a0 tx=0x7ff1b809b8c0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 shutdown_connections 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11319e5c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b11319db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b1131ade30 0x55b1131b0250 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1681937835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b11319e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff19f7fe700 0 client.18060 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 >> 172.21.15.16:0/1681937835 conn(0x55b1130f2210 msgr2=0x55b11318e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 shutdown_connections 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1681937835 wait complete. 2024-05-08T23:31:12.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.510+0000 7ff1d27bfb80 20 client.18060 trim_cache size 0 max 16384 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 Processor -- start 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 -- start start 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11318ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1131a0630 con 0x55b11319a720 2024-05-08T23:31:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11318bde0 con 0x55b113199e00 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.524+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b113190e50 con 0x55b113194890 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11318ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11318ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47652/0 (socket says 172.21.15.16:47652) 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1bffff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1724568629 learned_addr learned my addr 172.21.15.16:0/1724568629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1724568629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 msgr2=0x55b1131aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1724568629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 msgr2=0x55b1131a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/1724568629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1130aa560 con 0x55b11319a720 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11318ed20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80ce660 tx=0x7ff1b8097aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1110c05881b796 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1a6ffd700 1 -- 172.21.15.16:0/1724568629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8009b80 con 0x55b11319a720 2024-05-08T23:31:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1a6ffd700 1 -- 172.21.15.16:0/1724568629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1b8064070 con 0x55b11319a720 2024-05-08T23:31:12.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.525+0000 7ff1a6ffd700 1 -- 172.21.15.16:0/1724568629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8003e50 con 0x55b11319a720 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1724568629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 msgr2=0x55b11318ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11318ed20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80ce660 tx=0x7ff1b8097aa0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1724568629 shutdown_connections 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b11318ed20 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/1724568629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1724568629 >> 172.21.15.16:0/1724568629 conn(0x55b1130f2210 msgr2=0x55b11319c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1724568629 shutdown_connections 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/1724568629 wait complete. 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.526+0000 7ff1d27bfb80 1 -- start start 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 Processor -- start 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b1131a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b113203330 con 0x55b11319a720 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b113203470 con 0x55b113199e00 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1132035b0 con 0x55b113194890 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1d27bfb80 10 client.0 _ll_register_callbacks cb 0x55b113089b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b1131a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b1131a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47662/0 (socket says 172.21.15.16:47662) 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/4261718276 learned_addr learned my addr 172.21.15.16:0/4261718276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1bffff700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1bf7fe700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/4261718276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 msgr2=0x55b1131a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 msgr2=0x55b1131a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.527+0000 7ff1c4ba2700 1 -- 172.21.15.16:0/4261718276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b112f11b60 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1c4ba2700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b1131a4da0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff1b809b150 tx=0x7ff1b809b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18381877b4bcd52f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff19f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff19f7fe700 1 -- 172.21.15.16:0/4261718276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b809b420 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff19f7fe700 1 -- 172.21.15.16:0/4261718276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7ff1b8065080 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1b400fce0 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff19f7fe700 1 -- 172.21.15.16:0/4261718276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1b8097500 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 10 client.18066 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b112f69cd0 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff19f7fe700 1 -- 172.21.15.16:0/4261718276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1b400fce0 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff19f7fe700 1 -- 172.21.15.16:0/4261718276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1b80a10b0 con 0x55b11319a720 2024-05-08T23:31:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 10 client.18066 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 20 client.18066 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b113204830 0x55b113206c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 4 client.18066 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.528+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b1130dcec0 con 0x55b113204830 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1bffff700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b113204830 0x55b113206c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff19f7fe700 10 client.18066 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1bffff700 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b113204830 0x55b113206c20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb95c6a67dff7df server_cookie=fd918725443d1793 in_seq=0 out_seq=0 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff19f7fe700 1 -- 172.21.15.16:0/4261718276 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b1130dcec0 con 0x55b113204830 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff19f7fe700 10 client.18066 handle_command_reply: tid=0 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 client.18066 shutdown 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 msgr2=0x55b1131a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b1131a4da0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff1b809b150 tx=0x7ff1b809b340 comp rx=0 tx=0).stop 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 shutdown_connections 2024-05-08T23:31:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b11319a720 0x55b1131a4da0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b113194890 0x55b1131a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x55b113204830 0x55b113206c20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 --2- 172.21.15.16:0/4261718276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b113199e00 0x55b1131a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 >> 172.21.15.16:0/4261718276 conn(0x55b1130f2210 msgr2=0x55b113198c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.529+0000 7ff19f7fe700 0 client.18066 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 shutdown_connections 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1d27bfb80 1 -- 172.21.15.16:0/4261718276 wait complete. 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1d27bfb80 20 client.18066 trim_cache size 0 max 16384 2024-05-08T23:31:12.535 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1a80a1290 msgr2=0x7ff1a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1a80a1290 0x7ff1a80a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000bc10 tx=0x7ff1b000b5f0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 msgr2=0x7ff1c01a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 0x7ff1c01a5640 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80094c0 tx=0x7ff1b80075c0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 shutdown_connections 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1a80a1290 0x7ff1a80a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c0100c40 0x7ff1c01a5640 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1c00693e0 0x7ff1c01001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 --2- 172.21.15.16:0/174060703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1c01062a0 0x7ff1c0100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 >> 172.21.15.16:0/174060703 conn(0x7ff1c0076ba0 msgr2=0x7ff1c0107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 shutdown_connections 2024-05-08T23:31:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:12.530+0000 7ff1c6605700 1 -- 172.21.15.16:0/174060703 wait complete. 2024-05-08T23:31:13.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3611448745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:31:13.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:12 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3611448745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:31:13.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:12 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3611448745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:31:13.544 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T23:31:13.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T23:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 -- 172.21.15.16:0/1853644762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 msgr2=0x7f1f6c10ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/1853644762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c10ccf0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400b0a0 tx=0x7f1f6400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 -- 172.21.15.16:0/1853644762 shutdown_connections 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/1853644762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c10ccf0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/1853644762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 0x7f1f6c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/1853644762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6c100a00 0x7f1f6c104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 -- 172.21.15.16:0/1853644762 >> 172.21.15.16:0/1853644762 conn(0x7f1f6c076fa0 msgr2=0x7f1f6c0773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 -- 172.21.15.16:0/1853644762 shutdown_connections 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.697+0000 7f1f71b4f700 1 -- 172.21.15.16:0/1853644762 wait complete. 2024-05-08T23:31:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 Processor -- start 2024-05-08T23:31:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 -- start start 2024-05-08T23:31:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 0x7f1f6c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6c19de50 0x7f1f6c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19e8d0 con 0x7f1f6c105170 2024-05-08T23:31:13.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19ea40 con 0x7f1f6c100050 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f71b4f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19ec00 con 0x7f1f6c19de50 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.698+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47678/0 (socket says 172.21.15.16:47678) 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 -- 172.21.15.16:0/3796016336 learned_addr learned my addr 172.21.15.16:0/3796016336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6c19de50 0x7f1f6c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 0x7f1f6c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 -- 172.21.15.16:0/3796016336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6c19de50 msgr2=0x7f1f6c19e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6c19de50 0x7f1f6c19e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 -- 172.21.15.16:0/3796016336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 msgr2=0x7f1f6c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 0x7f1f6c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 -- 172.21.15.16:0/3796016336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64009d00 con 0x7f1f6c105170 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 0x7f1f6c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f6affd700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c199b00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400e970 tx=0x7f1f5400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f68ff9700 1 -- 172.21.15.16:0/3796016336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54009790 con 0x7f1f6c105170 2024-05-08T23:31:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f68ff9700 1 -- 172.21.15.16:0/3796016336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5400eec0 con 0x7f1f6c105170 2024-05-08T23:31:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c1a6160 con 0x7f1f6c105170 2024-05-08T23:31:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f68ff9700 1 -- 172.21.15.16:0/3796016336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f540054b0 con 0x7f1f6c105170 2024-05-08T23:31:13.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.699+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1a6690 con 0x7f1f6c105170 2024-05-08T23:31:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.702+0000 7f1f68ff9700 1 -- 172.21.15.16:0/3796016336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1f54004750 con 0x7f1f6c105170 2024-05-08T23:31:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.703+0000 7f1f68ff9700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f580a1310 0x7f1f580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.703+0000 7f1f68ff9700 1 -- 172.21.15.16:0/3796016336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f1f54014070 con 0x7f1f6c105170 2024-05-08T23:31:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.703+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f580a1310 0x7f1f580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.703+0000 7f1f7dd09b80 1 Processor -- start 2024-05-08T23:31:13.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.703+0000 7f1f7dd09b80 1 -- start start 2024-05-08T23:31:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f580a1310 0x7f1f580a37d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c005e30 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eda260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ed43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3e2b1d0 con 0x5558e3ed9e80 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3e28b90 con 0x5558e3ed4910 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3e24480 con 0x5558e3eda7a0 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eda260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eda260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47686/0 (socket says 172.21.15.16:47686) 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/2724472314 learned_addr learned my addr 172.21.15.16:0/2724472314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6affd700 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ed43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/2724472314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 msgr2=0x5558e3ed43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ed43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/2724472314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 msgr2=0x5558e3ed4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/2724472314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e3ca5430 con 0x5558e3ed9e80 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6affd700 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ed43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.704+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eda260 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00a5c0 tx=0x7f1f5c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b7c7a110496f52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f61ffb700 1 -- 172.21.15.16:0/2724472314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c010e50 con 0x5558e3ed9e80 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f61ffb700 1 -- 172.21.15.16:0/2724472314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c00eec0 con 0x5558e3ed9e80 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f61ffb700 1 -- 172.21.15.16:0/2724472314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00f7c0 con 0x5558e3ed9e80 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2724472314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 msgr2=0x5558e3eda260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eda260 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00a5c0 tx=0x7f1f5c011000 comp rx=0 tx=0).stop 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2724472314 shutdown_connections 2024-05-08T23:31:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eda260 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2724472314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ed43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2724472314 >> 172.21.15.16:0/2724472314 conn(0x5558e3e32290 msgr2=0x5558e3ecee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.705+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2724472314 shutdown_connections 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2724472314 wait complete. 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 -- start start 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 Processor -- start 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 0x5558e3ee0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed9e80 0x5558e3edda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ede0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3dc05e0 con 0x5558e3ed4910 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3dc0280 con 0x5558e3ed9e80 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ca7720 con 0x5558e3eda7a0 2024-05-08T23:31:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.706+0000 7f1f7dd09b80 10 client.0 _ll_register_callbacks cb 0x5558e3dc9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 0x5558e3ee0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 0x5558e3ee0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47696/0 (socket says 172.21.15.16:47696) 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/113694719 learned_addr learned my addr 172.21.15.16:0/113694719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ede0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6affd700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed9e80 0x5558e3edda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/113694719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 msgr2=0x5558e3ede0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ede0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed9e80 msgr2=0x5558e3edda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed9e80 0x5558e3edda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 -- 172.21.15.16:0/113694719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c00eba0 con 0x5558e3ed4910 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ede0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 0x5558e3ee0730 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c01d9f0 tx=0x7f1f5c01da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52537b4f043c6ce1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f4affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f4affd700 1 -- 172.21.15.16:0/113694719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c02c070 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f4affd700 1 -- 172.21.15.16:0/113694719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f1f5c031470 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.707+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558e3ea3890 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f4affd700 1 -- 172.21.15.16:0/113694719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f5c031630 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 10 client.18078 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5558e3c51be0 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f4affd700 1 -- 172.21.15.16:0/113694719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5558e3ea3890 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f4affd700 1 -- 172.21.15.16:0/113694719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f5c026450 con 0x5558e3ed4910 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 10 client.18078 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 20 client.18078 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3eee050 0x5558e3ef0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 4 client.18078 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558e3e1cf40 con 0x5558e3eee050 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.708+0000 7f1f6affd700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3eee050 0x5558e3ef0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.709+0000 7f1f6affd700 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3eee050 0x5558e3ef0470 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b0ff9219f343ae server_cookie=ddd92ddc43428b4a in_seq=0 out_seq=0 2024-05-08T23:31:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.709+0000 7f1f4affd700 10 client.18078 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.709+0000 7f1f4affd700 1 -- 172.21.15.16:0/113694719 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558e3e1cf40 con 0x5558e3eee050 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.709+0000 7f1f4affd700 10 client.18078 handle_command_reply: tid=0 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 client.18078 shutdown 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 msgr2=0x5558e3ee0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 0x5558e3ee0730 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c01d9f0 tx=0x7f1f5c01da20 comp rx=0 tx=0).stop 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 shutdown_connections 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ede0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed4910 0x5558e3ee0730 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3eee050 0x5558e3ef0470 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/113694719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed9e80 0x5558e3edda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f4affd700 0 client.18078 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 >> 172.21.15.16:0/113694719 conn(0x5558e3e32290 msgr2=0x5558e3ece620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 shutdown_connections 2024-05-08T23:31:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/113694719 wait complete. 2024-05-08T23:31:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.710+0000 7f1f7dd09b80 20 client.18078 trim_cache size 0 max 16384 2024-05-08T23:31:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 Processor -- start 2024-05-08T23:31:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 -- start start 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ee51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ee0d10 con 0x5558e3ed9e80 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ec70c0 con 0x5558e3ed4910 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ecfb40 con 0x5558e3eda7a0 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47702/0 (socket says 172.21.15.16:47702) 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 -- 172.21.15.16:0/2107585272 learned_addr learned my addr 172.21.15.16:0/2107585272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ee51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 -- 172.21.15.16:0/2107585272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 msgr2=0x5558e3ee51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ee51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 -- 172.21.15.16:0/2107585272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 msgr2=0x5558e3ed43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6affd700 1 -- 172.21.15.16:0/2107585272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558e3ed4fc0 con 0x5558e3ed9e80 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.729+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ee51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f6affd700 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eceda0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1f540969c0 tx=0x7f1f5400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6fd1e57cbedf73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f627fc700 1 -- 172.21.15.16:0/2107585272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5409dd80 con 0x5558e3ed9e80 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f627fc700 1 -- 172.21.15.16:0/2107585272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f540a23d0 con 0x5558e3ed9e80 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f627fc700 1 -- 172.21.15.16:0/2107585272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5409a1b0 con 0x5558e3ed9e80 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2107585272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 msgr2=0x5558e3eceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eceda0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1f540969c0 tx=0x7f1f5400ced0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2107585272 shutdown_connections 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3eda7a0 0x5558e3ee51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3ed4910 0x5558e3ed43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/2107585272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3eceda0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2107585272 >> 172.21.15.16:0/2107585272 conn(0x5558e3e32290 msgr2=0x5558e3edc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.730+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2107585272 shutdown_connections 2024-05-08T23:31:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/2107585272 wait complete. 2024-05-08T23:31:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 -- start start 2024-05-08T23:31:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 Processor -- start 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3ed4910 0x5558e3f3e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3f3ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3eda7a0 0x5558e3ee2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ee2be0 con 0x5558e3ed9e80 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ee2d50 con 0x5558e3eda7a0 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558e3ee2f10 con 0x5558e3ed4910 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f7dd09b80 10 client.0 _ll_register_callbacks cb 0x5558e3dc9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.731+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3f3ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3f3ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47704/0 (socket says 172.21.15.16:47704) 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 -- 172.21.15.16:0/411045975 learned_addr learned my addr 172.21.15.16:0/411045975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6bfff700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3eda7a0 0x5558e3ee2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3ed4910 0x5558e3f3e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 -- 172.21.15.16:0/411045975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3ed4910 msgr2=0x5558e3f3e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3ed4910 0x5558e3f3e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 -- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3eda7a0 msgr2=0x5558e3ee2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3eda7a0 0x5558e3ee2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 -- 172.21.15.16:0/411045975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1a6690 con 0x5558e3ed9e80 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f6affd700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3f3ee80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c5b0 tx=0x7f1f5400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f521732afb4267 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.732+0000 7f1f4affd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.733+0000 7f1f4affd700 1 -- 172.21.15.16:0/411045975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f540983b0 con 0x5558e3ed9e80 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.733+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558e3d7d2f0 con 0x5558e3ed9e80 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.733+0000 7f1f4affd700 1 -- 172.21.15.16:0/411045975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f1f54099440 con 0x5558e3ed9e80 2024-05-08T23:31:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.733+0000 7f1f4affd700 1 -- 172.21.15.16:0/411045975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f54099600 con 0x5558e3ed9e80 2024-05-08T23:31:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.733+0000 7f1f7dd09b80 10 client.18084 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.733+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5558e3dea5e0 con 0x5558e3ed9e80 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.736+0000 7f1f4affd700 1 -- 172.21.15.16:0/411045975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5558e3d7d2f0 con 0x5558e3ed9e80 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f4affd700 1 -- 172.21.15.16:0/411045975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1f540a1fd0 con 0x5558e3ed9e80 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f7dd09b80 10 client.18084 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f7dd09b80 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f7dd09b80 20 client.18084 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3f46540 0x5558e3f48930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f7dd09b80 4 client.18084 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.737+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5558e3c72d70 con 0x5558e3f46540 2024-05-08T23:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.738+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3f46540 0x5558e3f48930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.739+0000 7f1f6b7fe700 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3f46540 0x5558e3f48930 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b9b2a7241bcf6e server_cookie=bbbf209d92f31572 in_seq=0 out_seq=0 2024-05-08T23:31:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.739+0000 7f1f4affd700 10 client.18084 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.740+0000 7f1f4affd700 1 -- 172.21.15.16:0/411045975 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5558e3c72d70 con 0x5558e3f46540 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.740+0000 7f1f4affd700 10 client.18084 handle_command_reply: tid=0 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 client.18084 shutdown 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 msgr2=0x5558e3f3ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3f3ee80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c5b0 tx=0x7f1f5400c740 comp rx=0 tx=0).stop 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 shutdown_connections 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x5558e3f46540 0x5558e3f48930 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5558e3eda7a0 0x5558e3ee2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5558e3ed4910 0x5558e3f3e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 --2- 172.21.15.16:0/411045975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558e3ed9e80 0x5558e3f3ee80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 >> 172.21.15.16:0/411045975 conn(0x5558e3e32290 msgr2=0x5558e3ed8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f4affd700 0 client.18084 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 shutdown_connections 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 1 -- 172.21.15.16:0/411045975 wait complete. 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.741+0000 7f1f7dd09b80 20 client.18084 trim_cache size 0 max 16384 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "90a6d260-db36-44ac-8fb8-644644cd5c27", 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-05-08T23:31:13.742 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.743+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f580a1310 msgr2=0x7f1f580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.743+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f580a1310 0x7f1f580a37d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c005e30 comp rx=0 tx=0).stop 2024-05-08T23:31:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.744+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 msgr2=0x7f1f6c199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.744+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c199b00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400e970 tx=0x7f1f5400ec80 comp rx=0 tx=0).stop 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 shutdown_connections 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1f580a1310 0x7f1f580a37d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6c19de50 0x7f1f6c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1f6c100050 0x7f1f6c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 --2- 172.21.15.16:0/3796016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f6c105170 0x7f1f6c199b00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 >> 172.21.15.16:0/3796016336 conn(0x7f1f6c076fa0 msgr2=0x7f1f6c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 shutdown_connections 2024-05-08T23:31:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.746+0000 7f1f71b4f700 1 -- 172.21.15.16:0/3796016336 wait complete. 2024-05-08T23:31:13.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:31:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 -- 172.21.15.16:0/2912655260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100060 msgr2=0x7f0548100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 --2- 172.21.15.16:0/2912655260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100060 0x7f0548100440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0538009990 tx=0x7f0538009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 -- 172.21.15.16:0/2912655260 shutdown_connections 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 --2- 172.21.15.16:0/2912655260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0548105160 0x7f054810cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 --2- 172.21.15.16:0/2912655260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548100060 0x7f0548100440 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 --2- 172.21.15.16:0/2912655260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100a10 0x7f0548104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 -- 172.21.15.16:0/2912655260 >> 172.21.15.16:0/2912655260 conn(0x7f0548076f40 msgr2=0x7f0548077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 -- 172.21.15.16:0/2912655260 shutdown_connections 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.903+0000 7f054da71700 1 -- 172.21.15.16:0/2912655260 wait complete. 2024-05-08T23:31:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.904+0000 7f054da71700 1 Processor -- start 2024-05-08T23:31:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.904+0000 7f054da71700 1 -- start start 2024-05-08T23:31:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f054da71700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100060 0x7f0548197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f054da71700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0548105160 0x7f0548197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f054da71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 0x7f054819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f054da71700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819c680 con 0x7f054819bc00 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f054da71700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819c7f0 con 0x7f0548100060 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f054da71700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054819c9b0 con 0x7f0548105160 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 0x7f054819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 0x7f054819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47720/0 (socket says 172.21.15.16:47720) 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 -- 172.21.15.16:0/3954115359 learned_addr learned my addr 172.21.15.16:0/3954115359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0546ffd700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0548105160 0x7f0548197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f05477fe700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100060 0x7f0548197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 -- 172.21.15.16:0/3954115359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0548105160 msgr2=0x7f0548197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0548105160 0x7f0548197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 -- 172.21.15.16:0/3954115359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100060 msgr2=0x7f0548197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100060 0x7f0548197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 -- 172.21.15.16:0/3954115359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380095f0 con 0x7f054819bc00 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f05477fe700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100060 0x7f0548197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0547fff700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 0x7f054819c080 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0540007ee0 tx=0x7f054000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.905+0000 7f0544ff9700 1 -- 172.21.15.16:0/3954115359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540017070 con 0x7f054819bc00 2024-05-08T23:31:13.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.906+0000 7f0544ff9700 1 -- 172.21.15.16:0/3954115359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f054000f040 con 0x7f054819bc00 2024-05-08T23:31:13.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.906+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481a3f10 con 0x7f054819bc00 2024-05-08T23:31:13.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.906+0000 7f0544ff9700 1 -- 172.21.15.16:0/3954115359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540005710 con 0x7f054819bc00 2024-05-08T23:31:13.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.906+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481a43a0 con 0x7f054819bc00 2024-05-08T23:31:13.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.908+0000 7f0544ff9700 1 -- 172.21.15.16:0/3954115359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0540005870 con 0x7f054819bc00 2024-05-08T23:31:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.909+0000 7f0544ff9700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f05300a1290 0x7f05300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.909+0000 7f05477fe700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f05300a1290 0x7f05300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.909+0000 7f0544ff9700 1 -- 172.21.15.16:0/3954115359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f05400cc940 con 0x7f054819bc00 2024-05-08T23:31:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.909+0000 7f05477fe700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f05300a1290 0x7f05300a3750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0538000c00 tx=0x7f0538005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 Processor -- start 2024-05-08T23:31:13.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 -- start start 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d0f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d09350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d09cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196c60150 con 0x564196d0ee00 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196c5db10 con 0x564196d0f720 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196c59400 con 0x564196d09890 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f05477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d0f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f05477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d0f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47730/0 (socket says 172.21.15.16:47730) 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f05477fe700 1 -- 172.21.15.16:0/4249836583 learned_addr learned my addr 172.21.15.16:0/4249836583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0546ffd700 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d09350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.910+0000 7f0547fff700 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d09cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f05477fe700 1 -- 172.21.15.16:0/4249836583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 msgr2=0x564196d09cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f05477fe700 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d09cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f05477fe700 1 -- 172.21.15.16:0/4249836583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 msgr2=0x564196d09350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f05477fe700 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d09350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f05477fe700 1 -- 172.21.15.16:0/4249836583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564196ada3b0 con 0x564196d0ee00 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f05477fe700 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d0f1e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0538005850 tx=0x7f053801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606c34acbe3d2a83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f052dffb700 1 -- 172.21.15.16:0/4249836583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053802a070 con 0x564196d0ee00 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f052dffb700 1 -- 172.21.15.16:0/4249836583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f053801d040 con 0x564196d0ee00 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.911+0000 7f052dffb700 1 -- 172.21.15.16:0/4249836583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538023410 con 0x564196d0ee00 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 -- 172.21.15.16:0/4249836583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 msgr2=0x564196d0f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d0f1e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0538005850 tx=0x7f053801e000 comp rx=0 tx=0).stop 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 -- 172.21.15.16:0/4249836583 shutdown_connections 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d09cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d0f1e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/4249836583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d09350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 -- 172.21.15.16:0/4249836583 >> 172.21.15.16:0/4249836583 conn(0x564196c67210 msgr2=0x564196d03df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 -- 172.21.15.16:0/4249836583 shutdown_connections 2024-05-08T23:31:13.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 -- 172.21.15.16:0/4249836583 wait complete. 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.912+0000 7f0559c2bb80 1 -- start start 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 Processor -- start 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d12bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d13130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 0x564196d13670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196bf5560 con 0x564196d0f720 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196bf5200 con 0x564196d0ee00 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196adc6a0 con 0x564196d09890 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0559c2bb80 10 client.0 _ll_register_callbacks cb 0x564196bfeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 0x564196d13670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0547fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 0x564196d13670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47744/0 (socket says 172.21.15.16:47744) 2024-05-08T23:31:13.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0547fff700 1 -- 172.21.15.16:0/1039959582 learned_addr learned my addr 172.21.15.16:0/1039959582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f05477fe700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d12bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0546ffd700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d13130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0547fff700 1 -- 172.21.15.16:0/1039959582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 msgr2=0x564196d12bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0547fff700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d12bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.913+0000 7f0547fff700 1 -- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 msgr2=0x564196d13130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0547fff700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d13130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0547fff700 1 -- 172.21.15.16:0/1039959582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054810d550 con 0x564196d0f720 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0546ffd700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d13130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0547fff700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 0x564196d13670 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0540009510 tx=0x7f0540004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba9fcb53b76baa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0526ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0526ffd700 1 -- 172.21.15.16:0/1039959582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400b90d0 con 0x564196d0f720 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0526ffd700 1 -- 172.21.15.16:0/1039959582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f0540099650 con 0x564196d0f720 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564196bb2270 con 0x564196d0f720 2024-05-08T23:31:13.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0526ffd700 1 -- 172.21.15.16:0/1039959582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f054009d480 con 0x564196d0f720 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 10 client.18096 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564196a86b60 con 0x564196d0f720 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0526ffd700 1 -- 172.21.15.16:0/1039959582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564196bb2270 con 0x564196d0f720 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0526ffd700 1 -- 172.21.15.16:0/1039959582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05400998b0 con 0x564196d0f720 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 10 client.18096 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 20 client.18096 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d23210 0x564196d25630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.914+0000 7f0559c2bb80 4 client.18096 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.915+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564196cd8810 con 0x564196d23210 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.915+0000 7f05477fe700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d23210 0x564196d25630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.915+0000 7f0526ffd700 10 client.18096 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.915+0000 7f05477fe700 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d23210 0x564196d25630 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9d041c527f7f72 server_cookie=309a4292fca92ca5 in_seq=0 out_seq=0 2024-05-08T23:31:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0526ffd700 1 -- 172.21.15.16:0/1039959582 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564196cd8810 con 0x564196d23210 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0526ffd700 10 client.18096 handle_command_reply: tid=0 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 client.18096 shutdown 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 msgr2=0x564196d13670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 0x564196d13670 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0540009510 tx=0x7f0540004b50 comp rx=0 tx=0).stop 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 shutdown_connections 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0f720 0x564196d13670 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d12bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d23210 0x564196d25630 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1039959582 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d13130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 >> 172.21.15.16:0/1039959582 conn(0x564196c67210 msgr2=0x564196d035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0526ffd700 0 client.18096 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 shutdown_connections 2024-05-08T23:31:13.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.916+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1039959582 wait complete. 2024-05-08T23:31:13.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.917+0000 7f0559c2bb80 20 client.18096 trim_cache size 0 max 16384 2024-05-08T23:31:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.934+0000 7f0559c2bb80 1 Processor -- start 2024-05-08T23:31:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.934+0000 7f0559c2bb80 1 -- start start 2024-05-08T23:31:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 0x564196d03d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d0f720 0x564196d0e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196cfc110 con 0x564196d09890 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196d141a0 con 0x564196d0ee00 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196d04ca0 con 0x564196d0f720 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 0x564196d03d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 0x564196d03d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47760/0 (socket says 172.21.15.16:47760) 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 -- 172.21.15.16:0/1584166099 learned_addr learned my addr 172.21.15.16:0/1584166099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0547fff700 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d0f720 0x564196d0e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 -- 172.21.15.16:0/1584166099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d0f720 msgr2=0x564196d0e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d0f720 0x564196d0e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 -- 172.21.15.16:0/1584166099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 msgr2=0x564196d0dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0546ffd700 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d0dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 -- 172.21.15.16:0/1584166099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564196c1f560 con 0x564196d09890 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f0546ffd700 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d0dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f05477fe700 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 0x564196d03d20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0538009990 tx=0x7f0538009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51fe383aca81dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.935+0000 7f052e7fc700 1 -- 172.21.15.16:0/1584166099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053802a040 con 0x564196d09890 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f052e7fc700 1 -- 172.21.15.16:0/1584166099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f053800bd80 con 0x564196d09890 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f052e7fc700 1 -- 172.21.15.16:0/1584166099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053803b160 con 0x564196d09890 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1584166099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 msgr2=0x564196d03d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 0x564196d03d20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0538009990 tx=0x7f0538009310 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1584166099 shutdown_connections 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d0f720 0x564196d0e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d09890 0x564196d03d20 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/1584166099 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0ee00 0x564196d0dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1584166099 >> 172.21.15.16:0/1584166099 conn(0x564196c67210 msgr2=0x564196d0cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1584166099 shutdown_connections 2024-05-08T23:31:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 -- 172.21.15.16:0/1584166099 wait complete. 2024-05-08T23:31:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.936+0000 7f0559c2bb80 1 -- start start 2024-05-08T23:31:13.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 Processor -- start 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d18600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d18b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196d7cb20 con 0x564196d0ee00 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196d7cc60 con 0x564196d0f720 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564196d7cda0 con 0x564196d09890 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0559c2bb80 10 client.0 _ll_register_callbacks cb 0x564196bfeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d18600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0546ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d18600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47766/0 (socket says 172.21.15.16:47766) 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0546ffd700 1 -- 172.21.15.16:0/764733219 learned_addr learned my addr 172.21.15.16:0/764733219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f05477fe700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.937+0000 7f0547fff700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d18b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0546ffd700 1 -- 172.21.15.16:0/764733219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 msgr2=0x564196d180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0546ffd700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0546ffd700 1 -- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 msgr2=0x564196d18b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0546ffd700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d18b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0546ffd700 1 -- 172.21.15.16:0/764733219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564196ada3b0 con 0x564196d0ee00 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0547fff700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d18b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0546ffd700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d18600 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f054806a6d0 tx=0x7f053c003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c99606400cc275 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0526ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0526ffd700 1 -- 172.21.15.16:0/764733219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c017070 con 0x564196d0ee00 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0526ffd700 1 -- 172.21.15.16:0/764733219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f053c00da50 con 0x564196d0ee00 2024-05-08T23:31:13.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f053801d040 con 0x564196d0ee00 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0526ffd700 1 -- 172.21.15.16:0/764733219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c00dc10 con 0x564196d0ee00 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0559c2bb80 10 client.18102 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564196adecd0 con 0x564196d0ee00 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.938+0000 7f0526ffd700 1 -- 172.21.15.16:0/764733219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f053801d040 con 0x564196d0ee00 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0526ffd700 1 -- 172.21.15.16:0/764733219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f053c01c420 con 0x564196d0ee00 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0559c2bb80 10 client.18102 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0559c2bb80 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0559c2bb80 20 client.18102 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d7e5d0 0x564196d809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0559c2bb80 4 client.18102 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564196c51ec0 con 0x564196d7e5d0 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f05477fe700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d7e5d0 0x564196d809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f0526ffd700 10 client.18102 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.939+0000 7f05477fe700 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d7e5d0 0x564196d809f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd91f4a4fc63f656 server_cookie=94034f8b7aabc431 in_seq=0 out_seq=0 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0526ffd700 1 -- 172.21.15.16:0/764733219 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564196c51ec0 con 0x564196d7e5d0 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0526ffd700 10 client.18102 handle_command_reply: tid=0 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 client.18102 shutdown 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 msgr2=0x564196d18600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d18600 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f054806a6d0 tx=0x7f053c003be0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 shutdown_connections 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x564196d7e5d0 0x564196d809f0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564196d0f720 0x564196d18b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0526ffd700 0 client.18102 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564196d09890 0x564196d180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 --2- 172.21.15.16:0/764733219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564196d0ee00 0x564196d18600 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 >> 172.21.15.16:0/764733219 conn(0x564196c67210 msgr2=0x564196d0dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 shutdown_connections 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 1 -- 172.21.15.16:0/764733219 wait complete. 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f0559c2bb80 20 client.18102 trim_cache size 0 max 16384 2024-05-08T23:31:13.947 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: "90a6d260-db36-44ac-8fb8-644644cd5c27": { 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "90a6d260-db36-44ac-8fb8-644644cd5c27", 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:13.948 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f05300a1290 msgr2=0x7f05300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f054da71700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f05300a1290 0x7f05300a3750 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0538000c00 tx=0x7f0538005fb0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 msgr2=0x7f054819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.940+0000 7f054da71700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 0x7f054819c080 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0540007ee0 tx=0x7f054000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 shutdown_connections 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f05300a1290 0x7f05300a3750 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054819bc00 0x7f054819c080 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0548100060 0x7f0548197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 --2- 172.21.15.16:0/3954115359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0548105160 0x7f0548197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 >> 172.21.15.16:0/3954115359 conn(0x7f0548076f40 msgr2=0x7f0548068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 shutdown_connections 2024-05-08T23:31:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:13.941+0000 7f054da71700 1 -- 172.21.15.16:0/3954115359 wait complete. 2024-05-08T23:31:13.954 INFO:tasks.cephfs.filesystem:scrub status for tag:90a6d260-db36-44ac-8fb8-644644cd5c27 - {'path': '/', 'tag': '90a6d260-db36-44ac-8fb8-644644cd5c27', 'options': 'recursive,force'} 2024-05-08T23:31:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:13 smithi016 ceph-mon[42541]: pgmap v363: 129 pgs: 129 active+clean; 22 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 32 op/s 2024-05-08T23:31:14.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:13 smithi016 ceph-mon[42541]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:31:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:13 smithi189 ceph-mon[46919]: pgmap v363: 129 pgs: 129 active+clean; 22 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 32 op/s 2024-05-08T23:31:14.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:13 smithi189 ceph-mon[46919]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:31:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:13 smithi155 ceph-mon[47105]: pgmap v363: 129 pgs: 129 active+clean; 22 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 121 KiB/s wr, 32 op/s 2024-05-08T23:31:14.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:13 smithi155 ceph-mon[47105]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T23:31:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:14 smithi016 ceph-mon[42541]: scrub queued for path: / 2024-05-08T23:31:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:14 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:31:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:14 smithi016 ceph-mon[42541]: scrub summary: active paths [/] 2024-05-08T23:31:15.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:14 smithi016 ceph-mon[42541]: scrub summary: idle+waiting paths [/] 2024-05-08T23:31:15.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:14 smithi189 ceph-mon[46919]: scrub queued for path: / 2024-05-08T23:31:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:14 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:31:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:14 smithi189 ceph-mon[46919]: scrub summary: active paths [/] 2024-05-08T23:31:15.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:14 smithi189 ceph-mon[46919]: scrub summary: idle+waiting paths [/] 2024-05-08T23:31:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:14 smithi155 ceph-mon[47105]: scrub queued for path: / 2024-05-08T23:31:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:14 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:31:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:14 smithi155 ceph-mon[47105]: scrub summary: active paths [/] 2024-05-08T23:31:15.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:14 smithi155 ceph-mon[47105]: scrub summary: idle+waiting paths [/] 2024-05-08T23:31:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:15 smithi016 ceph-mon[42541]: pgmap v365: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 80 KiB/s wr, 15 op/s 2024-05-08T23:31:16.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:15 smithi016 ceph-mon[42541]: scrub summary: idle 2024-05-08T23:31:16.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:15 smithi189 ceph-mon[46919]: pgmap v365: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 80 KiB/s wr, 15 op/s 2024-05-08T23:31:16.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:15 smithi189 ceph-mon[46919]: scrub summary: idle 2024-05-08T23:31:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:15 smithi155 ceph-mon[47105]: pgmap v365: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 80 KiB/s wr, 15 op/s 2024-05-08T23:31:16.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:15 smithi155 ceph-mon[47105]: scrub summary: idle 2024-05-08T23:31:18.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:17 smithi016 ceph-mon[42541]: pgmap v366: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-05-08T23:31:18.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:17 smithi189 ceph-mon[46919]: pgmap v366: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-05-08T23:31:18.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:17 smithi155 ceph-mon[47105]: pgmap v366: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-05-08T23:31:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:19 smithi016 ceph-mon[42541]: pgmap v367: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-05-08T23:31:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:19 smithi189 ceph-mon[46919]: pgmap v367: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-05-08T23:31:20.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:19 smithi155 ceph-mon[47105]: pgmap v367: 129 pgs: 129 active+clean; 22 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 67 KiB/s wr, 12 op/s 2024-05-08T23:31:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:21 smithi016 ceph-mon[42541]: pgmap v368: 129 pgs: 129 active+clean; 23 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-05-08T23:31:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:21 smithi189 ceph-mon[46919]: pgmap v368: 129 pgs: 129 active+clean; 23 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-05-08T23:31:22.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:21 smithi155 ceph-mon[47105]: pgmap v368: 129 pgs: 129 active+clean; 23 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-05-08T23:31:24.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:23 smithi016 ceph-mon[42541]: pgmap v369: 129 pgs: 129 active+clean; 23 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-05-08T23:31:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:23 smithi189 ceph-mon[46919]: pgmap v369: 129 pgs: 129 active+clean; 23 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-05-08T23:31:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:23 smithi155 ceph-mon[47105]: pgmap v369: 129 pgs: 129 active+clean; 23 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 12 op/s 2024-05-08T23:31:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:25 smithi016 ceph-mon[42541]: pgmap v370: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 88 KiB/s wr, 17 op/s 2024-05-08T23:31:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:25 smithi189 ceph-mon[46919]: pgmap v370: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 88 KiB/s wr, 17 op/s 2024-05-08T23:31:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:25 smithi155 ceph-mon[47105]: pgmap v370: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 88 KiB/s wr, 17 op/s 2024-05-08T23:31:28.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:27 smithi016 ceph-mon[42541]: pgmap v371: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-08T23:31:28.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:27 smithi189 ceph-mon[46919]: pgmap v371: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-08T23:31:28.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:27 smithi155 ceph-mon[47105]: pgmap v371: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-08T23:31:30.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:29 smithi189 ceph-mon[46919]: pgmap v372: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-08T23:31:30.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:29 smithi155 ceph-mon[47105]: pgmap v372: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-08T23:31:30.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:29 smithi016 ceph-mon[42541]: pgmap v372: 129 pgs: 129 active+clean; 23 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-08T23:31:32.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:31 smithi189 ceph-mon[46919]: pgmap v373: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-05-08T23:31:32.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:31 smithi155 ceph-mon[47105]: pgmap v373: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-05-08T23:31:32.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:31 smithi016 ceph-mon[42541]: pgmap v373: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-05-08T23:31:34.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:33 smithi189 ceph-mon[46919]: pgmap v374: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:34.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:33 smithi155 ceph-mon[47105]: pgmap v374: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:34.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:33 smithi016 ceph-mon[42541]: pgmap v374: 129 pgs: 129 active+clean; 23 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:36.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:35 smithi189 ceph-mon[46919]: pgmap v375: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 82 KiB/s wr, 15 op/s 2024-05-08T23:31:36.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:35 smithi155 ceph-mon[47105]: pgmap v375: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 82 KiB/s wr, 15 op/s 2024-05-08T23:31:36.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:35 smithi016 ceph-mon[42541]: pgmap v375: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 82 KiB/s wr, 15 op/s 2024-05-08T23:31:38.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:37 smithi189 ceph-mon[46919]: pgmap v376: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:38.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:37 smithi155 ceph-mon[47105]: pgmap v376: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:38.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:37 smithi016 ceph-mon[42541]: pgmap v376: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:40.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:39 smithi189 ceph-mon[46919]: pgmap v377: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:40.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:39 smithi155 ceph-mon[47105]: pgmap v377: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:40.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:39 smithi016 ceph-mon[42541]: pgmap v377: 129 pgs: 129 active+clean; 24 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:42.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:41 smithi189 ceph-mon[46919]: pgmap v378: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-05-08T23:31:42.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:41 smithi155 ceph-mon[47105]: pgmap v378: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:561/600: open fd = 3 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:562/600: open fd = 3 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.264 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:563/600: open fd = 3 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:564/600: open fd = 3 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:565/600: open fd = 3 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.265 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:566/600: open fd = 3 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:567/600: open fd = 3 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:568/600: open fd = 3 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.266 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:569/600: open fd = 3 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:570/600: open fd = 3 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:571/600: open fd = 3 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.267 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:572/600: open fd = 3 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:573/600: open fd = 3 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:574/600: open fd = 3 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.268 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:575/600: open fd = 3 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:576/600: open fd = 3 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:577/600: open fd = 3 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.269 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:578/600: open fd = 3 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:579/600: open fd = 3 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.270 INFO:tasks.workunit.client.0.smithi016.stdout:580/600: open fd = 3 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:581/600: open fd = 3 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:582/600: open fd = 3 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.271 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:583/600: open fd = 3 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:584/600: open fd = 3 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:585/600: open fd = 3 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:586/600: open fd = 3 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.272 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:587/600: open fd = 3 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:588/600: open fd = 3 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:589/600: open fd = 3 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.273 INFO:tasks.workunit.client.0.smithi016.stdout:590/600: open fd = 3 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:591/600: open fd = 3 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:592/600: open fd = 3 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:593/600: open fd = 3 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.274 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:594/600: open fd = 3 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:595/600: open fd = 3 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:596/600: open fd = 3 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.275 INFO:tasks.workunit.client.0.smithi016.stdout:597/600: open fd = 3 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:598/600: open fd = 3 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:599/600: open fd = 3 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:600/600: open fd = 3 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.276 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-05-08T23:31:42.277 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-05-08T23:31:42.277 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-05-08T23:31:42.278 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-05-08T23:31:42.278 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T23:31:42.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T23:31:42.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:41 smithi016 ceph-mon[42541]: pgmap v378: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-05-08T23:31:42.332 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-05-08T23:31:42.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T23:31:42.690 DEBUG:teuthology.parallel:result is None 2024-05-08T23:31:42.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T23:31:42.731 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T23:31:42.731 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-08T23:31:42.742 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-08T23:31:42.743 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T23:31:42.743 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:31:42.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-08T23:31:42.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-08T23:31:42.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T23:31:43.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 -- 172.21.15.16:0/1233821213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 msgr2=0x7f04401073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 --2- 172.21.15.16:0/1233821213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f04401073e0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0430009990 tx=0x7f0430009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 -- 172.21.15.16:0/1233821213 shutdown_connections 2024-05-08T23:31:43.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 --2- 172.21.15.16:0/1233821213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0440068ca0 0x7f044010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 --2- 172.21.15.16:0/1233821213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f04401073e0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 --2- 172.21.15.16:0/1233821213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04400682e0 0x7f0440068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 -- 172.21.15.16:0/1233821213 >> 172.21.15.16:0/1233821213 conn(0x7f0440076f10 msgr2=0x7f0440077320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 -- 172.21.15.16:0/1233821213 shutdown_connections 2024-05-08T23:31:43.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.072+0000 7f0444c44700 1 -- 172.21.15.16:0/1233821213 wait complete. 2024-05-08T23:31:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 Processor -- start 2024-05-08T23:31:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 -- start start 2024-05-08T23:31:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04400682e0 0x7f04401993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0440068ca0 0x7f0440199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f044019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044019e2e0 con 0x7f0440107000 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044019e450 con 0x7f04400682e0 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.073+0000 7f0444c44700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044019e610 con 0x7f0440068ca0 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f044019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f044019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48760/0 (socket says 172.21.15.16:48760) 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 -- 172.21.15.16:0/1960901160 learned_addr learned my addr 172.21.15.16:0/1960901160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043dd9b700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0440068ca0 0x7f0440199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043e59c700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04400682e0 0x7f04401993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 -- 172.21.15.16:0/1960901160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0440068ca0 msgr2=0x7f0440199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0440068ca0 0x7f0440199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 -- 172.21.15.16:0/1960901160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04400682e0 msgr2=0x7f04401993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04400682e0 0x7f04401993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 -- 172.21.15.16:0/1960901160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300095f0 con 0x7f0440107000 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f043ed9d700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f044019dca0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f043800ef30 tx=0x7f043800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438018070 con 0x7f0440107000 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043800f040 con 0x7f0440107000 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044019e890 con 0x7f0440107000 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800cc70 con 0x7f0440107000 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.074+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04401a6130 con 0x7f0440107000 2024-05-08T23:31:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.075+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04401025a0 con 0x7f0440107000 2024-05-08T23:31:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.077+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0438007500 con 0x7f0440107000 2024-05-08T23:31:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.077+0000 7f042f7fe700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04280a1310 0x7f04280a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.077+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f04380ccb30 con 0x7f0440107000 2024-05-08T23:31:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.077+0000 7f043e59c700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04280a1310 0x7f04280a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.078+0000 7f043e59c700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04280a1310 0x7f04280a37d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04300059d0 tx=0x7f0430005940 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.082+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0438095c70 con 0x7f0440107000 2024-05-08T23:31:43.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.326+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0440110c50 con 0x7f04280a1310 2024-05-08T23:31:43.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.334+0000 7f042f7fe700 1 -- 172.21.15.16:0/1960901160 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f0440110c50 con 0x7f04280a1310 2024-05-08T23:31:43.335 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T23:19:48", "first": "2024-05-08T23:20:00", "last": "2024-05-08T23:31:00", "last_pruned": "2024-05-08T23:31:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-05-08T23:31:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04280a1310 msgr2=0x7f04280a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04280a1310 0x7f04280a37d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04300059d0 tx=0x7f0430005940 comp rx=0 tx=0).stop 2024-05-08T23:31:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 msgr2=0x7f044019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f044019dca0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f043800ef30 tx=0x7f043800c3a0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 shutdown_connections 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04280a1310 0x7f04280a37d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0440107000 0x7f044019dca0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04400682e0 0x7f04401993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 --2- 172.21.15.16:0/1960901160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0440068ca0 0x7f0440199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 >> 172.21.15.16:0/1960901160 conn(0x7f0440076f10 msgr2=0x7f044010c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 shutdown_connections 2024-05-08T23:31:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.336+0000 7f0444c44700 1 -- 172.21.15.16:0/1960901160 wait complete. 2024-05-08T23:31:43.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T23:31:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 -- 172.21.15.16:0/2182239193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 msgr2=0x7f6d841014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/2182239193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d841014d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009990 tx=0x7f6d74009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 -- 172.21.15.16:0/2182239193 shutdown_connections 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/2182239193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d84106210 0x7f6d8410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/2182239193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d841014d0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/2182239193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d84101aa0 0x7f6d84105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 -- 172.21.15.16:0/2182239193 >> 172.21.15.16:0/2182239193 conn(0x7f6d840fc970 msgr2=0x7f6d840fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 -- 172.21.15.16:0/2182239193 shutdown_connections 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.489+0000 7f6d89ecf700 1 -- 172.21.15.16:0/2182239193 wait complete. 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 Processor -- start 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 -- start start 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d84197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d84106210 0x7f6d841977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d8419bb40 0x7f6d8419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8419c5c0 con 0x7f6d841010f0 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8419c730 con 0x7f6d84106210 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.490+0000 7f6d89ecf700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8419c8f0 con 0x7f6d8419bb40 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d84197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d84197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48768/0 (socket says 172.21.15.16:48768) 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 -- 172.21.15.16:0/428024570 learned_addr learned my addr 172.21.15.16:0/428024570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d82ffd700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d84106210 0x7f6d841977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d83fff700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d8419bb40 0x7f6d8419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 -- 172.21.15.16:0/428024570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d8419bb40 msgr2=0x7f6d8419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d8419bb40 0x7f6d8419bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 -- 172.21.15.16:0/428024570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d84106210 msgr2=0x7f6d841977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d84106210 0x7f6d841977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 -- 172.21.15.16:0/428024570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740095f0 con 0x7f6d841010f0 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d83fff700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d8419bb40 0x7f6d8419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d837fe700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d84197260 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f6d740036c0 tx=0x7f6d740057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7401d070 con 0x7f6d841010f0 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d7400fcc0 con 0x7f6d841010f0 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d841a3df0 con 0x7f6d841010f0 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74022b40 con 0x7f6d841010f0 2024-05-08T23:31:43.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.491+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d841a4210 con 0x7f6d841010f0 2024-05-08T23:31:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.492+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d84109480 con 0x7f6d841010f0 2024-05-08T23:31:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.494+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6d7400f830 con 0x7f6d841010f0 2024-05-08T23:31:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.494+0000 7f6d80ff9700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d6c0a1290 0x7f6d6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.494+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f6d740cece0 con 0x7f6d841010f0 2024-05-08T23:31:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.494+0000 7f6d82ffd700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d6c0a1290 0x7f6d6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:43.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.495+0000 7f6d82ffd700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d6c0a1290 0x7f6d6c0a3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6d841987b0 tx=0x7f6d7800a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.498+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6d74097e20 con 0x7f6d841010f0 2024-05-08T23:31:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.747+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 --> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6d84198560 con 0x7f6d6c0a1290 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.754+0000 7f6d80ff9700 1 -- 172.21.15.16:0/428024570 <== mgr.25310 v2:172.21.15.16:6800/198235485 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6d84198560 con 0x7f6d6c0a1290 2024-05-08T23:31:43.755 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d6c0a1290 msgr2=0x7f6d6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d6c0a1290 0x7f6d6c0a3750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6d841987b0 tx=0x7f6d7800a380 comp rx=0 tx=0).stop 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 msgr2=0x7f6d84197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d84197260 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f6d740036c0 tx=0x7f6d740057c0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 shutdown_connections 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f6d6c0a1290 0x7f6d6c0a3750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d8419bb40 0x7f6d8419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d841010f0 0x7f6d84197260 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 --2- 172.21.15.16:0/428024570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d84106210 0x7f6d841977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 >> 172.21.15.16:0/428024570 conn(0x7f6d840fc970 msgr2=0x7f6d840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 shutdown_connections 2024-05-08T23:31:43.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:43.757+0000 7f6d89ecf700 1 -- 172.21.15.16:0/428024570 wait complete. 2024-05-08T23:31:43.771 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-08T23:31:43.781 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-08T23:31:43.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T23:31:44.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 -- 172.21.15.16:0/267962473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 msgr2=0x7fb558105a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/267962473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558105a20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fb5400099c0 tx=0x7fb540009cd0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 -- 172.21.15.16:0/267962473 shutdown_connections 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/267962473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580fff40 0x7fb55810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/267962473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558105a20 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/267962473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5580ff580 0x7fb5580ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 -- 172.21.15.16:0/267962473 >> 172.21.15.16:0/267962473 conn(0x7fb5580763a0 msgr2=0x7fb5580fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 -- 172.21.15.16:0/267962473 shutdown_connections 2024-05-08T23:31:44.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.104+0000 7fb55cdb7700 1 -- 172.21.15.16:0/267962473 wait complete. 2024-05-08T23:31:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 Processor -- start 2024-05-08T23:31:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 -- start start 2024-05-08T23:31:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580ff580 0x7fb558101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5580fff40 0x7fb558101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581a5640 con 0x7fb558105640 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581a57b0 con 0x7fb5580ff580 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55cdb7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581a5970 con 0x7fb5580fff40 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48784/0 (socket says 172.21.15.16:48784) 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 -- 172.21.15.16:0/4180925173 learned_addr learned my addr 172.21.15.16:0/4180925173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb55659c700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580ff580 0x7fb558101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb555d9b700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5580fff40 0x7fb558101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 -- 172.21.15.16:0/4180925173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5580fff40 msgr2=0x7fb558101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5580fff40 0x7fb558101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 -- 172.21.15.16:0/4180925173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580ff580 msgr2=0x7fb558101560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580ff580 0x7fb558101560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.105+0000 7fb556d9d700 1 -- 172.21.15.16:0/4180925173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400095f0 con 0x7fb558105640 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb55659c700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580ff580 0x7fb558101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb556d9d700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558101fe0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb55000ef30 tx=0x7fb55000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb54f7fe700 1 -- 172.21.15.16:0/4180925173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550018070 con 0x7fb558105640 2024-05-08T23:31:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb54f7fe700 1 -- 172.21.15.16:0/4180925173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55000f040 con 0x7fb558105640 2024-05-08T23:31:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5581a5bf0 con 0x7fb558105640 2024-05-08T23:31:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb54f7fe700 1 -- 172.21.15.16:0/4180925173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb550003d00 con 0x7fb558105640 2024-05-08T23:31:44.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.106+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5581a60b0 con 0x7fb558105640 2024-05-08T23:31:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.108+0000 7fb54f7fe700 1 -- 172.21.15.16:0/4180925173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb55000caa0 con 0x7fb558105640 2024-05-08T23:31:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.109+0000 7fb54f7fe700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5440a1290 0x7fb5440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.109+0000 7fb54f7fe700 1 -- 172.21.15.16:0/4180925173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb5500cce20 con 0x7fb558105640 2024-05-08T23:31:44.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.109+0000 7fb55659c700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5440a1290 0x7fb5440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb55659c700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5440a1290 0x7fb5440a3750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb540006010 tx=0x7fb540005ee0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 Processor -- start 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 -- start start 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18cffe00 0x559b18d001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 0x559b18cfa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cfa890 0x559b18cfacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18c51150 con 0x559b18d00720 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18c4eb10 con 0x559b18cfa890 2024-05-08T23:31:44.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb568f71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18c4a400 con 0x559b18cffe00 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb555d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 0x559b18cfa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb555d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 0x559b18cfa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48786/0 (socket says 172.21.15.16:48786) 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.110+0000 7fb555d9b700 1 -- 172.21.15.16:0/981238421 learned_addr learned my addr 172.21.15.16:0/981238421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb556d9d700 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cfa890 0x559b18cfacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb55659c700 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18cffe00 0x559b18d001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb555d9b700 1 -- 172.21.15.16:0/981238421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18cffe00 msgr2=0x559b18d001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb555d9b700 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18cffe00 0x559b18d001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb555d9b700 1 -- 172.21.15.16:0/981238421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cfa890 msgr2=0x559b18cfacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb555d9b700 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cfa890 0x559b18cfacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb555d9b700 1 -- 172.21.15.16:0/981238421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b18acb3b0 con 0x559b18d00720 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb555d9b700 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 0x559b18cfa350 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb54800bdb0 tx=0x7fb54800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61c5362c2deddac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb54cff9700 1 -- 172.21.15.16:0/981238421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54800d880 con 0x559b18d00720 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb54cff9700 1 -- 172.21.15.16:0/981238421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb54800de70 con 0x559b18d00720 2024-05-08T23:31:44.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.111+0000 7fb54cff9700 1 -- 172.21.15.16:0/981238421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548017e30 con 0x559b18d00720 2024-05-08T23:31:44.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 -- 172.21.15.16:0/981238421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 msgr2=0x559b18cfa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 0x559b18cfa350 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb54800bdb0 tx=0x7fb54800f740 comp rx=0 tx=0).stop 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 -- 172.21.15.16:0/981238421 shutdown_connections 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cfa890 0x559b18cfacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18cffe00 0x559b18d001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 --2- 172.21.15.16:0/981238421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18d00720 0x559b18cfa350 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 -- 172.21.15.16:0/981238421 >> 172.21.15.16:0/981238421 conn(0x559b18c58210 msgr2=0x559b18cf4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 -- 172.21.15.16:0/981238421 shutdown_connections 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 -- 172.21.15.16:0/981238421 wait complete. 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.112+0000 7fb568f71b80 1 -- start start 2024-05-08T23:31:44.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 Processor -- start 2024-05-08T23:31:44.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:43 smithi016 ceph-mon[42541]: pgmap v379: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d03fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18be6560 con 0x559b18cfa890 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18be6200 con 0x559b18cffe00 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18acd6a0 con 0x559b18d00720 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb568f71b80 10 client.0 _ll_register_callbacks cb 0x559b18befb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb55659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d03a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb55659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d03a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48788/0 (socket says 172.21.15.16:48788) 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb55659c700 1 -- 172.21.15.16:0/4021968365 learned_addr learned my addr 172.21.15.16:0/4021968365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb555d9b700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d03fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.113+0000 7fb556d9d700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb55659c700 1 -- 172.21.15.16:0/4021968365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 msgr2=0x559b18d044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb55659c700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb55659c700 1 -- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 msgr2=0x559b18d03fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb55659c700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d03fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb55659c700 1 -- 172.21.15.16:0/4021968365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54800b9e0 con 0x559b18cfa890 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb55659c700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d03a70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb5400098e0 tx=0x7fb540019050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a1014a72353acf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb53dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb53dffb700 1 -- 172.21.15.16:0/4021968365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54002a050 con 0x559b18cfa890 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb53dffb700 1 -- 172.21.15.16:0/4021968365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb540003c50 con 0x559b18cfa890 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b18ba3270 con 0x559b18cfa890 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb53dffb700 1 -- 172.21.15.16:0/4021968365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb540003e10 con 0x559b18cfa890 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb568f71b80 10 client.18126 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559b18a77b60 con 0x559b18cfa890 2024-05-08T23:31:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.114+0000 7fb53dffb700 1 -- 172.21.15.16:0/4021968365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559b18ba3270 con 0x559b18cfa890 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb53dffb700 1 -- 172.21.15.16:0/4021968365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb540023410 con 0x559b18cfa890 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb568f71b80 10 client.18126 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb568f71b80 10 client.18126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb568f71b80 20 client.18126 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb568f71b80 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d142b0 0x559b18d166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb568f71b80 4 client.18126 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b18cc9810 con 0x559b18d142b0 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb555d9b700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d142b0 0x559b18d166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb53dffb700 10 client.18126 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.115+0000 7fb555d9b700 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d142b0 0x559b18d166d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcce49fc238025c server_cookie=9f526479373da697 in_seq=0 out_seq=0 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb53dffb700 1 -- 172.21.15.16:0/4021968365 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b18cc9810 con 0x559b18d142b0 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb53dffb700 10 client.18126 handle_command_reply: tid=0 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb568f71b80 1 client.18126 shutdown 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 msgr2=0x559b18d03a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb568f71b80 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d03a70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb5400098e0 tx=0x7fb540019050 comp rx=0 tx=0).stop 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 shutdown_connections 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.116+0000 7fb568f71b80 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d03a70 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d142b0 0x559b18d166d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 1 --2- 172.21.15.16:0/4021968365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d03fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 >> 172.21.15.16:0/4021968365 conn(0x559b18c58210 msgr2=0x559b18cf45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb53dffb700 0 client.18126 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 shutdown_connections 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 1 -- 172.21.15.16:0/4021968365 wait complete. 2024-05-08T23:31:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.117+0000 7fb568f71b80 20 client.18126 trim_cache size 0 max 16384 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 Processor -- start 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 -- start start 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18cf4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d0b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d0b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18d1b8e0 con 0x559b18cfa890 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18d01fc0 con 0x559b18cffe00 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb568f71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18cfa350 con 0x559b18d00720 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.135+0000 7fb55659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18cf4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18cf4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48792/0 (socket says 172.21.15.16:48792) 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 -- 172.21.15.16:0/2538083782 learned_addr learned my addr 172.21.15.16:0/2538083782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 -- 172.21.15.16:0/2538083782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 msgr2=0x559b18d0b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d0b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 -- 172.21.15.16:0/2538083782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 msgr2=0x559b18d0b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d0b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 -- 172.21.15.16:0/2538083782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b18c10560 con 0x559b18cfa890 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb55659c700 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18cf4d20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb540023770 tx=0x7fb540005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd21b515eb6c7025 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb54d7fa700 1 -- 172.21.15.16:0/2538083782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54002a040 con 0x559b18cfa890 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb54d7fa700 1 -- 172.21.15.16:0/2538083782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb54002b040 con 0x559b18cfa890 2024-05-08T23:31:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.136+0000 7fb54d7fa700 1 -- 172.21.15.16:0/2538083782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000f6c0 con 0x559b18cfa890 2024-05-08T23:31:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 -- 172.21.15.16:0/2538083782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 msgr2=0x559b18cf4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18cf4d20 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fb540023770 tx=0x7fb540005d30 comp rx=0 tx=0).stop 2024-05-08T23:31:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 -- 172.21.15.16:0/2538083782 shutdown_connections 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18d0b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18cf4d20 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 --2- 172.21.15.16:0/2538083782 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18d0b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 -- 172.21.15.16:0/2538083782 >> 172.21.15.16:0/2538083782 conn(0x559b18c58210 msgr2=0x559b18d02340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 -- 172.21.15.16:0/2538083782 shutdown_connections 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 -- 172.21.15.16:0/2538083782 wait complete. 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 -- start start 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.137+0000 7fb568f71b80 1 Processor -- start 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d0b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18cfbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18cfc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18d0b9f0 con 0x559b18cfa890 2024-05-08T23:31:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18cfc830 con 0x559b18cffe00 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18cfc9f0 con 0x559b18d00720 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb568f71b80 10 client.0 _ll_register_callbacks cb 0x559b18befb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d0b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d0b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48800/0 (socket says 172.21.15.16:48800) 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 -- 172.21.15.16:0/692491634 learned_addr learned my addr 172.21.15.16:0/692491634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb555d9b700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18cfbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb556d9d700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18cfc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 -- 172.21.15.16:0/692491634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 msgr2=0x559b18cfc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18cfc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 -- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 msgr2=0x559b18cfbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18cfbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 -- 172.21.15.16:0/692491634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b18a77b60 con 0x559b18cfa890 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb556d9d700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18cfc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb55659c700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d0b1a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb5400098e0 tx=0x7fb540023b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea0d652482cebaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.138+0000 7fb53dffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb53dffb700 1 -- 172.21.15.16:0/692491634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540003e60 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb53dffb700 1 -- 172.21.15.16:0/692491634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb54000f460 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb54800de70 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb53dffb700 1 -- 172.21.15.16:0/692491634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54000f620 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 10 client.18132 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559b18acfcd0 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb53dffb700 1 -- 172.21.15.16:0/692491634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb54800de70 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb53dffb700 1 -- 172.21.15.16:0/692491634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb540048070 con 0x559b18cfa890 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 10 client.18132 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 20 client.18132 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d6b670 0x559b18d6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 4 client.18132 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b18c42ec0 con 0x559b18d6b670 2024-05-08T23:31:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.139+0000 7fb555d9b700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d6b670 0x559b18d6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb53dffb700 10 client.18132 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb555d9b700 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d6b670 0x559b18d6da60 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e193e5ef636d7d9b server_cookie=1618944376f9be54 in_seq=0 out_seq=0 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb53dffb700 1 -- 172.21.15.16:0/692491634 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559b18c42ec0 con 0x559b18d6b670 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb53dffb700 10 client.18132 handle_command_reply: tid=0 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 client.18132 shutdown 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 msgr2=0x559b18d0b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d0b1a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fb5400098e0 tx=0x7fb540023b40 comp rx=0 tx=0).stop 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 shutdown_connections 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b18d00720 0x559b18cfc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b18cfa890 0x559b18d0b1a0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x559b18d6b670 0x559b18d6da60 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 --2- 172.21.15.16:0/692491634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x559b18cffe00 0x559b18cfbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb53dffb700 0 client.18132 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 >> 172.21.15.16:0/692491634 conn(0x559b18c58210 msgr2=0x559b18cfeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 shutdown_connections 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.140+0000 7fb568f71b80 1 -- 172.21.15.16:0/692491634 wait complete. 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb568f71b80 20 client.18132 trim_cache size 0 max 16384 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-05-08T23:31:44.147 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5440a1290 msgr2=0x7fb5440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5440a1290 0x7fb5440a3750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb540006010 tx=0x7fb540005ee0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 msgr2=0x7fb558101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558101fe0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fb55000ef30 tx=0x7fb55000c450 comp rx=0 tx=0).stop 2024-05-08T23:31:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 shutdown_connections 2024-05-08T23:31:44.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb5440a1290 0x7fb5440a3750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb558105640 0x7fb558101fe0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5580ff580 0x7fb558101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 --2- 172.21.15.16:0/4180925173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5580fff40 0x7fb558101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 >> 172.21.15.16:0/4180925173 conn(0x7fb5580763a0 msgr2=0x7fb5581039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 shutdown_connections 2024-05-08T23:31:44.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.141+0000 7fb55cdb7700 1 -- 172.21.15.16:0/4180925173 wait complete. 2024-05-08T23:31:44.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T23:31:44.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:31:44.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:43 smithi189 ceph-mon[46919]: pgmap v379: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:44.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:43 smithi155 ceph-mon[47105]: pgmap v379: 129 pgs: 129 active+clean; 24 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 10 op/s 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 -- 172.21.15.16:0/2515515864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 msgr2=0x7f49d810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 --2- 172.21.15.16:0/2515515864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d810cbe0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b0a0 tx=0x7f49d000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 -- 172.21.15.16:0/2515515864 shutdown_connections 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 --2- 172.21.15.16:0/2515515864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d810cbe0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 --2- 172.21.15.16:0/2515515864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d80fff40 0x7f49d8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 --2- 172.21.15.16:0/2515515864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d81008f0 0x7f49d8104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 -- 172.21.15.16:0/2515515864 >> 172.21.15.16:0/2515515864 conn(0x7f49d80fb7e0 msgr2=0x7f49d80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 -- 172.21.15.16:0/2515515864 shutdown_connections 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.294+0000 7f49df0bf700 1 -- 172.21.15.16:0/2515515864 wait complete. 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 Processor -- start 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 -- start start 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80fff40 0x7f49d8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d8198c50 0x7f49d81a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a3940 con 0x7f49d8105060 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a3ab0 con 0x7f49d80fff40 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49df0bf700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a3c70 con 0x7f49d8198c50 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48814/0 (socket says 172.21.15.16:48814) 2024-05-08T23:31:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 -- 172.21.15.16:0/1224833525 learned_addr learned my addr 172.21.15.16:0/1224833525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49dce5b700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80fff40 0x7f49d8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49dd65c700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d8198c50 0x7f49d81a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 -- 172.21.15.16:0/1224833525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d8198c50 msgr2=0x7f49d81a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d8198c50 0x7f49d81a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 -- 172.21.15.16:0/1224833525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80fff40 msgr2=0x7f49d8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80fff40 0x7f49d8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.295+0000 7f49d7fff700 1 -- 172.21.15.16:0/1224833525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009d00 con 0x7f49d8105060 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49dd65c700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d8198c50 0x7f49d81a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49d7fff700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d81986c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e970 tx=0x7f49cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc009790 con 0x7f49d8105060 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49cc00eec0 con 0x7f49d8105060 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d81a3f50 con 0x7f49d8105060 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0054b0 con 0x7f49d8105060 2024-05-08T23:31:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.296+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d81a43d0 con 0x7f49d8105060 2024-05-08T23:31:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.297+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8108330 con 0x7f49d8105060 2024-05-08T23:31:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.298+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f49cc004750 con 0x7f49d8105060 2024-05-08T23:31:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.299+0000 7f49d5ffb700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f49c00a1290 0x7f49c00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.299+0000 7f49dce5b700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f49c00a1290 0x7f49c00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.299+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f49cc014070 con 0x7f49d8105060 2024-05-08T23:31:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.300+0000 7f49dce5b700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f49c00a1290 0x7f49c00a3750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009c60 tx=0x7f49c8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.304+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f49cc096310 con 0x7f49d8105060 2024-05-08T23:31:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.576+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49d80685b0 con 0x7f49d8105060 2024-05-08T23:31:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.577+0000 7f49d5ffb700 1 -- 172.21.15.16:0/1224833525 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f49cc099930 con 0x7f49d8105060 2024-05-08T23:31:44.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:31:44.578 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f49c00a1290 msgr2=0x7f49c00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f49c00a1290 0x7f49c00a3750 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009c60 tx=0x7f49c8009260 comp rx=0 tx=0).stop 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 msgr2=0x7f49d81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d81986c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00e970 tx=0x7f49cc00ec80 comp rx=0 tx=0).stop 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 shutdown_connections 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f49c00a1290 0x7f49c00a3750 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d8198c50 0x7f49d81a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49d80fff40 0x7f49d8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 --2- 172.21.15.16:0/1224833525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49d8105060 0x7f49d81986c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 >> 172.21.15.16:0/1224833525 conn(0x7f49d80fb7e0 msgr2=0x7f49d80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 shutdown_connections 2024-05-08T23:31:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.579+0000 7f49df0bf700 1 -- 172.21.15.16:0/1224833525 wait complete. 2024-05-08T23:31:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:31:44.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T23:31:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 -- 172.21.15.16:0/2639049961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c101b10 msgr2=0x7f116c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 --2- 172.21.15.16:0/2639049961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c101b10 0x7f116c105b50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1154009990 tx=0x7f1154009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 -- 172.21.15.16:0/2639049961 shutdown_connections 2024-05-08T23:31:44.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 --2- 172.21.15.16:0/2639049961 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c106280 0x7f116c10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 --2- 172.21.15.16:0/2639049961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f116c101160 0x7f116c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 --2- 172.21.15.16:0/2639049961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c101b10 0x7f116c105b50 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 -- 172.21.15.16:0/2639049961 >> 172.21.15.16:0/2639049961 conn(0x7f116c0fc9e0 msgr2=0x7f116c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 -- 172.21.15.16:0/2639049961 shutdown_connections 2024-05-08T23:31:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.734+0000 7f1170a97700 1 -- 172.21.15.16:0/2639049961 wait complete. 2024-05-08T23:31:44.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.735+0000 7f1170a97700 1 Processor -- start 2024-05-08T23:31:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.735+0000 7f1170a97700 1 -- start start 2024-05-08T23:31:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1170a97700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c101160 0x7f116c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1170a97700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f116c106280 0x7f116c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1170a97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 0x7f116c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1170a97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c19c680 con 0x7f116c19bc00 2024-05-08T23:31:44.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1170a97700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c19c7f0 con 0x7f116c101160 2024-05-08T23:31:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1170a97700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c19c9b0 con 0x7f116c106280 2024-05-08T23:31:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 0x7f116c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 0x7f116c19c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48834/0 (socket says 172.21.15.16:48834) 2024-05-08T23:31:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 -- 172.21.15.16:0/3872726725 learned_addr learned my addr 172.21.15.16:0/3872726725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f1169d9b700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f116c106280 0x7f116c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116a59c700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c101160 0x7f116c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 -- 172.21.15.16:0/3872726725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f116c106280 msgr2=0x7f116c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f116c106280 0x7f116c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 -- 172.21.15.16:0/3872726725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c101160 msgr2=0x7f116c197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c101160 0x7f116c197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 -- 172.21.15.16:0/3872726725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540095f0 con 0x7f116c19bc00 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.736+0000 7f116ad9d700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 0x7f116c19c080 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1164007ee0 tx=0x7f116400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.737+0000 7f11637fe700 1 -- 172.21.15.16:0/3872726725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164017070 con 0x7f116c19bc00 2024-05-08T23:31:44.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.737+0000 7f11637fe700 1 -- 172.21.15.16:0/3872726725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f116400f040 con 0x7f116c19bc00 2024-05-08T23:31:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.737+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c1a3f10 con 0x7f116c19bc00 2024-05-08T23:31:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.737+0000 7f11637fe700 1 -- 172.21.15.16:0/3872726725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164005710 con 0x7f116c19bc00 2024-05-08T23:31:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.737+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c1a43a0 con 0x7f116c19bc00 2024-05-08T23:31:44.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.739+0000 7f11637fe700 1 -- 172.21.15.16:0/3872726725 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1164005870 con 0x7f116c19bc00 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.740+0000 7f11637fe700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f11580a1290 0x7f11580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.740+0000 7f11637fe700 1 -- 172.21.15.16:0/3872726725 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f11640cc8e0 con 0x7f116c19bc00 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.740+0000 7f116a59c700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f11580a1290 0x7f11580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.740+0000 7f116a59c700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f11580a1290 0x7f11580a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f115c009bf0 tx=0x7f115c009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 Processor -- start 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 -- start start 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cee71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cee1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1ce38150 con 0x558c1cee6e00 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1ce35b10 con 0x558c1cee1890 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f117cc51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1ce31400 con 0x558c1cee7720 2024-05-08T23:31:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f116a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cee71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f116a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cee71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48848/0 (socket says 172.21.15.16:48848) 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.741+0000 7f116a59c700 1 -- 172.21.15.16:0/139683552 learned_addr learned my addr 172.21.15.16:0/139683552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f1169d9b700 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116ad9d700 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cee1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116a59c700 1 -- 172.21.15.16:0/139683552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 msgr2=0x558c1cee1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116a59c700 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116a59c700 1 -- 172.21.15.16:0/139683552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 msgr2=0x558c1cee1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116a59c700 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cee1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116a59c700 1 -- 172.21.15.16:0/139683552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c1ccb23b0 con 0x558c1cee6e00 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f1169d9b700 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f116a59c700 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cee71e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f115c011e40 tx=0x7f115c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72b59cee1805009 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f1160ff9700 1 -- 172.21.15.16:0/139683552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c004220 con 0x558c1cee6e00 2024-05-08T23:31:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f1160ff9700 1 -- 172.21.15.16:0/139683552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f115c016070 con 0x558c1cee6e00 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.742+0000 7f1160ff9700 1 -- 172.21.15.16:0/139683552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c01f680 con 0x558c1cee6e00 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 -- 172.21.15.16:0/139683552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 msgr2=0x558c1cee71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cee71e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f115c011e40 tx=0x7f115c014000 comp rx=0 tx=0).stop 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 -- 172.21.15.16:0/139683552 shutdown_connections 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cee1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cee71e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 --2- 172.21.15.16:0/139683552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 -- 172.21.15.16:0/139683552 >> 172.21.15.16:0/139683552 conn(0x558c1ce3f210 msgr2=0x558c1cedbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 -- 172.21.15.16:0/139683552 shutdown_connections 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.743+0000 7f117cc51b80 1 -- 172.21.15.16:0/139683552 wait complete. 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 -- start start 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 Processor -- start 2024-05-08T23:31:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1ceea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1ceeaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee7720 0x558c1ceeb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cdcd560 con 0x558c1cee6e00 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cdcd200 con 0x558c1cee7720 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1ccb46a0 con 0x558c1cee1890 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f117cc51b80 10 client.0 _ll_register_callbacks cb 0x558c1cdd6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1ceeaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1ceeaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48854/0 (socket says 172.21.15.16:48854) 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 -- 172.21.15.16:0/2201319828 learned_addr learned my addr 172.21.15.16:0/2201319828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f116a59c700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1ceea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f116ad9d700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee7720 0x558c1ceeb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 -- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 msgr2=0x558c1ceea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1ceea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 -- 172.21.15.16:0/2201319828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee7720 msgr2=0x558c1ceeb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee7720 0x558c1ceeb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f1169d9b700 1 -- 172.21.15.16:0/2201319828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c011b20 con 0x558c1cee6e00 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.744+0000 7f116ad9d700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee7720 0x558c1ceeb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1169d9b700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1ceeaf20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f116c067dd0 tx=0x7f11540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ab26680423178c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1151ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1151ffb700 1 -- 172.21.15.16:0/2201319828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400bed0 con 0x558c1cee6e00 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1151ffb700 1 -- 172.21.15.16:0/2201319828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1154004500 con 0x558c1cee6e00 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c1cd8a270 con 0x558c1cee6e00 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1151ffb700 1 -- 172.21.15.16:0/2201319828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11540046c0 con 0x558c1cee6e00 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f117cc51b80 10 client.18150 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558c1cc5eb60 con 0x558c1cee6e00 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1151ffb700 1 -- 172.21.15.16:0/2201319828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558c1cd8a270 con 0x558c1cee6e00 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f1151ffb700 1 -- 172.21.15.16:0/2201319828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f115400fa40 con 0x558c1cee6e00 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.745+0000 7f117cc51b80 10 client.18150 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f117cc51b80 10 client.18150 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f117cc51b80 20 client.18150 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f117cc51b80 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cefb1b0 0x558c1cefd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f117cc51b80 4 client.18150 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c1ceb0810 con 0x558c1cefb1b0 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f116a59c700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cefb1b0 0x558c1cefd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f1151ffb700 10 client.18150 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.746+0000 7f116a59c700 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cefb1b0 0x558c1cefd5d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7e1b90908f1d0ff server_cookie=af43d3a55eeb9f4f in_seq=0 out_seq=0 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f1151ffb700 1 -- 172.21.15.16:0/2201319828 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c1ceb0810 con 0x558c1cefb1b0 2024-05-08T23:31:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f1151ffb700 10 client.18150 handle_command_reply: tid=0 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 client.18150 shutdown 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 msgr2=0x558c1ceeaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1ceeaf20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f116c067dd0 tx=0x7f11540057f0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 shutdown_connections 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cefb1b0 0x558c1cefd5d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee7720 0x558c1ceeb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1ceea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 --2- 172.21.15.16:0/2201319828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1ceeaf20 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f1151ffb700 0 client.18150 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 >> 172.21.15.16:0/2201319828 conn(0x558c1ce3f210 msgr2=0x558c1cedb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 shutdown_connections 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.747+0000 7f117cc51b80 1 -- 172.21.15.16:0/2201319828 wait complete. 2024-05-08T23:31:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.748+0000 7f117cc51b80 20 client.18150 trim_cache size 0 max 16384 2024-05-08T23:31:44.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 Processor -- start 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 -- start start 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1cef6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee6e00 0x558c1cedbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 0x558c1cef2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cedcae0 con 0x558c1cee7720 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cf027a0 con 0x558c1cee6e00 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f117cc51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1ced6800 con 0x558c1cee1890 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f116ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 0x558c1cef2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f116ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 0x558c1cef2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48862/0 (socket says 172.21.15.16:48862) 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.762+0000 7f116ad9d700 1 -- 172.21.15.16:0/765842484 learned_addr learned my addr 172.21.15.16:0/765842484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116a59c700 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1cef6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f1169d9b700 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee6e00 0x558c1cedbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116ad9d700 1 -- 172.21.15.16:0/765842484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 msgr2=0x558c1cef6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116ad9d700 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1cef6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116ad9d700 1 -- 172.21.15.16:0/765842484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee6e00 msgr2=0x558c1cedbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116ad9d700 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee6e00 0x558c1cedbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116ad9d700 1 -- 172.21.15.16:0/765842484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c1cdf7560 con 0x558c1cee7720 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f1169d9b700 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee6e00 0x558c1cedbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f116ad9d700 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 0x558c1cef2160 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1164009510 tx=0x7f1164084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538397927ac4f0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f11617fa700 1 -- 172.21.15.16:0/765842484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164022250 con 0x558c1cee7720 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f11617fa700 1 -- 172.21.15.16:0/765842484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1164099d50 con 0x558c1cee7720 2024-05-08T23:31:44.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.763+0000 7f11617fa700 1 -- 172.21.15.16:0/765842484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11640a1ad0 con 0x558c1cee7720 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 -- 172.21.15.16:0/765842484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 msgr2=0x558c1cef2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 0x558c1cef2160 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f1164009510 tx=0x7f1164084040 comp rx=0 tx=0).stop 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 -- 172.21.15.16:0/765842484 shutdown_connections 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee7720 0x558c1cef2160 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee1890 0x558c1cef6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 --2- 172.21.15.16:0/765842484 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee6e00 0x558c1cedbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 -- 172.21.15.16:0/765842484 >> 172.21.15.16:0/765842484 conn(0x558c1ce3f210 msgr2=0x558c1ced8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 -- 172.21.15.16:0/765842484 shutdown_connections 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 -- 172.21.15.16:0/765842484 wait complete. 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 -- start start 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.764+0000 7f117cc51b80 1 Processor -- start 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cef2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cef26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cee2e70 con 0x558c1cee6e00 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cee2fe0 con 0x558c1cee1890 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c1cee31a0 con 0x558c1cee7720 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 10 client.0 _ll_register_callbacks cb 0x558c1cdd6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cef26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cef26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48866/0 (socket says 172.21.15.16:48866) 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 -- 172.21.15.16:0/156802134 learned_addr learned my addr 172.21.15.16:0/156802134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f116a59c700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cef2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f116ad9d700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 -- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 msgr2=0x558c1cee2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 -- 172.21.15.16:0/156802134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 msgr2=0x558c1cef2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cef2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 -- 172.21.15.16:0/156802134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c1a43a0 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f116a59c700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cef2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1169d9b700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cef26b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1154005f50 tx=0x7f115400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68be672fb5d8d5ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1151ffb700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1151ffb700 1 -- 172.21.15.16:0/156802134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400f660 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f1151ffb700 1 -- 172.21.15.16:0/156802134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1154000e40 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.765+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f115c016070 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f1151ffb700 1 -- 172.21.15.16:0/156802134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f115400f7c0 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 10 client.18156 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558c1ccb6cd0 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f1151ffb700 1 -- 172.21.15.16:0/156802134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f115c016070 con 0x558c1cee6e00 2024-05-08T23:31:44.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f1151ffb700 1 -- 172.21.15.16:0/156802134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1154017c50 con 0x558c1cee6e00 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 10 client.18156 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 10 client.18156 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 20 client.18156 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cf52130 0x558c1cf54550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 4 client.18156 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 --> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c1ce29ec0 con 0x558c1cf52130 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.766+0000 7f116a59c700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cf52130 0x558c1cf54550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f116a59c700 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cf52130 0x558c1cf54550 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1dcfca39728c415a server_cookie=6391423e9ce4cff1 in_seq=0 out_seq=0 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f1151ffb700 10 client.18156 ms_handle_connect on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f1151ffb700 1 -- 172.21.15.16:0/156802134 <== mds.2 v2:172.21.15.189:6836/2098584238 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c1ce29ec0 con 0x558c1cf52130 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f1151ffb700 10 client.18156 handle_command_reply: tid=0 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 client.18156 shutdown 2024-05-08T23:31:44.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 msgr2=0x558c1cef26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cef26b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1154005f50 tx=0x7f115400b940 comp rx=0 tx=0).stop 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 shutdown_connections 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c1cee7720 0x558c1cee2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.189:6836/2098584238,v1:172.21.15.189:6837/2098584238] conn(0x558c1cf52130 0x558c1cf54550 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x558c1cee1890 0x558c1cef2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 --2- 172.21.15.16:0/156802134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c1cee6e00 0x558c1cef26b0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f1151ffb700 0 client.18156 ms_handle_reset on v2:172.21.15.189:6836/2098584238 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 >> 172.21.15.16:0/156802134 conn(0x558c1ce3f210 msgr2=0x558c1cee92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 shutdown_connections 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 1 -- 172.21.15.16:0/156802134 wait complete. 2024-05-08T23:31:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.767+0000 7f117cc51b80 20 client.18156 trim_cache size 0 max 16384 2024-05-08T23:31:44.776 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:31:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f11580a1290 msgr2=0x7f11580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f11580a1290 0x7f11580a3750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f115c009bf0 tx=0x7f115c009260 comp rx=0 tx=0).stop 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 msgr2=0x7f116c19c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 0x7f116c19c080 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1164007ee0 tx=0x7f116400d3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 shutdown_connections 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f11580a1290 0x7f11580a3750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116c19bc00 0x7f116c19c080 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f116c101160 0x7f116c197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 --2- 172.21.15.16:0/3872726725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f116c106280 0x7f116c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 >> 172.21.15.16:0/3872726725 conn(0x7f116c0fc9e0 msgr2=0x7f116c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 shutdown_connections 2024-05-08T23:31:44.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.768+0000 7f1170a97700 1 -- 172.21.15.16:0/3872726725 wait complete. 2024-05-08T23:31:44.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T23:31:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 -- 172.21.15.16:0/3947738026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e4106290 msgr2=0x7fb7e410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 --2- 172.21.15.16:0/3947738026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e4106290 0x7fb7e410de10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00b0d0 tx=0x7fb7dc00b3e0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 -- 172.21.15.16:0/3947738026 shutdown_connections 2024-05-08T23:31:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 --2- 172.21.15.16:0/3947738026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e4106290 0x7fb7e410de10 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 --2- 172.21.15.16:0/3947738026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7e40682e0 0x7fb7e40686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 --2- 172.21.15.16:0/3947738026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e4068c90 0x7fb7e4105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 -- 172.21.15.16:0/3947738026 >> 172.21.15.16:0/3947738026 conn(0x7fb7e4076f40 msgr2=0x7fb7e4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 -- 172.21.15.16:0/3947738026 shutdown_connections 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.921+0000 7fb7e916a700 1 -- 172.21.15.16:0/3947738026 wait complete. 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 Processor -- start 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 -- start start 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e40682e0 0x7fb7e40ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7e4106290 0x7fb7e41003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 0x7fb7e41a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41a5b70 con 0x7fb7e41008e0 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41a5ce0 con 0x7fb7e40682e0 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.922+0000 7fb7e916a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e41a5ea0 con 0x7fb7e4106290 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 0x7fb7e41a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 0x7fb7e41a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48880/0 (socket says 172.21.15.16:48880) 2024-05-08T23:31:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 -- 172.21.15.16:0/1644233570 learned_addr learned my addr 172.21.15.16:0/1644233570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e259c700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7e4106290 0x7fb7e41003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e40682e0 0x7fb7e40ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 -- 172.21.15.16:0/1644233570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7e4106290 msgr2=0x7fb7e41003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7e4106290 0x7fb7e41003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 -- 172.21.15.16:0/1644233570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e40682e0 msgr2=0x7fb7e40ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e40682e0 0x7fb7e40ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 -- 172.21.15.16:0/1644233570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc009d00 con 0x7fb7e41008e0 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e359e700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 0x7fb7e41a55a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc006070 tx=0x7fb7dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7dbfff700 1 -- 172.21.15.16:0/1644233570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00e040 con 0x7fb7e41008e0 2024-05-08T23:31:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e41a6120 con 0x7fb7e41008e0 2024-05-08T23:31:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7dbfff700 1 -- 172.21.15.16:0/1644233570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7dc003e90 con 0x7fb7e41008e0 2024-05-08T23:31:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7dbfff700 1 -- 172.21.15.16:0/1644233570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc004440 con 0x7fb7e41008e0 2024-05-08T23:31:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.923+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e41a65f0 con 0x7fb7e41008e0 2024-05-08T23:31:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.926+0000 7fb7dbfff700 1 -- 172.21.15.16:0/1644233570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb7dc003a00 con 0x7fb7e41008e0 2024-05-08T23:31:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.927+0000 7fb7dbfff700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb7d00a1310 0x7fb7d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.927+0000 7fb7dbfff700 1 -- 172.21.15.16:0/1644233570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb7dc0ce2b0 con 0x7fb7e41008e0 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.927+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb7d00a1310 0x7fb7d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.927+0000 7fb7f5324b80 1 Processor -- start 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.927+0000 7fb7f5324b80 1 -- start start 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.927+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb7d00a1310 0x7fb7d00a37d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005950 tx=0x7fb7d40058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18aa001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18a9fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a951150 con 0x55b18a9fa890 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a94eb10 con 0x55b18aa00720 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a94a400 con 0x55b18a9ffe00 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9facd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48894/0 (socket says 172.21.15.16:48894) 2024-05-08T23:31:44.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 -- 172.21.15.16:0/2063206059 learned_addr learned my addr 172.21.15.16:0/2063206059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18aa001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e259c700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18a9fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 -- 172.21.15.16:0/2063206059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 msgr2=0x55b18aa001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18aa001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 -- 172.21.15.16:0/2063206059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 msgr2=0x55b18a9fa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18a9fa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 -- 172.21.15.16:0/2063206059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b18a7cb3b0 con 0x55b18a9fa890 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e259c700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18a9fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18aa001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.928+0000 7fb7e359e700 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9facd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a2f10 tx=0x7fb7dc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e4f6dbfc85b09f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.929+0000 7fb7d97fa700 1 -- 172.21.15.16:0/2063206059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc09b750 con 0x55b18a9fa890 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.929+0000 7fb7d97fa700 1 -- 172.21.15.16:0/2063206059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7dc09b050 con 0x55b18a9fa890 2024-05-08T23:31:44.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.929+0000 7fb7d97fa700 1 -- 172.21.15.16:0/2063206059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc099480 con 0x55b18a9fa890 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.929+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2063206059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 msgr2=0x55b18a9facd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.929+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9facd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a2f10 tx=0x7fb7dc0099e0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2063206059 shutdown_connections 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9facd0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18aa001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2063206059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18a9fa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2063206059 >> 172.21.15.16:0/2063206059 conn(0x55b18a958210 msgr2=0x55b18a9f4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2063206059 shutdown_connections 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2063206059 wait complete. 2024-05-08T23:31:44.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- start start 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 Processor -- start 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa03a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa03f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18aa044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a8e6560 con 0x55b18a9ffe00 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a8e6200 con 0x55b18a9fa890 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a7cd6a0 con 0x55b18aa00720 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.930+0000 7fb7f5324b80 10 client.0 _ll_register_callbacks cb 0x55b18a8efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa03f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa03f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48896/0 (socket says 172.21.15.16:48896) 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 -- 172.21.15.16:0/3122933204 learned_addr learned my addr 172.21.15.16:0/3122933204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e359e700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18aa044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa03a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 -- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 msgr2=0x55b18aa044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18aa044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 -- 172.21.15.16:0/3122933204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 msgr2=0x55b18aa03a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa03a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 -- 172.21.15.16:0/3122933204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e41a65f0 con 0x55b18a9ffe00 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7e259c700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa03f90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0116f0 tx=0x7fb7cc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6c0a218a6b1622 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7ca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3122933204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc016070 con 0x55b18a9ffe00 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3122933204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb7cc004320 con 0x55b18a9ffe00 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b18a8a3270 con 0x55b18a9ffe00 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.931+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3122933204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7cc0044e0 con 0x55b18a9ffe00 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 10 client.18168 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b18a777b60 con 0x55b18a9ffe00 2024-05-08T23:31:44.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3122933204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b18a8a3270 con 0x55b18a9ffe00 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3122933204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb7cc00f780 con 0x55b18a9ffe00 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 10 client.18168 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 10 client.18168 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 20 client.18168 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa14080 0x55b18aa164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 4 client.18168 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b18a9c9810 con 0x55b18aa14080 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.932+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa14080 0x55b18aa164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.933+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa14080 0x55b18aa164a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ab979fda30a9a7d server_cookie=5b6678995b56cd39 in_seq=0 out_seq=0 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.933+0000 7fb7ca7fc700 10 client.18168 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.933+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/3122933204 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b18a9c9810 con 0x55b18aa14080 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.933+0000 7fb7ca7fc700 10 client.18168 handle_command_reply: tid=0 2024-05-08T23:31:44.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 client.18168 shutdown 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 msgr2=0x55b18aa03f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa03f90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0116f0 tx=0x7fb7cc011a00 comp rx=0 tx=0).stop 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 shutdown_connections 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18aa044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa14080 0x55b18aa164a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa03a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/3122933204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa03f90 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7ca7fc700 0 client.18168 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 >> 172.21.15.16:0/3122933204 conn(0x55b18a958210 msgr2=0x55b18a9f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 shutdown_connections 2024-05-08T23:31:44.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 1 -- 172.21.15.16:0/3122933204 wait complete. 2024-05-08T23:31:44.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.934+0000 7fb7f5324b80 20 client.18168 trim_cache size 0 max 16384 2024-05-08T23:31:44.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 Processor -- start 2024-05-08T23:31:44.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 -- start start 2024-05-08T23:31:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18a9f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18aa02a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a9c9810 con 0x55b18a9fa890 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a8a3270 con 0x55b18aa00720 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.948+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a9f5b30 con 0x55b18a9ffe00 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9fa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48910/0 (socket says 172.21.15.16:48910) 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 -- 172.21.15.16:0/1004823912 learned_addr learned my addr 172.21.15.16:0/1004823912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e259c700 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18a9f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e359e700 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18aa02a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 -- 172.21.15.16:0/1004823912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 msgr2=0x55b18a9f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18a9f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 -- 172.21.15.16:0/1004823912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 msgr2=0x55b18aa02a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18aa02a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 -- 172.21.15.16:0/1004823912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b18a910560 con 0x55b18a9fa890 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9fa350 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4014040 tx=0x7fb7d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9b08dc665cc58d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7d9ffb700 1 -- 172.21.15.16:0/1004823912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4025040 con 0x55b18a9fa890 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7d9ffb700 1 -- 172.21.15.16:0/1004823912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7d4027750 con 0x55b18a9fa890 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.949+0000 7fb7d9ffb700 1 -- 172.21.15.16:0/1004823912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4011440 con 0x55b18a9fa890 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 -- 172.21.15.16:0/1004823912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 msgr2=0x55b18a9fa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9fa350 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4014040 tx=0x7fb7d4015000 comp rx=0 tx=0).stop 2024-05-08T23:31:44.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 -- 172.21.15.16:0/1004823912 shutdown_connections 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18aa00720 0x55b18aa02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9fa890 0x55b18a9fa350 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/1004823912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18a9ffe00 0x55b18a9f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 -- 172.21.15.16:0/1004823912 >> 172.21.15.16:0/1004823912 conn(0x55b18a958210 msgr2=0x55b18aa02340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 -- 172.21.15.16:0/1004823912 shutdown_connections 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 -- 172.21.15.16:0/1004823912 wait complete. 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.950+0000 7fb7f5324b80 1 -- start start 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 Processor -- start 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa0af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa0b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18a9fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a9fc860 con 0x55b18a9ffe00 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a9fc9d0 con 0x55b18a9fa890 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b18a9fcb90 con 0x55b18aa00720 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7f5324b80 10 client.0 _ll_register_callbacks cb 0x55b18a8efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa0b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa0b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48924/0 (socket says 172.21.15.16:48924) 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 -- 172.21.15.16:0/2467805307 learned_addr learned my addr 172.21.15.16:0/2467805307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa0af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e359e700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18a9fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 -- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 msgr2=0x55b18a9fc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18a9fc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 -- 172.21.15.16:0/2467805307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 msgr2=0x55b18aa0af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:44.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa0af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e259c700 1 -- 172.21.15.16:0/2467805307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d400f960 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.951+0000 7fb7e359e700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18a9fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7e259c700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa0b4b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0116c0 tx=0x7fb7cc009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bb006ded49c729 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7ca7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/2467805307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc016040 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/2467805307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb7cc000e40 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7dc09b050 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/2467805307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7cc00f480 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 10 client.18174 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b18a7cfcd0 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/2467805307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7dc09b050 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/2467805307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb7cc01a460 con 0x55b18a9ffe00 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 10 client.18174 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:44.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 10 client.18174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 20 client.18174 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa6b820 0x55b18aa6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 4 client.18174 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.952+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 --> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b18a942ec0 con 0x55b18aa6b820 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa6b820 0x55b18aa6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7e2d9d700 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa6b820 0x55b18aa6dc40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0ac23d1a8195178 server_cookie=5f1d382ef233b2bf in_seq=0 out_seq=0 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7ca7fc700 10 client.18174 ms_handle_connect on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7ca7fc700 1 -- 172.21.15.16:0/2467805307 <== mds.1 v2:172.21.15.189:6838/3614950817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b18a942ec0 con 0x55b18aa6b820 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7ca7fc700 10 client.18174 handle_command_reply: tid=0 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 client.18174 shutdown 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 msgr2=0x55b18aa0b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa0b4b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0116c0 tx=0x7fb7cc009340 comp rx=0 tx=0).stop 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 shutdown_connections 2024-05-08T23:31:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b18aa00720 0x55b18a9fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.189:6838/3614950817,v1:172.21.15.189:6839/3614950817] conn(0x55b18aa6b820 0x55b18aa6dc40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55b18a9fa890 0x55b18aa0af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7ca7fc700 0 client.18174 ms_handle_reset on v2:172.21.15.189:6838/3614950817 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 --2- 172.21.15.16:0/2467805307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b18a9ffe00 0x55b18aa0b4b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 >> 172.21.15.16:0/2467805307 conn(0x55b18a958210 msgr2=0x55b18aa02bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 shutdown_connections 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.953+0000 7fb7f5324b80 1 -- 172.21.15.16:0/2467805307 wait complete. 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7f5324b80 20 client.18174 trim_cache size 0 max 16384 2024-05-08T23:31:44.961 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:31:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb7d00a1310 msgr2=0x7fb7d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb7d00a1310 0x7fb7d00a37d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4005950 tx=0x7fb7d40058e0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 msgr2=0x7fb7e41a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:44.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 0x7fb7e41a55a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc006070 tx=0x7fb7dc00cce0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 shutdown_connections 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb7d00a1310 0x7fb7d00a37d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7e41008e0 0x7fb7e41a55a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e40682e0 0x7fb7e40ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 --2- 172.21.15.16:0/1644233570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7e4106290 0x7fb7e41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 >> 172.21.15.16:0/1644233570 conn(0x7fb7e4076f40 msgr2=0x7fb7e40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 shutdown_connections 2024-05-08T23:31:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:44.954+0000 7fb7e916a700 1 -- 172.21.15.16:0/1644233570 wait complete. 2024-05-08T23:31:44.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T23:31:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 -- 172.21.15.16:0/2686384451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 msgr2=0x7f1ab4100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2686384451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4100440 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009990 tx=0x7f1aa4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 -- 172.21.15.16:0/2686384451 shutdown_connections 2024-05-08T23:31:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2686384451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab4105160 0x7f1ab410cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2686384451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4100440 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2686384451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab4100a10 0x7f1ab4104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 -- 172.21.15.16:0/2686384451 >> 172.21.15.16:0/2686384451 conn(0x7f1ab4076f40 msgr2=0x7f1ab4077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 -- 172.21.15.16:0/2686384451 shutdown_connections 2024-05-08T23:31:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.109+0000 7f1ab9105700 1 -- 172.21.15.16:0/2686384451 wait complete. 2024-05-08T23:31:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.110+0000 7f1ab9105700 1 Processor -- start 2024-05-08T23:31:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 -- start start 2024-05-08T23:31:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab4105160 0x7f1ab4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab419bc10 0x7f1ab419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419c690 con 0x7f1ab4100060 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419c800 con 0x7f1ab4105160 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab9105700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419c9c0 con 0x7f1ab419bc10 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48950/0 (socket says 172.21.15.16:48950) 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/2322566344 learned_addr learned my addr 172.21.15.16:0/2322566344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab259c700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab4105160 0x7f1ab4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab359e700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab419bc10 0x7f1ab419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/2322566344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab419bc10 msgr2=0x7f1ab419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab419bc10 0x7f1ab419c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/2322566344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab4105160 msgr2=0x7f1ab4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab4105160 0x7f1ab4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/2322566344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40095f0 con 0x7f1ab4100060 2024-05-08T23:31:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.111+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4197330 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40059d0 tx=0x7f1aa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.112+0000 7f1aabfff700 1 -- 172.21.15.16:0/2322566344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401d070 con 0x7f1ab4100060 2024-05-08T23:31:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.112+0000 7f1aabfff700 1 -- 172.21.15.16:0/2322566344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1aa400fb80 con 0x7f1ab4100060 2024-05-08T23:31:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.112+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab41a3ec0 con 0x7f1ab4100060 2024-05-08T23:31:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.112+0000 7f1aabfff700 1 -- 172.21.15.16:0/2322566344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4022b00 con 0x7f1ab4100060 2024-05-08T23:31:45.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.112+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab41a4280 con 0x7f1ab4100060 2024-05-08T23:31:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.114+0000 7f1aabfff700 1 -- 172.21.15.16:0/2322566344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1aa400f6f0 con 0x7f1ab4100060 2024-05-08T23:31:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.115+0000 7f1aabfff700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1aa00a1290 0x7f1aa00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.115+0000 7f1aabfff700 1 -- 172.21.15.16:0/2322566344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1aa40ce880 con 0x7f1ab4100060 2024-05-08T23:31:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.115+0000 7f1ab259c700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1aa00a1290 0x7f1aa00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.115+0000 7f1ab259c700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1aa00a1290 0x7f1aa00a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4198880 tx=0x7f1a9c00b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:45.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 Processor -- start 2024-05-08T23:31:45.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 -- start start 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b7015f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b6fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b6fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b64e150 con 0x56106b6fbca0 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b64bb10 con 0x56106b701b30 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.116+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b647400 con 0x56106b701210 2024-05-08T23:31:45.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b6fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b6fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48960/0 (socket says 172.21.15.16:48960) 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 -- 172.21.15.16:0/1583878655 learned_addr learned my addr 172.21.15.16:0/1583878655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b7015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab259c700 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b6fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 -- 172.21.15.16:0/1583878655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 msgr2=0x56106b7015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b7015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 -- 172.21.15.16:0/1583878655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 msgr2=0x56106b6fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b6fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 -- 172.21.15.16:0/1583878655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56106b4c83b0 con 0x56106b6fbca0 2024-05-08T23:31:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1ab359e700 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b6fc0e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f20 tx=0x7f1aac009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15802e644ecb2e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1aa97fa700 1 -- 172.21.15.16:0/1583878655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011070 con 0x56106b6fbca0 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1aa97fa700 1 -- 172.21.15.16:0/1583878655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1aac009a50 con 0x56106b6fbca0 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.117+0000 7f1aa97fa700 1 -- 172.21.15.16:0/1583878655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac015db0 con 0x56106b6fbca0 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/1583878655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 msgr2=0x56106b6fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b6fc0e0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009f20 tx=0x7f1aac009f50 comp rx=0 tx=0).stop 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/1583878655 shutdown_connections 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b6fc0e0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b7015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/1583878655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b6fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/1583878655 >> 172.21.15.16:0/1583878655 conn(0x56106b655210 msgr2=0x56106b6f1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/1583878655 shutdown_connections 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.118+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/1583878655 wait complete. 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 -- start start 2024-05-08T23:31:45.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 Processor -- start 2024-05-08T23:31:45.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b704ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b705400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b705940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b5e3560 con 0x56106b6fbca0 2024-05-08T23:31:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b5e3200 con 0x56106b701b30 2024-05-08T23:31:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b4ca6a0 con 0x56106b701210 2024-05-08T23:31:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ac52bfb80 10 client.0 _ll_register_callbacks cb 0x56106b5ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b704ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b704ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48970/0 (socket says 172.21.15.16:48970) 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/252226971 learned_addr learned my addr 172.21.15.16:0/252226971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab259c700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b705400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab359e700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b705940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/252226971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 msgr2=0x56106b705400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b705400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 msgr2=0x56106b705940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b705940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab2d9d700 1 -- 172.21.15.16:0/252226971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac009bd0 con 0x56106b6fbca0 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab359e700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b705940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.119+0000 7f1ab259c700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b705400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b704ec0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400fb50 tx=0x7f1aa409b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c592be84b3ec3af3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1a9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/252226971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa409f370 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/252226971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1aa409f570 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56106b6c6810 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/252226971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aa40a3b30 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ac52bfb80 10 client.18186 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56106b474b60 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/252226971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56106b6c6810 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/252226971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1aa409ac30 con 0x56106b6fbca0 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ac52bfb80 10 client.18186 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ac52bfb80 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.120+0000 7f1ac52bfb80 20 client.18186 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.121+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b7151d0 0x56106b7175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.121+0000 7f1ac52bfb80 4 client.18186 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.121+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56106b63fec0 con 0x56106b7151d0 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.121+0000 7f1ab259c700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b7151d0 0x56106b7175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.121+0000 7f1a9a7fc700 10 client.18186 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.121+0000 7f1ab259c700 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b7151d0 0x56106b7175f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777fd8c0f6e12d9c server_cookie=4bc181ceeb5dc724 in_seq=0 out_seq=0 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/252226971 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56106b63fec0 con 0x56106b7151d0 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1a9a7fc700 10 client.18186 handle_command_reply: tid=0 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 client.18186 shutdown 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 msgr2=0x56106b704ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b704ec0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400fb50 tx=0x7f1aa409b1d0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 shutdown_connections 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b7151d0 0x56106b7175f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701b30 0x56106b705940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b6fbca0 0x56106b704ec0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/252226971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701210 0x56106b705400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1a9a7fc700 0 client.18186 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 >> 172.21.15.16:0/252226971 conn(0x56106b655210 msgr2=0x56106b6f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 shutdown_connections 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/252226971 wait complete. 2024-05-08T23:31:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.122+0000 7f1ac52bfb80 20 client.18186 trim_cache size 0 max 16384 2024-05-08T23:31:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 Processor -- start 2024-05-08T23:31:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 -- start start 2024-05-08T23:31:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b6fbca0 0x56106b7116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 0x56106b6f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701b30 0x56106b70c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b6ea040 con 0x56106b701210 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b6f2260 con 0x56106b6fbca0 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b7175c0 con 0x56106b701b30 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ab259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 0x56106b6f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ab259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 0x56106b6f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48980/0 (socket says 172.21.15.16:48980) 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ab259c700 1 -- 172.21.15.16:0/2604547524 learned_addr learned my addr 172.21.15.16:0/2604547524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ab359e700 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701b30 0x56106b70c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.137+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b6fbca0 0x56106b7116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1ab259c700 1 -- 172.21.15.16:0/2604547524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701b30 msgr2=0x56106b70c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1ab259c700 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701b30 0x56106b70c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1ab259c700 1 -- 172.21.15.16:0/2604547524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b6fbca0 msgr2=0x56106b7116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1ab259c700 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b6fbca0 0x56106b7116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1ab259c700 1 -- 172.21.15.16:0/2604547524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56106b60d560 con 0x56106b701210 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1ab259c700 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 0x56106b6f1d20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c005950 tx=0x7f1a9c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74792b27ac1d6fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1aa9ffb700 1 -- 172.21.15.16:0/2604547524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c02a070 con 0x56106b701210 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1aa9ffb700 1 -- 172.21.15.16:0/2604547524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a9c015070 con 0x56106b701210 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.138+0000 7f1aa9ffb700 1 -- 172.21.15.16:0/2604547524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c024070 con 0x56106b701210 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/2604547524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 msgr2=0x56106b6f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 0x56106b6f1d20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c005950 tx=0x7f1a9c013000 comp rx=0 tx=0).stop 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/2604547524 shutdown_connections 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b701b30 0x56106b70c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b6fbca0 0x56106b7116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/2604547524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701210 0x56106b6f1d20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/2604547524 >> 172.21.15.16:0/2604547524 conn(0x56106b655210 msgr2=0x56106b6fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/2604547524 shutdown_connections 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/2604547524 wait complete. 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- start start 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 Processor -- start 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b6fbca0 0x56106b708570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701210 0x56106b708ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 0x56106b708ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.139+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b76a7f0 con 0x56106b701b30 2024-05-08T23:31:45.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b76a960 con 0x56106b701210 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ac52bfb80 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106b76ab20 con 0x56106b6fbca0 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ac52bfb80 10 client.0 _ll_register_callbacks cb 0x56106b5ecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 0x56106b708ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 0x56106b708ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48982/0 (socket says 172.21.15.16:48982) 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 -- 172.21.15.16:0/3147827771 learned_addr learned my addr 172.21.15.16:0/3147827771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b6fbca0 0x56106b708570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab259c700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701210 0x56106b708ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 -- 172.21.15.16:0/3147827771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b6fbca0 msgr2=0x56106b708570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b6fbca0 0x56106b708570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 -- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701210 msgr2=0x56106b708ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701210 0x56106b708ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 -- 172.21.15.16:0/3147827771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c010c20 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ab359e700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 0x56106b708ff0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009ec0 tx=0x7f1aac00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8aa7e031fde168 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1a9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/3147827771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011040 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/3147827771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f1aac006e40 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.140+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56106b5a0270 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/3147827771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aac01e840 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 10 client.18192 fetch_fsmap learned FSMap version 25 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56106b4cccd0 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/3147827771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56106b5a0270 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/3147827771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1aac020b50 con 0x56106b701b30 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 10 client.18192 fetch_fsmap finished waiting for FSMap version 25 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 20 client.18192 populate_metadata read hostname 'smithi016' 2024-05-08T23:31:45.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b76c250 0x56106b76e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 4 client.18192 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 --> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56106b495cf0 con 0x56106b76c250 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.141+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b76c250 0x56106b76e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ab2d9d700 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b76c250 0x56106b76e670 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586b81e5128d1c7d server_cookie=1fe44c790b20a9fd in_seq=0 out_seq=0 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1a9a7fc700 10 client.18192 ms_handle_connect on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1a9a7fc700 1 -- 172.21.15.16:0/3147827771 <== mds.0 v2:172.21.15.155:6838/773665585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56106b495cf0 con 0x56106b76c250 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1a9a7fc700 10 client.18192 handle_command_reply: tid=0 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 client.18192 shutdown 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 msgr2=0x56106b708ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 0x56106b708ff0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009ec0 tx=0x7f1aac00cf80 comp rx=0 tx=0).stop 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 shutdown_connections 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56106b701b30 0x56106b708ff0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56106b6fbca0 0x56106b708570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:6838/773665585,v1:172.21.15.155:6839/773665585] conn(0x56106b76c250 0x56106b76e670 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 --2- 172.21.15.16:0/3147827771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56106b701210 0x56106b708ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1a9a7fc700 0 client.18192 ms_handle_reset on v2:172.21.15.155:6838/773665585 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 >> 172.21.15.16:0/3147827771 conn(0x56106b655210 msgr2=0x56106b6fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 shutdown_connections 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.142+0000 7f1ac52bfb80 1 -- 172.21.15.16:0/3147827771 wait complete. 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ac52bfb80 20 client.18192 trim_cache size 0 max 16384 2024-05-08T23:31:45.150 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-05-08T23:31:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1aa00a1290 msgr2=0x7f1aa00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1aa00a1290 0x7f1aa00a3750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4198880 tx=0x7f1a9c00b410 comp rx=0 tx=0).stop 2024-05-08T23:31:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 msgr2=0x7f1ab4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4197330 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40059d0 tx=0x7f1aa4004730 comp rx=0 tx=0).stop 2024-05-08T23:31:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 shutdown_connections 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f1aa00a1290 0x7f1aa00a3750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab419bc10 0x7f1ab419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ab4100060 0x7f1ab4197330 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 --2- 172.21.15.16:0/2322566344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ab4105160 0x7f1ab4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 >> 172.21.15.16:0/2322566344 conn(0x7f1ab4076f40 msgr2=0x7f1ab40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 shutdown_connections 2024-05-08T23:31:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:45.143+0000 7f1ab9105700 1 -- 172.21.15.16:0/2322566344 wait complete. 2024-05-08T23:31:45.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:44 smithi016 ceph-mon[42541]: from='client.18108 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:31:45.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:44 smithi016 ceph-mon[42541]: from='client.18114 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:31:45.156 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:44 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1224833525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:31:45.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:44 smithi189 ceph-mon[46919]: from='client.18108 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:31:45.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:44 smithi189 ceph-mon[46919]: from='client.18114 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:31:45.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:44 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1224833525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:31:45.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:44 smithi155 ceph-mon[47105]: from='client.18108 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:31:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:44 smithi155 ceph-mon[47105]: from='client.18114 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T23:31:45.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:44 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1224833525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:31:46.158 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-08T23:31:46.158 INFO:tasks.fwd_scrub:done joining 2024-05-08T23:31:46.159 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T23:31:46.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:31:46.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:45 smithi189 ceph-mon[46919]: pgmap v380: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-05-08T23:31:46.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:45 smithi016 ceph-mon[42541]: pgmap v380: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-05-08T23:31:46.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:45 smithi155 ceph-mon[47105]: pgmap v380: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-05-08T23:31:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 -- 172.21.15.16:0/3741976993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8105060 msgr2=0x7fcdf810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/3741976993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8105060 0x7fcdf810cbe0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fcdf000b0a0 tx=0x7fcdf000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 -- 172.21.15.16:0/3741976993 shutdown_connections 2024-05-08T23:31:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/3741976993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf8105060 0x7fcdf810cbe0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/3741976993 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf80fff40 0x7fcdf8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/3741976993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdf81008f0 0x7fcdf8104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 -- 172.21.15.16:0/3741976993 >> 172.21.15.16:0/3741976993 conn(0x7fcdf80fb7e0 msgr2=0x7fcdf80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 -- 172.21.15.16:0/3741976993 shutdown_connections 2024-05-08T23:31:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.314+0000 7fcdfc91f700 1 -- 172.21.15.16:0/3741976993 wait complete. 2024-05-08T23:31:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.315+0000 7fcdfc91f700 1 Processor -- start 2024-05-08T23:31:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 -- start start 2024-05-08T23:31:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 0x7fcdf8196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf8105060 0x7fcdf81967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdf819ab70 0x7fcdf819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf819b5f0 con 0x7fcdf80fff40 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf819b760 con 0x7fcdf8105060 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdfc91f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf819b920 con 0x7fcdf819ab70 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdf659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 0x7fcdf8196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdf659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 0x7fcdf8196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49002/0 (socket says 172.21.15.16:49002) 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdf659c700 1 -- 172.21.15.16:0/328413269 learned_addr learned my addr 172.21.15.16:0/328413269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdedd9b700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf8105060 0x7fcdf81967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.316+0000 7fcdf6d9d700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdf819ab70 0x7fcdf819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdf659c700 1 -- 172.21.15.16:0/328413269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdf819ab70 msgr2=0x7fcdf819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdf659c700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdf819ab70 0x7fcdf819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdf659c700 1 -- 172.21.15.16:0/328413269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf8105060 msgr2=0x7fcdf81967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdf659c700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf8105060 0x7fcdf81967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdf659c700 1 -- 172.21.15.16:0/328413269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0009d00 con 0x7fcdf80fff40 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdf659c700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 0x7fcdf8196290 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e9a0 tx=0x7fcde800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8009790 con 0x7fcdf80fff40 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcde800cbd0 con 0x7fcdf80fff40 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf81a2e80 con 0x7fcdf80fff40 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8018680 con 0x7fcdf80fff40 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.317+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf81a3300 con 0x7fcdf80fff40 2024-05-08T23:31:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.318+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf8108330 con 0x7fcdf80fff40 2024-05-08T23:31:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.319+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcde800cd40 con 0x7fcdf80fff40 2024-05-08T23:31:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.320+0000 7fcdeffff700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcde40a1290 0x7fcde40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.320+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fcde8014070 con 0x7fcdf80fff40 2024-05-08T23:31:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.320+0000 7fcdedd9b700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcde40a1290 0x7fcde40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.321+0000 7fcdedd9b700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcde40a1290 0x7fcde40a3750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcdf81977e0 tx=0x7fcde000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.324+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcde8096740 con 0x7fcdf80fff40 2024-05-08T23:31:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.542+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdf8197630 con 0x7fcdf80fff40 2024-05-08T23:31:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.543+0000 7fcdeffff700 1 -- 172.21.15.16:0/328413269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+25183 (secure 0 0 0) 0x7fcde8099d60 con 0x7fcdf80fff40 2024-05-08T23:31:46.544 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:31:46.545 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:31:13.733005+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcde40a1290 msgr2=0x7fcde40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcde40a1290 0x7fcde40a3750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcdf81977e0 tx=0x7fcde000b410 comp rx=0 tx=0).stop 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 msgr2=0x7fcdf8196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 0x7fcdf8196290 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e9a0 tx=0x7fcde800ecb0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 shutdown_connections 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fcde40a1290 0x7fcde40a3750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcdf819ab70 0x7fcdf819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf80fff40 0x7fcdf8196290 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 --2- 172.21.15.16:0/328413269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdf8105060 0x7fcdf81967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 >> 172.21.15.16:0/328413269 conn(0x7fcdf80fb7e0 msgr2=0x7fcdf80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 shutdown_connections 2024-05-08T23:31:46.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.546+0000 7fcdfc91f700 1 -- 172.21.15.16:0/328413269 wait complete. 2024-05-08T23:31:46.561 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:31:46.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 -- 172.21.15.16:0/2227597818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101b10 msgr2=0x7f77c0105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/2227597818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101b10 0x7f77c0105b50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009990 tx=0x7f77b4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 -- 172.21.15.16:0/2227597818 shutdown_connections 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/2227597818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0106280 0x7f77c010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/2227597818 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0101160 0x7f77c0101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/2227597818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101b10 0x7f77c0105b50 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 -- 172.21.15.16:0/2227597818 >> 172.21.15.16:0/2227597818 conn(0x7f77c00fc9e0 msgr2=0x7f77c00fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 -- 172.21.15.16:0/2227597818 shutdown_connections 2024-05-08T23:31:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.744+0000 7f77c6a1c700 1 -- 172.21.15.16:0/2227597818 wait complete. 2024-05-08T23:31:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 Processor -- start 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 -- start start 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 0x7f77c00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0106280 0x7f77c00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0073d20 0x7f77c01a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c01a5b80 con 0x7f77c0101160 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c01a5cf0 con 0x7f77c0106280 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c6a1c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c01a5eb0 con 0x7f77c0073d20 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77bffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 0x7f77c00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77bffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 0x7f77c00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60246/0 (socket says 172.21.15.16:60246) 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77bffff700 1 -- 172.21.15.16:0/1983371061 learned_addr learned my addr 172.21.15.16:0/1983371061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77bf7fe700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0106280 0x7f77c00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.745+0000 7f77c4fb9700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0073d20 0x7f77c01a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bffff700 1 -- 172.21.15.16:0/1983371061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0073d20 msgr2=0x7f77c01a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bffff700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0073d20 0x7f77c01a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bffff700 1 -- 172.21.15.16:0/1983371061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0106280 msgr2=0x7f77c00737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bffff700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0106280 0x7f77c00737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bffff700 1 -- 172.21.15.16:0/1983371061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40095f0 con 0x7f77c0101160 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bf7fe700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0106280 0x7f77c00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77c4fb9700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0073d20 0x7f77c01a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bffff700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 0x7f77c00732a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f77b000e9a0 tx=0x7f77b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0009790 con 0x7f77c0101160 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77b000cbd0 con 0x7f77c0101160 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c01a6190 con 0x7f77c0101160 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0018680 con 0x7f77c0101160 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.746+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c01a6670 con 0x7f77c0101160 2024-05-08T23:31:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.747+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c0109550 con 0x7f77c0101160 2024-05-08T23:31:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.748+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f77b000cd40 con 0x7f77c0101160 2024-05-08T23:31:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.749+0000 7f77bd7fa700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f77a80a1290 0x7f77a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.749+0000 7f77bf7fe700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f77a80a1290 0x7f77a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.749+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f77b0014070 con 0x7f77c0101160 2024-05-08T23:31:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.750+0000 7f77bf7fe700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f77a80a1290 0x7f77a80a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77c0074d50 tx=0x7f77b4005fd0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.753+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f77b0096680 con 0x7f77c0101160 2024-05-08T23:31:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.971+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77c0074780 con 0x7f77c0101160 2024-05-08T23:31:46.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.971+0000 7f77bd7fa700 1 -- 172.21.15.16:0/1983371061 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77b0099ca0 con 0x7f77c0101160 2024-05-08T23:31:46.974 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f77a80a1290 msgr2=0x7f77a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f77a80a1290 0x7f77a80a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77c0074d50 tx=0x7f77b4005fd0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 msgr2=0x7f77c00732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 0x7f77c00732a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f77b000e9a0 tx=0x7f77b000ed60 comp rx=0 tx=0).stop 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 shutdown_connections 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f77a80a1290 0x7f77a80a3750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77c0073d20 0x7f77c01a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c0101160 0x7f77c00732a0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 --2- 172.21.15.16:0/1983371061 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77c0106280 0x7f77c00737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:46.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 >> 172.21.15.16:0/1983371061 conn(0x7f77c00fc9e0 msgr2=0x7f77c00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:46.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 shutdown_connections 2024-05-08T23:31:46.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:46.974+0000 7f77c6a1c700 1 -- 172.21.15.16:0/1983371061 wait complete. 2024-05-08T23:31:46.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:31:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 -- 172.21.15.16:0/3590226635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520105160 msgr2=0x7ff52010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 --2- 172.21.15.16:0/3590226635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520105160 0x7ff52010cce0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7ff51800b0a0 tx=0x7ff51800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 -- 172.21.15.16:0/3590226635 shutdown_connections 2024-05-08T23:31:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 --2- 172.21.15.16:0/3590226635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520105160 0x7ff52010cce0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 --2- 172.21.15.16:0/3590226635 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff520100060 0x7ff520100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 --2- 172.21.15.16:0/3590226635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff520100a10 0x7ff520104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 -- 172.21.15.16:0/3590226635 >> 172.21.15.16:0/3590226635 conn(0x7ff520076f40 msgr2=0x7ff520077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 -- 172.21.15.16:0/3590226635 shutdown_connections 2024-05-08T23:31:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.129+0000 7ff524ed9700 1 -- 172.21.15.16:0/3590226635 wait complete. 2024-05-08T23:31:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 Processor -- start 2024-05-08T23:31:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 -- start start 2024-05-08T23:31:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 0x7ff520101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff520105160 0x7ff520102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff520102550 0x7ff5201a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201a5ca0 con 0x7ff520100060 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201a5e10 con 0x7ff520105160 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff524ed9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201a5fd0 con 0x7ff520102550 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.130+0000 7ff51e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 0x7ff520101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 0x7ff520101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60272/0 (socket says 172.21.15.16:60272) 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 -- 172.21.15.16:0/2106050991 learned_addr learned my addr 172.21.15.16:0/2106050991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51ed9d700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff520102550 0x7ff5201a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51dd9b700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff520105160 0x7ff520102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 -- 172.21.15.16:0/2106050991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff520102550 msgr2=0x7ff5201a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff520102550 0x7ff5201a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 -- 172.21.15.16:0/2106050991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff520105160 msgr2=0x7ff520102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff520105160 0x7ff520102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 -- 172.21.15.16:0/2106050991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518009d00 con 0x7ff520100060 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff51e59c700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 0x7ff520101ad0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514009b40 con 0x7ff520100060 2024-05-08T23:31:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5201a62b0 con 0x7ff520100060 2024-05-08T23:31:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff514009ca0 con 0x7ff520100060 2024-05-08T23:31:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400f620 con 0x7ff520100060 2024-05-08T23:31:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.131+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201a67c0 con 0x7ff520100060 2024-05-08T23:31:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.132+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520109440 con 0x7ff520100060 2024-05-08T23:31:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.134+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff514010460 con 0x7ff520100060 2024-05-08T23:31:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.134+0000 7ff50f7fe700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff5080a1290 0x7ff5080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.135+0000 7ff51dd9b700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff5080a1290 0x7ff5080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.135+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7ff5140cd4d0 con 0x7ff520100060 2024-05-08T23:31:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.135+0000 7ff51dd9b700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff5080a1290 0x7ff5080a3750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff510005950 tx=0x7ff5100058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.138+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff514096520 con 0x7ff520100060 2024-05-08T23:31:47.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:46 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/328413269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:31:47.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:46 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/328413269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:31:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.360+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff520067d20 con 0x7ff520100060 2024-05-08T23:31:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.360+0000 7ff50f7fe700 1 -- 172.21.15.16:0/2106050991 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff514099b40 con 0x7ff520100060 2024-05-08T23:31:47.361 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:31:47.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:46 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/328413269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff5080a1290 msgr2=0x7ff5080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff5080a1290 0x7ff5080a3750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff510005950 tx=0x7ff5100058e0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 msgr2=0x7ff520101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 0x7ff520101ad0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400dd10 comp rx=0 tx=0).stop 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 shutdown_connections 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff5080a1290 0x7ff5080a3750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff520102550 0x7ff5201a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff520100060 0x7ff520101ad0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 --2- 172.21.15.16:0/2106050991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff520105160 0x7ff520102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 >> 172.21.15.16:0/2106050991 conn(0x7ff520076f40 msgr2=0x7ff520068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 shutdown_connections 2024-05-08T23:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.363+0000 7ff524ed9700 1 -- 172.21.15.16:0/2106050991 wait complete. 2024-05-08T23:31:47.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:31:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/383455054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c8106300 msgr2=0x7fb6c810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/383455054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c8106300 0x7fb6c810de80 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000b0a0 tx=0x7fb6c000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/383455054 shutdown_connections 2024-05-08T23:31:47.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/383455054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c8106300 0x7fb6c810de80 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/383455054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c81011e0 0x7fb6c81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/383455054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c8101b90 0x7fb6c8105bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/383455054 >> 172.21.15.16:0/383455054 conn(0x7fb6c80fca80 msgr2=0x7fb6c80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/383455054 shutdown_connections 2024-05-08T23:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.518+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/383455054 wait complete. 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 Processor -- start 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 -- start start 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 0x7fb6c8197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8106300 0x7fb6c8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c819bd60 0x7fb6c819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c819c7e0 con 0x7fb6c81011e0 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c819c950 con 0x7fb6c8106300 2024-05-08T23:31:47.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cf4f7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c819cb10 con 0x7fb6c819bd60 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.519+0000 7fb6cd293700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 0x7fb6c8197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 0x7fb6c8197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60288/0 (socket says 172.21.15.16:60288) 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 -- 172.21.15.16:0/3918161151 learned_addr learned my addr 172.21.15.16:0/3918161151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cca92700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8106300 0x7fb6c8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cda94700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c819bd60 0x7fb6c819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 -- 172.21.15.16:0/3918161151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c819bd60 msgr2=0x7fb6c819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c819bd60 0x7fb6c819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 -- 172.21.15.16:0/3918161151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8106300 msgr2=0x7fb6c8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8106300 0x7fb6c8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 -- 172.21.15.16:0/3918161151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0009d00 con 0x7fb6c81011e0 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cd293700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 0x7fb6c8197540 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400e9a0 tx=0x7fb6c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4009790 con 0x7fb6c81011e0 2024-05-08T23:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c81a4070 con 0x7fb6c81011e0 2024-05-08T23:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6c4010430 con 0x7fb6c81011e0 2024-05-08T23:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400cea0 con 0x7fb6c81011e0 2024-05-08T23:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.520+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c81a45a0 con 0x7fb6c81011e0 2024-05-08T23:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.521+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c81095d0 con 0x7fb6c81011e0 2024-05-08T23:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.522+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb6c40105a0 con 0x7fb6c81011e0 2024-05-08T23:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.523+0000 7fb6be7fc700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb6b40a1290 0x7fb6b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.523+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fb6c4014070 con 0x7fb6c81011e0 2024-05-08T23:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.523+0000 7fb6cca92700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb6b40a1290 0x7fb6b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.524+0000 7fb6cca92700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb6b40a1290 0x7fb6b40a3750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6c81001e0 tx=0x7fb6b800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.528+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb6c4052d60 con 0x7fb6c81011e0 2024-05-08T23:31:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.744+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6c81987f0 con 0x7fb6c81011e0 2024-05-08T23:31:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.745+0000 7fb6be7fc700 1 -- 172.21.15.16:0/3918161151 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6c4096320 con 0x7fb6c81011e0 2024-05-08T23:31:47.746 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb6b40a1290 msgr2=0x7fb6b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb6b40a1290 0x7fb6b40a3750 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6c81001e0 tx=0x7fb6b800b410 comp rx=0 tx=0).stop 2024-05-08T23:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 msgr2=0x7fb6c8197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 0x7fb6c8197540 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400e9a0 tx=0x7fb6c400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 shutdown_connections 2024-05-08T23:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fb6b40a1290 0x7fb6b40a3750 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c819bd60 0x7fb6c819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c81011e0 0x7fb6c8197540 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 --2- 172.21.15.16:0/3918161151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6c8106300 0x7fb6c8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 >> 172.21.15.16:0/3918161151 conn(0x7fb6c80fca80 msgr2=0x7fb6c80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.747+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 shutdown_connections 2024-05-08T23:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.748+0000 7fb6cf4f7700 1 -- 172.21.15.16:0/3918161151 wait complete. 2024-05-08T23:31:47.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:31:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 -- 172.21.15.16:0/1752704523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc106280 msgr2=0x7f01fc10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 --2- 172.21.15.16:0/1752704523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc106280 0x7f01fc10de00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b0a0 tx=0x7f01f400b3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 -- 172.21.15.16:0/1752704523 shutdown_connections 2024-05-08T23:31:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 --2- 172.21.15.16:0/1752704523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc106280 0x7f01fc10de00 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 --2- 172.21.15.16:0/1752704523 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc101160 0x7f01fc101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 --2- 172.21.15.16:0/1752704523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101b10 0x7f01fc105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 -- 172.21.15.16:0/1752704523 >> 172.21.15.16:0/1752704523 conn(0x7f01fc0fc9c0 msgr2=0x7f01fc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 -- 172.21.15.16:0/1752704523 shutdown_connections 2024-05-08T23:31:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.903+0000 7f02040ea700 1 -- 172.21.15.16:0/1752704523 wait complete. 2024-05-08T23:31:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 Processor -- start 2024-05-08T23:31:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 -- start start 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 0x7f01fc197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc106280 0x7f01fc1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 0x7f01fc19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19c790 con 0x7f01fc19bd10 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19c900 con 0x7f01fc106280 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f02040ea700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc19cac0 con 0x7f01fc101160 2024-05-08T23:31:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f0202687700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 0x7f01fc19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f0201e86700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 0x7f01fc197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f0201685700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc106280 0x7f01fc1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f0201e86700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 0x7f01fc197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.16:58164/0 (socket says 172.21.15.16:58164) 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f0202687700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 0x7f01fc19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60306/0 (socket says 172.21.15.16:60306) 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.904+0000 7f0201e86700 1 -- 172.21.15.16:0/602315679 learned_addr learned my addr 172.21.15.16:0/602315679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0202687700 1 -- 172.21.15.16:0/602315679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 msgr2=0x7f01fc197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0202687700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 0x7f01fc197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0202687700 1 -- 172.21.15.16:0/602315679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc106280 msgr2=0x7f01fc1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0202687700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc106280 0x7f01fc1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0202687700 1 -- 172.21.15.16:0/602315679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4009d00 con 0x7f01fc19bd10 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0201e86700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 0x7f01fc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0201685700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc106280 0x7f01fc1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:31:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f0202687700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 0x7f01fc19c190 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b070 tx=0x7f01f4004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400e040 con 0x7f01fc19bd10 2024-05-08T23:31:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01f400be60 con 0x7f01fc19bd10 2024-05-08T23:31:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc1a3fc0 con 0x7f01fc19bd10 2024-05-08T23:31:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f401be50 con 0x7f01fc19bd10 2024-05-08T23:31:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.905+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc1a4440 con 0x7f01fc19bd10 2024-05-08T23:31:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.906+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc109550 con 0x7f01fc19bd10 2024-05-08T23:31:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.908+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f01f401b550 con 0x7f01fc19bd10 2024-05-08T23:31:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.909+0000 7f01f2ffd700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01e80a1290 0x7f01e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.909+0000 7f0201e86700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01e80a1290 0x7f01e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.909+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f01f40ce980 con 0x7f01fc19bd10 2024-05-08T23:31:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.909+0000 7f0201e86700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01e80a1290 0x7f01e80a3750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01f8011550 tx=0x7f01f8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:47.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:47.912+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f01f4097ac0 con 0x7f01fc19bd10 2024-05-08T23:31:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:47 smithi016 ceph-mon[42541]: pgmap v381: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:31:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1983371061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2106050991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:47 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3918161151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.130+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f01fc198730 con 0x7f01fc19bd10 2024-05-08T23:31:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.130+0000 7f01f2ffd700 1 -- 172.21.15.16:0/602315679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) v1 ==== 103+0+11 (secure 0 0 0) 0x7f01f409b0e0 con 0x7f01fc19bd10 2024-05-08T23:31:48.131 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:31:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01e80a1290 msgr2=0x7f01e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01e80a1290 0x7f01e80a3750 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f01f8011550 tx=0x7f01f8009260 comp rx=0 tx=0).stop 2024-05-08T23:31:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 msgr2=0x7f01fc19c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 0x7f01fc19c190 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b070 tx=0x7f01f4004da0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 shutdown_connections 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f01e80a1290 0x7f01e80a3750 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc19bd10 0x7f01fc19c190 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01fc101160 0x7f01fc197480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 --2- 172.21.15.16:0/602315679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01fc106280 0x7f01fc1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 >> 172.21.15.16:0/602315679 conn(0x7f01fc0fc9c0 msgr2=0x7f01fc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 shutdown_connections 2024-05-08T23:31:48.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.133+0000 7f02040ea700 1 -- 172.21.15.16:0/602315679 wait complete. 2024-05-08T23:31:48.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T23:31:48.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:47 smithi189 ceph-mon[46919]: pgmap v381: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:31:48.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:47 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1983371061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:47 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2106050991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:47 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3918161151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:47 smithi155 ceph-mon[47105]: pgmap v381: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:31:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:47 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1983371061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:47 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2106050991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:47 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3918161151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 -- 172.21.15.16:0/907581555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2988100920 msgr2=0x7f2988104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 --2- 172.21.15.16:0/907581555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2988100920 0x7f2988104960 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f297c009990 tx=0x7f297c009ca0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 -- 172.21.15.16:0/907581555 shutdown_connections 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 --2- 172.21.15.16:0/907581555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 0x7f298810cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 --2- 172.21.15.16:0/907581555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29880fff70 0x7f2988100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 --2- 172.21.15.16:0/907581555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2988100920 0x7f2988104960 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 -- 172.21.15.16:0/907581555 >> 172.21.15.16:0/907581555 conn(0x7f29880fb7f0 msgr2=0x7f29880fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 -- 172.21.15.16:0/907581555 shutdown_connections 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 -- 172.21.15.16:0/907581555 wait complete. 2024-05-08T23:31:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.284+0000 7f298d670700 1 Processor -- start 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 -- start start 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29880fff70 0x7f29881993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 0x7f2988199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 0x7f298819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819e730 con 0x7f298819dcb0 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819e8a0 con 0x7f2988105090 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f298d670700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298819ea60 con 0x7f29880fff70 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 0x7f298819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 0x7f298819e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60328/0 (socket says 172.21.15.16:60328) 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 -- 172.21.15.16:0/3126560836 learned_addr learned my addr 172.21.15.16:0/3126560836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f2986ffd700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29880fff70 0x7f29881993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29867fc700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 0x7f2988199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 -- 172.21.15.16:0/3126560836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29880fff70 msgr2=0x7f29881993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29880fff70 0x7f29881993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 -- 172.21.15.16:0/3126560836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 msgr2=0x7f2988199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 0x7f2988199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29877fe700 1 -- 172.21.15.16:0/3126560836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0095f0 con 0x7f298819dcb0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.285+0000 7f29867fc700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 0x7f2988199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.286+0000 7f29877fe700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 0x7f298819e130 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2980007ee0 tx=0x7f298000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.286+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980017070 con 0x7f298819dcb0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.286+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f298000f040 con 0x7f298819dcb0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.286+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980005710 con 0x7f298819dcb0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.286+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29881a5fc0 con 0x7f298819dcb0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.286+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29881a6450 con 0x7f298819dcb0 2024-05-08T23:31:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.287+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29881030d0 con 0x7f298819dcb0 2024-05-08T23:31:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.288+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2980005870 con 0x7f298819dcb0 2024-05-08T23:31:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.289+0000 7f296ffff700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29700a1310 0x7f29700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:31:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.289+0000 7f2986ffd700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29700a1310 0x7f29700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:31:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.289+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f29800ccdc0 con 0x7f298819dcb0 2024-05-08T23:31:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.290+0000 7f2986ffd700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29700a1310 0x7f29700a37d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2978009c60 tx=0x7f2978009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:31:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.293+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2980095e70 con 0x7f298819dcb0 2024-05-08T23:31:48.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.579+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f2988101440 con 0x7f298819dcb0 2024-05-08T23:31:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.587+0000 7f296ffff700 1 -- 172.21.15.16:0/3126560836 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733937 (secure 0 0 0) 0x7f2980099490 con 0x7f298819dcb0 2024-05-08T23:31:48.598 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 25310, 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 198235485 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 198235485 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/198235485", 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-05-08T23:19:37.918037+0000", 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540138322906710015, 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 25333, 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-05-08T23:31:48.600 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540138322906710015, 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.603 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.603 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.605 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.605 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.606 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.606 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.606 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-05-08T23:31:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-05-08T23:31:48.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-05-08T23:31:48.622 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-05-08T23:31:48.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:31:48.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:48.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.634 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.639 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-05-08T23:31:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T23:31:48.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T23:31:48.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.643 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:48.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-05-08T23:31:48.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T23:31:48.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T23:31:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-05-08T23:31:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-05-08T23:31:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-05-08T23:31:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-05-08T23:31:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-05-08T23:31:48.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.690 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.691 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-05-08T23:31:48.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-05-08T23:31:48.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.701 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.702 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-05-08T23:31:48.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.708 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.709 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-05-08T23:31:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T23:31:48.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T23:31:48.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T23:31:48.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.741 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T23:31:48.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.762 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T23:31:48.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T23:31:48.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.764 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T23:31:48.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T23:31:48.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.766 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T23:31:48.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T23:31:48.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T23:31:48.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:48.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.770 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-05-08T23:31:48.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-05-08T23:31:48.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-05-08T23:31:48.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.777 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-05-08T23:31:48.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-05-08T23:31:48.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-05-08T23:31:48.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.779 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-08T23:31:48.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-08T23:31:48.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.782 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.783 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.783 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.784 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.784 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.784 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.784 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.784 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.785 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.788 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-05-08T23:31:48.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.790 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.791 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:48.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-08T23:31:48.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-05-08T23:31:48.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.795 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-08T23:31:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.801 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.816 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.818 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.819 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-05-08T23:31:48.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-05-08T23:31:48.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.837 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.838 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.842 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.848 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.852 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.853 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.860 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-05-08T23:31:48.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.867 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.868 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.872 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.873 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.885 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-05-08T23:31:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.896 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.900 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:48.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.908 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T23:31:48.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.918 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T23:31:48.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-05-08T23:31:48.934 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-08T23:31:48.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.939 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:48.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-08T23:31:48.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:48.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.954 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.964 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-05-08T23:31:48.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.967 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.968 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.969 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-05-08T23:31:48.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:48.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.994 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:48.997 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.000 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.004 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-05-08T23:31:49.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.010 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.012 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-05-08T23:31:49.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.014 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-05-08T23:31:49.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-05-08T23:31:49.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-05-08T23:31:49.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.031 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.042 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.043 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.046 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-05-08T23:31:49.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-05-08T23:31:49.051 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.055 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 25336, 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540138322906710015, 2024-05-08T23:31:49.063 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.067 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.068 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-05-08T23:31:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T23:31:49.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T23:31:49.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-05-08T23:31:49.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-08T23:31:49.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.090 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T23:31:49.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T23:31:49.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T23:31:49.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T23:31:49.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-05-08T23:31:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-05-08T23:31:49.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.134 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-05-08T23:31:49.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T23:31:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T23:31:49.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-05-08T23:31:49.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T23:31:49.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T23:31:49.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-05-08T23:31:49.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-05-08T23:31:49.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T23:31:49.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T23:31:49.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T23:31:49.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T23:31:49.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.201 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.202 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-05-08T23:31:49.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.234 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.237 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T23:31:49.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T23:31:49.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.248 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.249 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.250 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.255 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.260 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.273 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.276 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.277 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.287 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.291 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-05-08T23:31:49.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.306 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.308 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.312 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.313 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.326 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.334 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.336 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.337 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.343 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.346 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-05-08T23:31:49.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-05-08T23:31:49.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-05-08T23:31:49.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.356 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-05-08T23:31:49.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-05-08T23:31:49.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-08T23:31:49.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.369 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.376 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.382 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:49.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.386 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.390 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-05-08T23:31:49.392 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.393 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.400 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.402 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-05-08T23:31:49.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-05-08T23:31:49.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-05-08T23:31:49.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-05-08T23:31:49.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.410 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.412 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-05-08T23:31:49.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.416 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.422 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-05-08T23:31:49.424 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.425 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.428 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T23:31:49.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-05-08T23:31:49.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-05-08T23:31:49.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-05-08T23:31:49.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.446 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.458 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.461 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T23:31:49.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.466 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.467 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.471 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.472 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T23:31:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-05-08T23:31:49.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.483 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-05-08T23:31:49.491 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.496 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-05-08T23:31:49.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-05-08T23:31:49.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-05-08T23:31:49.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-05-08T23:31:49.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-05-08T23:31:49.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T23:31:49.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T23:31:49.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-05-08T23:31:49.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T23:31:49.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T23:31:49.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.515 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.520 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-05-08T23:31:49.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-05-08T23:31:49.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-05-08T23:31:49.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.530 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.532 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.533 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.534 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.535 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.536 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-05-08T23:31:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.565 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-05-08T23:31:49.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-05-08T23:31:49.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-05-08T23:31:49.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-05-08T23:31:49.574 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-05-08T23:31:49.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-05-08T23:31:49.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-05-08T23:31:49.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-05-08T23:31:49.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.585 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T23:31:49.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.594 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T23:31:49.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T23:31:49.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T23:31:49.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T23:31:49.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T23:31:49.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T23:31:49.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T23:31:49.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T23:31:49.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T23:31:49.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T23:31:49.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-05-08T23:31:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T23:31:49.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-05-08T23:31:49.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-05-08T23:31:49.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-05-08T23:31:49.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.668 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-05-08T23:31:49.677 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.678 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-05-08T23:31:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T23:31:49.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.691 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-05-08T23:31:49.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-05-08T23:31:49.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-05-08T23:31:49.702 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.708 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.709 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.721 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.723 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-05-08T23:31:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-05-08T23:31:49.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-05-08T23:31:49.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.738 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-05-08T23:31:49.744 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.745 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.748 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.750 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.753 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.755 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.769 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.770 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-05-08T23:31:49.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-08T23:31:49.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.782 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-05-08T23:31:49.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-05-08T23:31:49.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T23:31:49.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-05-08T23:31:49.800 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-05-08T23:31:49.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.802 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.803 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-05-08T23:31:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-05-08T23:31:49.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.816 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-05-08T23:31:49.820 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.821 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.829 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.832 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.833 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.844 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.845 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-05-08T23:31:49.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-05-08T23:31:49.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.848 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.849 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.851 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.855 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.857 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.865 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.875 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.877 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T23:31:49.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.902 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.904 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.905 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.907 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-05-08T23:31:49.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-05-08T23:31:49.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-05-08T23:31:49.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.912 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-05-08T23:31:49.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-08T23:31:49.924 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-08T23:31:49.925 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-05-08T23:31:49.926 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 84, 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-05-08T23:31:49.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2275364120 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 618467651 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1689502560 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1656562520 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:49.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 404642958 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2698204521 2024-05-08T23:31:49.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.932 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:49.932 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-05-08T23:31:49.932 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:49.933 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:48 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/602315679' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:31:49.933 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:48 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3126560836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T23:31:49.934 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:48 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/602315679' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:31:49.934 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:48 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3126560836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T23:31:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29700a1310 msgr2=0x7f29700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29700a1310 0x7f29700a37d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2978009c60 tx=0x7f2978009260 comp rx=0 tx=0).stop 2024-05-08T23:31:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 msgr2=0x7f298819e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:31:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 0x7f298819e130 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2980007ee0 tx=0x7f298000d3b0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 shutdown_connections 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f29700a1310 0x7f29700a37d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f298819dcb0 0x7f298819e130 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29880fff70 0x7f29881993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 --2- 172.21.15.16:0/3126560836 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2988105090 0x7f2988199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 >> 172.21.15.16:0/3126560836 conn(0x7f29880fb7f0 msgr2=0x7f29880fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 shutdown_connections 2024-05-08T23:31:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:31:48.593+0000 7f298d670700 1 -- 172.21.15.16:0/3126560836 wait complete. 2024-05-08T23:31:49.940 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T23:31:49.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T23:31:49.941 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:48 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/602315679' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:31:49.941 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:48 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3126560836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T23:31:50.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:49 smithi189 ceph-mon[46919]: pgmap v382: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:31:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:31:50.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:49 smithi155 ceph-mon[47105]: pgmap v382: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:31:50.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:49 smithi016 ceph-mon[42541]: pgmap v382: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-05-08T23:31:51.230 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:51.230 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:31:51.230 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001882253, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000433671, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:51.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 238, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 228, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1427792, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 130400, 2024-05-08T23:31:51.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.046989471, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.016071297, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.033552315, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005100853, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 227, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012012716, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000052919 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:51.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1427792, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 130400 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 177, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3232, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183488, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.016665148, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.010823527, 2024-05-08T23:31:51.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002770107, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 175, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.018592780, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000106244 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183488 2024-05-08T23:31:51.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:31:51.236 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:31:51.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148967, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 29, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:31:51.238 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-05-08T23:31:51.239 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 240, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1394178, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 240, 2024-05-08T23:31:51.240 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 240, 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1394178, 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:51.241 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:51.597 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-08T23:31:51.597 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-08T23:31:51.597 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-08T23:31:51.597 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-08T23:31:51.597 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-08T23:31:51.597 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T23:31:51.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T23:31:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:31:52.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:51 smithi189 ceph-mon[46919]: pgmap v383: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-05-08T23:31:52.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:51 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:31:52.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:51 smithi016 ceph-mon[42541]: pgmap v383: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-05-08T23:31:52.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:51 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:31:52.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:51 smithi155 ceph-mon[47105]: pgmap v383: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 54 KiB/s wr, 15 op/s 2024-05-08T23:31:52.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:51 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T23:31:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:52.963 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:31:52.963 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001842020, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000390830, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 245, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 233, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1431520, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 132624, 2024-05-08T23:31:52.965 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.045646112, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013064438, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.033229926, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005840121, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 233, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.014711083, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000063137 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:52.966 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1431520, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 132624 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 177, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183392, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.015180748, 2024-05-08T23:31:52.967 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.010195860, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002098977, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 174, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.021579122, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000124017 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:31:52.968 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183392 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:31:52.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:31:52.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:31:52.971 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148955, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-08T23:31:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 243, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1394568, 2024-05-08T23:31:52.973 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 243, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 243, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1394568, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:52.974 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:53.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:52 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:31:53.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:52 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:31:53.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:52 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:31:53.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:52 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:31:53.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:52 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:31:53.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:52 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:31:53.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:52 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T23:31:53.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:52 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T23:31:53.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:52 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' 2024-05-08T23:31:53.333 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T23:31:53.333 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T23:31:53.333 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-08T23:31:53.333 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-08T23:31:53.333 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-08T23:31:53.333 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T23:31:53.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T23:31:53.591 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:31:54.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:53 smithi016 ceph-mon[42541]: pgmap v384: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:31:54.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:53 smithi155 ceph-mon[47105]: pgmap v384: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:31:54.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:53 smithi189 ceph-mon[46919]: pgmap v384: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 10 op/s 2024-05-08T23:31:54.846 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:54.846 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:31:54.846 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 239, 2024-05-08T23:31:54.846 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 229, 2024-05-08T23:31:54.846 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1428064, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 130992, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.049698521, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.016503744, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.033767345, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005450122, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 228, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015130778, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000066363 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.847 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1428064, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 130992 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 176, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183216, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-08T23:31:54.848 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.016365514, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.010917275, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002371320, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 173, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.020147528, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000116459 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.849 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183216 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2912, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001131947, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000059757, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001036807, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:54.850 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001090513, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000363504 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-05-08T23:31:54.851 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:31:54.852 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-08T23:31:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148977, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-08T23:31:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-08T23:31:54.855 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 241, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1394308, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 241, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 241, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1394308, 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-08T23:31:54.856 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:54.857 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:54.857 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:54.857 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:54.857 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:54.857 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:55.266 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-08T23:31:55.267 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-08T23:31:55.267 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-08T23:31:55.267 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-08T23:31:55.267 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-08T23:31:55.267 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T23:31:55.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T23:31:55.522 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T23:31:56.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:55 smithi155 ceph-mon[47105]: pgmap v385: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-05-08T23:31:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:55 smithi016 ceph-mon[42541]: pgmap v385: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-05-08T23:31:56.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:55 smithi189 ceph-mon[46919]: pgmap v385: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-05-08T23:31:56.527 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2912, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002446581, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000057317, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001009850, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:56.528 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001213216, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000404405 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.529 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 179, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 183744, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.016981084, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.011325909, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002516300, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 176, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.022489754, 2024-05-08T23:31:56.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000127782 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 183744 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 243, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 233, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1429152, 2024-05-08T23:31:56.531 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 133360, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.048762636, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.015794714, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.035851984, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005252054, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 232, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012847662, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000055377 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:56.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1429152, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 133360 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:31:56.533 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-05-08T23:31:56.534 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:31:56.535 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 150008, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-05-08T23:31:56.536 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-05-08T23:31:56.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 245, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1394828, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 245, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 245, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1394828, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-05-08T23:31:56.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-05-08T23:31:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-05-08T23:31:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:56.539 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-05-08T23:31:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-05-08T23:31:56.900 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T23:31:56.900 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T23:31:56.900 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-08T23:31:56.900 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-08T23:31:56.900 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-08T23:31:56.900 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T23:31:56.900 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T23:31:57.149 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:31:58.159 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-08T23:31:58.159 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:31:58.159 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 3188, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 3025, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1025308, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 15152327, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 182, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 4, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.347188444, 2024-05-08T23:31:58.160 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.098422649, 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.128708248, 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.086842944, 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3023, 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.785506382, 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000259843 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:58.161 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1434, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000283095, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000197 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 43152, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 3376 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2952, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 5360, 2024-05-08T23:31:58.162 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 968881, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 9556668, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 13, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.372716271, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.135545648, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.161159497, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.073625760, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5356, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.044612208, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000195035 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.163 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5847, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001329953, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000227 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 52176, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 500256 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 2662, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2040, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 2467981, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 4996629, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 19, 2024-05-08T23:31:58.164 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 6, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.315341594, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.089812543, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.133352162, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.094805924, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2040, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.375320656, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000183980 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2769, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000593592, 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000214 2024-05-08T23:31:58.165 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1913744, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 279104 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5042, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.664833651, 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000330193 2024-05-08T23:31:58.166 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 15, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000384770, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000025651 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 807, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 807, 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-05-08T23:31:58.167 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 807, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.555827549, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004406229 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 12, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 41, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 1, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 40, 2024-05-08T23:31:58.168 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 39, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 2, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 3, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 2, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 898, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 888, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 360, 2024-05-08T23:31:58.169 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 299, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 4, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 234, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 299, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:31:58.170 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 193, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 22, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 171, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 106, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 622, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 1206, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 599, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 638, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 613, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 3, 2024-05-08T23:31:58.171 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 26 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 5, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 2, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 3, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 1, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-05-08T23:31:58.172 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 22, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-05-08T23:31:58.173 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 2186, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2186, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 6, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 6, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 28234905, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 0, 2024-05-08T23:31:58.174 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2122, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.979303573, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001875260 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 0 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 35, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 58, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 23, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 20, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 20, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 0, 2024-05-08T23:31:58.175 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 41, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 42, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 1, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 3, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 20, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 17, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 54708, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 2063, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 1, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 807, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 36, 2024-05-08T23:31:58.176 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 2, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.004034377, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000806875 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 146, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.031800593, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000217812 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.177 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000502569, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000251284 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.178 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.179 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 12, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002286534, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000190544 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8, 2024-05-08T23:31:58.180 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.013487438, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001685929 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 12, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.055814739, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004651228 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-05-08T23:31:58.181 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 599, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3.375863760, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.005635832 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.004217540, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002108770 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002581607, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.001290803 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.182 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.009889938, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004944969 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.183 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 6, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.029857915, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.004976319 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.001656840, 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000828420 2024-05-08T23:31:58.184 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.185 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 8, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.021417336, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002677167 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.002416363, 2024-05-08T23:31:58.186 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.002416363 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 6, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 4, 2024-05-08T23:31:58.187 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 2, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 112, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 56, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 725, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:31:58.188 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:31:58.189 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:31:58.190 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 314770, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 124, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:31:58.191 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9648, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 202, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 182868, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 1045, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-05-08T23:31:58.192 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 3680, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 24060474, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 37, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 3680, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3680, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 23.881926176, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.006489653 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.193 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3680, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3717 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 3680, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 344, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 3336, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 0, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 11, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 23, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 2123, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 55, 2024-05-08T23:31:58.194 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 519, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 321, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 22, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-05-08T23:31:58.195 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 599, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-05-08T23:31:58.196 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 98, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 22, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 12, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 12, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 38, 2024-05-08T23:31:58.197 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 0, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 6 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 1, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-05-08T23:31:58.198 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 2, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 1, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 1, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 8798, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 3595844, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 8798, 2024-05-08T23:31:58.199 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 8798, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3595844, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:31:58.200 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3680, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 24060631, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3680, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 24060631, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-05-08T23:31:58.201 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3680, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 3680, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3680, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3680, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.202 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 1, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 157, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 1, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 1, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 157, 2024-05-08T23:31:58.203 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x5607aa5560a0": { 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 2186, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 24040601, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 2186, 2024-05-08T23:31:58.204 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 2122, 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 24040601, 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:58.205 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-08T23:31:58.206 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:57 smithi155 ceph-mon[47105]: pgmap v386: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 6 op/s 2024-05-08T23:31:58.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:57 smithi016 ceph-mon[42541]: pgmap v386: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 6 op/s 2024-05-08T23:31:58.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:57 smithi189 ceph-mon[46919]: pgmap v386: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 6 op/s 2024-05-08T23:31:58.511 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-08T23:31:58.511 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-08T23:31:58.511 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-08T23:31:58.511 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-05-08T23:31:58.511 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-05-08T23:31:58.511 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T23:31:58.511 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T23:31:58.732 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:31:59.672 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 267, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 256, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1460882, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 137517, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 5, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.049377244, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.015547461, 2024-05-08T23:31:59.673 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.032912177, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.006385647, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 255, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.015914825, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000062411 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1460048, 2024-05-08T23:31:59.674 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 137200 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 748, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 923, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 185786, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 691567, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.081511236, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.028482455, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.017141171, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.026130806, 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:59.675 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 921, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.045986465, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000049931 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-08T23:31:59.676 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 460800 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1658, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1660, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 498020, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 515696, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-05-08T23:31:59.677 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.169838721, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.059658874, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.041992832, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.051432525, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1658, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.072834616, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000043929 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.678 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-05-08T23:31:59.679 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 4017, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.116232326, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000028935 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-05-08T23:31:59.680 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-05-08T23:31:59.681 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 1, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-05-08T23:31:59.682 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-05-08T23:31:59.683 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 0, 2024-05-08T23:31:59.684 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:31:59.685 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-05-08T23:31:59.686 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-05-08T23:31:59.687 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-05-08T23:31:59.688 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 7, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 1, 2024-05-08T23:31:59.689 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 4196801, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 4196660, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 7 2024-05-08T23:31:59.690 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 3, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 3, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 1, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 1, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-05-08T23:31:59.691 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 47504, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.692 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-05-08T23:31:59.693 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.694 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-05-08T23:31:59.695 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.696 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.697 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-05-08T23:31:59.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.698 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.698 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.698 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-05-08T23:31:59.698 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.699 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.699 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.699 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-05-08T23:31:59.699 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.699 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.700 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.700 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-05-08T23:31:59.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.700 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.700 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.701 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.702 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.702 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-05-08T23:31:59.702 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.702 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.702 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.702 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.703 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.704 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.705 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-05-08T23:31:59.706 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.707 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.708 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 4, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 2, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 4, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3775, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-05-08T23:31:59.709 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:31:59.710 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:31:59.711 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-05-08T23:31:59.712 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 189393, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 51, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:31:59.713 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9648, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 202, 2024-05-08T23:31:59.714 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 8320, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 13, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-05-08T23:31:59.715 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 2401, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 21, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 3, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 2401, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2401, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.146996294, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000477716 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 2401, 2024-05-08T23:31:59.716 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 2402 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 2401, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 2401, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 1566, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 833, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-05-08T23:31:59.717 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 1, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-05-08T23:31:59.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-05-08T23:31:59.719 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 98, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 17, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 5, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 5, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-05-08T23:31:59.720 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:31:59.721 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 2669, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1887890, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 2669, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:31:59.722 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 2669, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1887890, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:31:59.723 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 2401, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 2502, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 2401, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2502, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:59.724 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 2401, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 2401, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 2401, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 2401, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.725 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:59.726 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x55ad1d292500": { 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:31:59.727 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:31:59.728 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-08T23:31:59.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:31:59 smithi155 ceph-mon[47105]: pgmap v387: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 6 op/s 2024-05-08T23:32:00.071 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-08T23:32:00.071 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-08T23:32:00.071 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-08T23:32:00.071 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:32:00.071 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:32:00.072 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T23:32:00.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T23:32:00.294 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:32:00.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:31:59 smithi016 ceph-mon[42541]: pgmap v387: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 6 op/s 2024-05-08T23:32:00.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:31:59 smithi189 ceph-mon[46919]: pgmap v387: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 6 op/s 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1835, 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1837, 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 12001504, 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 570695, 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-05-08T23:32:01.239 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.169797631, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.059745724, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.046049266, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.046752550, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1835, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.068302565, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000037222 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.240 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 533, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 522, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 5722082, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 219871, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.075888566, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.021430876, 2024-05-08T23:32:01.241 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.044470376, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.014786576, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 521, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.024505578, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000047035 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.242 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1459936, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 136768 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 1023, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 1197, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 8645709, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 776912, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 7, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 5, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.101272973, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.027473224, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.033783773, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.030654909, 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:01.243 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 1195, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.046620986, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000039013 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 460624 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.244 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 1, 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-MDSRank": { 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 5354, 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.121871087, 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000022762 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "finisher-PurgeQueue": { 2024-05-08T23:32:01.245 INFO:teuthology.orchestra.run.smithi155.stdout: "queue_len": 0, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "complete_latency": { 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "mds": { 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "request": 0, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "reply": 0, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "reply_latency": { 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.246 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_reply": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "forward": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_complete": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_fetch_keys": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_commit": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_split": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_merge": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes": 31, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_top": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_bottom": 31, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pin_tail": 0, 2024-05-08T23:32:01.247 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_pinned": 31, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_expired": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "inodes_with_caps": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "caps": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "subtrees": 2, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_hit": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_forward": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_discover": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_remote_ino": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "traverse_lock": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "load_cent": 0, 2024-05-08T23:32:01.248 INFO:teuthology.orchestra.run.smithi155.stdout: "q": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "exported": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "exported_inodes": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "imported": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "imported_inodes": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_dir_fetch": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "openino_peer_discover": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rfiles": 4, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rbytes": 234, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "root_rsnaps": 6, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_set_tag": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:32:01.249 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "scrub_file_inodes": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_cap_release": 0, 2024-05-08T23:32:01.250 INFO:teuthology.orchestra.run.smithi155.stdout: "process_request_cap_release": 0 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_cache": { 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_update_receipt": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_try_discover": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_send_discover": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "dir_handle_discover": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_delayed": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_created": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_enqueued": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_reintegrated": 0, 2024-05-08T23:32:01.251 INFO:teuthology.orchestra.run.smithi155.stdout: "strays_migrated": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_processing": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_started": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "recovery_completed": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_exportdir": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_flush": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_fragstats": 0, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "ireq_inodestats": 0 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_log": { 2024-05-08T23:32:01.252 INFO:teuthology.orchestra.run.smithi155.stdout: "evadd": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "evex": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "evtrm": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "ev": 2192, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "evexg": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "evexd": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "segadd": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "segex": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "segtrm": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "seg": 6, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "segexg": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "segexd": 0, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "expos": 4194304, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "wrpos": 28243651, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "rdpos": 28239512, 2024-05-08T23:32:01.253 INFO:teuthology.orchestra.run.smithi155.stdout: "jlat": { 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "replayed": 2192 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_mem": { 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "ino": 26, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "ino+": 27, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "ino-": 1, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "dir": 11, 2024-05-08T23:32:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: "dir+": 11, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "dir-": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "dn": 31, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "dn+": 31, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "dn-": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "cap": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "cap+": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "cap-": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "rss": 47924, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "heap": 207156 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_server": { 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_client_request": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "dispatch_server_request": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_request": 0, 2024-05-08T23:32:01.255 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_client_session": 0, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "handle_peer_request": 0, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "req_create_latency": { 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getattr_latency": { 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getfilelock_latency": { 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.256 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "req_link_latency": { 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookup_latency": { 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookuphash_latency": { 2024-05-08T23:32:01.257 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupino_latency": { 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupname_latency": { 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.258 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupparent_latency": { 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lookupsnap_latency": { 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.259 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "req_lssnap_latency": { 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mkdir_latency": { 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mknod_latency": { 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.260 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "req_mksnap_latency": { 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "req_open_latency": { 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.261 INFO:teuthology.orchestra.run.smithi155.stdout: "req_readdir_latency": { 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rename_latency": { 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "req_renamesnap_latency": { 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.262 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "req_snapdiff_latency": { 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmdir_latency": { 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.263 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmsnap_latency": { 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "req_rmxattr_latency": { 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setattr_latency": { 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.264 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setdirlayout_latency": { 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setfilelock_latency": { 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setlayout_latency": { 2024-05-08T23:32:01.265 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "req_setxattr_latency": { 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "req_symlink_latency": { 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.266 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "req_unlink_latency": { 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "req_getvxattr_latency": { 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.267 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_sessions": { 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "session_count": 2, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "session_add": 6, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "session_remove": 4, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_open": 4, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "sessions_stale": 0, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "total_load": 0, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "average_load": 0, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "avg_session_uptime": 3775, 2024-05-08T23:32:01.268 INFO:teuthology.orchestra.run.smithi155.stdout: "mdthresh_evicted": 0 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:32:01.269 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:32:01.270 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 206417, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 63, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:32:01.271 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 9648, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 202, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 151348, 2024-05-08T23:32:01.272 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 600, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "objecter": { 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_active": 0, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_laggy": 0, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send": 3119, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_send_bytes": 26, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_resend": 4, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_reply": 3119, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "op_latency": { 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3119, 2024-05-08T23:32:01.273 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 1.425995336, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000457196 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "op_inflight": 0, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "oplen_avg": { 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 3119, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 3120 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "op": 3119, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "op_r": 3119, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "op_w": 0, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "op_rmw": 0, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "op_pg": 0, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_stat": 1654, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_create": 0, 2024-05-08T23:32:01.274 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_read": 1462, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_write": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writefull": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_writesame": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_append": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_zero": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_truncate": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_delete": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_mapext": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_sparse_read": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_clonerange": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_getxattr": 2, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_setxattr": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_rmxattr": 0, 2024-05-08T23:32:01.275 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_call": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_watch": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_notify": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "osdop_other": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_active": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_send": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_resend": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "linger_ping": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_active": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_send": 0, 2024-05-08T23:32:01.276 INFO:teuthology.orchestra.run.smithi155.stdout: "poolop_resend": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_active": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_send": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "poolstat_resend": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_active": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_send": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "statfs_resend": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "command_active": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "command_send": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "command_resend": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "map_epoch": 98, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "map_full": 0, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "map_inc": 17, 2024-05-08T23:32:01.277 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_sessions": 7, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_open": 7, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_session_close": 0, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_laggy": 0, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_wr": 0, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_rd": 2, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_del": 0 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "oft": { 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_objs": 0, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_updates": 0, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "omap_total_removes": 0 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.278 INFO:teuthology.orchestra.run.smithi155.stdout: "osd-slow-ops": { 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "slow_ops_count": 0 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "purge_queue": { 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executing_high_water": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_executed": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "pq_item_in_journal": 0 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:32:01.279 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 3387, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 26047883, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 3387, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3387, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 26047883, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.280 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_bytes": { 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3119, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 24049357, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3119, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 24049357, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.281 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-objecter_ops": { 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 1024, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 3119, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 3119, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 3119, 2024-05-08T23:32:01.282 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 3119, 2024-05-08T23:32:01.283 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:32:01.283 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.285 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:32:01.286 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-write_buf_throttle-0x561a386d45a0": { 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 3758096384, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 0, 2024-05-08T23:32:01.287 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:01.288 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-08T23:32:01.601 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-08T23:32:01.602 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-08T23:32:01.602 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-08T23:32:01.602 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:32:01.602 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:32:01.602 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T23:32:01.602 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T23:32:01.827 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T23:32:02.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:01 smithi155 ceph-mon[47105]: pgmap v388: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-05-08T23:32:02.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:01 smithi016 ceph-mon[42541]: pgmap v388: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-05-08T23:32:02.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:01 smithi189 ceph-mon[46919]: pgmap v388: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 9 op/s 2024-05-08T23:32:02.762 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 0, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 2, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 0, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 0, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 4, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 0, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.001233277, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T23:32:02.763 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.000310144, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:32:02.764 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 6, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 182, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 3232, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 184368, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 5, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 3, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.012678409, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.008039487, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.001956755, 2024-05-08T23:32:02.765 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 180, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.020450927, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000113616 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 184368 2024-05-08T23:32:02.766 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_messages": 245, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages": 235, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_bytes": 1429696, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_bytes": 134560, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_created_connections": 2, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_active_connections": 1, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_total_time": 0.046764412, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_send_time": 0.014570230, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_recv_time": 0.032367087, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_running_fast_dispatch_time": 0.006418510, 2024-05-08T23:32:02.767 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 234, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.012463368, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000053262 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_recv_encrypted_bytes": 1429696, 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: "msgr_send_encrypted_bytes": 134560 2024-05-08T23:32:02.768 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "cct": { 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "total_workers": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "unhealthy_workers": 0 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "mempool": { 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bloom_filter_items": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:32:02.769 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_extent_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_blob_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:32:02.770 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_txc_items": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluestore_writing_items": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_items": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:32:02.771 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_bytes": 148994, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_anon_items": 31, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "buffer_meta_items": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_bytes": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_items": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_mapbl_items": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:32:02.772 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_pglog_items": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_bytes": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_items": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_bytes": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "pgmap_items": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_bytes": 928, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_co_items": 2, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_bytes": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_1_items": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_bytes": 0, 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: "unittest_2_items": 0 2024-05-08T23:32:02.773 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "val": 0, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "max": 104857600, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "get_started": 0, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "get": 247, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "get_sum": 1395088, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "get_or_fail_success": 247, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "take": 0, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "take_sum": 0, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "put": 247, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "put_sum": 1395088, 2024-05-08T23:32:02.774 INFO:teuthology.orchestra.run.smithi155.stdout: "wait": { 2024-05-08T23:32:02.775 INFO:teuthology.orchestra.run.smithi155.stdout: "avgcount": 0, 2024-05-08T23:32:02.775 INFO:teuthology.orchestra.run.smithi155.stdout: "sum": 0.000000000, 2024-05-08T23:32:02.775 INFO:teuthology.orchestra.run.smithi155.stdout: "avgtime": 0.000000000 2024-05-08T23:32:02.775 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:02.775 INFO:teuthology.orchestra.run.smithi155.stdout: } 2024-05-08T23:32:02.775 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-05-08T23:32:03.125 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-08T23:32:03.125 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-08T23:32:03.125 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-08T23:32:03.125 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-08T23:32:03.125 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-08T23:32:03.126 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T23:32:03.126 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T23:32:03.373 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:32:04.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:03 smithi189 ceph-mon[46919]: pgmap v389: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-05-08T23:32:04.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:03 smithi155 ceph-mon[47105]: pgmap v389: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-05-08T23:32:04.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:03 smithi016 ceph-mon[42541]: pgmap v389: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 595, 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 715, 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 156858, 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 402212, 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 10, 2024-05-08T23:32:04.386 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.076128381, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.027782241, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.038070541, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005635143, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 713, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.046109728, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000064670 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:04.387 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1244, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000358793, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000288 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3456, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 14592 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 90, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 139, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 27804, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 156735, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 10, 2024-05-08T23:32:04.388 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.013716146, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.005055109, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.007834265, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000410251, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 138, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.018352780, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000132991 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 163, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000036144, 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000221 2024-05-08T23:32:04.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2000, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 118176 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 361, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 500, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1482120, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 957004, 2024-05-08T23:32:04.390 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 55, 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.076773744, 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.023618888, 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.051424463, 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.006410378, 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:04.391 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 498, 2024-05-08T23:32:04.392 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.033640231, 2024-05-08T23:32:04.392 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000067550 2024-05-08T23:32:04.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.392 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:04.392 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 57, 2024-05-08T23:32:04.392 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000007142, 2024-05-08T23:32:04.393 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000125 2024-05-08T23:32:04.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.393 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1473792, 2024-05-08T23:32:04.393 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 491360 2024-05-08T23:32:04.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.393 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-05-08T23:32:04.394 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 56, 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001495821, 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000026711 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-05-08T23:32:04.395 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000015499, 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000007749 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 8, 2024-05-08T23:32:04.396 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 6, 2024-05-08T23:32:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-05-08T23:32:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6, 2024-05-08T23:32:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002242995, 2024-05-08T23:32:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000373832 2024-05-08T23:32:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-05-08T23:32:04.398 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 2, 2024-05-08T23:32:04.398 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-05-08T23:32:04.398 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-05-08T23:32:04.398 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-05-08T23:32:04.398 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-05-08T23:32:04.398 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-05-08T23:32:04.399 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 11, 2024-05-08T23:32:04.399 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-05-08T23:32:04.399 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 11, 2024-05-08T23:32:04.399 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-05-08T23:32:04.399 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 10, 2024-05-08T23:32:04.400 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 1, 2024-05-08T23:32:04.400 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 1, 2024-05-08T23:32:04.400 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 1, 2024-05-08T23:32:04.400 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 2, 2024-05-08T23:32:04.400 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 15, 2024-05-08T23:32:04.400 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 10, 2024-05-08T23:32:04.401 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-05-08T23:32:04.401 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 2, 2024-05-08T23:32:04.401 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:32:04.401 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-05-08T23:32:04.401 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-05-08T23:32:04.401 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-05-08T23:32:04.402 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-05-08T23:32:04.402 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-05-08T23:32:04.402 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-05-08T23:32:04.402 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-05-08T23:32:04.402 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-05-08T23:32:04.402 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-05-08T23:32:04.403 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:32:04.403 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-05-08T23:32:04.403 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 4, 2024-05-08T23:32:04.403 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 234, 2024-05-08T23:32:04.403 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-05-08T23:32:04.403 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:32:04.404 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-05-08T23:32:04.405 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:32:04.405 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:32:04.405 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:32:04.405 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:32:04.405 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:32:04.405 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:32:04.406 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-05-08T23:32:04.406 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:32:04.406 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-05-08T23:32:04.406 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-05-08T23:32:04.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.406 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-05-08T23:32:04.407 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-05-08T23:32:04.407 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-05-08T23:32:04.407 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-05-08T23:32:04.407 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 3, 2024-05-08T23:32:04.407 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-05-08T23:32:04.407 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-05-08T23:32:04.408 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-05-08T23:32:04.409 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 7, 2024-05-08T23:32:04.410 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 7, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 1, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 1, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-05-08T23:32:04.411 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4196801, 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 7, 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011763939, 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001680562 2024-05-08T23:32:04.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.413 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-05-08T23:32:04.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.413 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-05-08T23:32:04.413 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 14, 2024-05-08T23:32:04.413 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 14, 2024-05-08T23:32:04.413 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 13, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 14, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 1, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 11, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 12, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 1, 2024-05-08T23:32:04.414 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 1, 2024-05-08T23:32:04.415 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 3, 2024-05-08T23:32:04.415 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 2, 2024-05-08T23:32:04.415 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 50228, 2024-05-08T23:32:04.415 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 198964 2024-05-08T23:32:04.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.415 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 10, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 8, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 27, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.416 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000779939, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000194984 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001463056, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000731528 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.419 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-05-08T23:32:04.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.421 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.422 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-05-08T23:32:04.423 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.424 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.425 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.426 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-05-08T23:32:04.427 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 5, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 1, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 731, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:32:04.428 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:32:04.429 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-05-08T23:32:04.430 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 239719, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 85, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 88, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 1, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:32:04.431 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9648, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 202, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 56008, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 225, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.432 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 46, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 13034, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 46, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 46, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.242995415, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005282509 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 46, 2024-05-08T23:32:04.433 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 70 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 46, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 46, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 7, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 8, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-05-08T23:32:04.434 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-05-08T23:32:04.435 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-05-08T23:32:04.436 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 98, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 17, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 8, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 8, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-05-08T23:32:04.437 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 17, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 2 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-05-08T23:32:04.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1042, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1553765, 2024-05-08T23:32:04.439 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1042, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1042, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1553765, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:04.440 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 46, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 13034, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 46, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 13034, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 46, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 46, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 46, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 46, 2024-05-08T23:32:04.442 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:04.443 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55b4207660a0": { 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:04.444 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 7, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2497, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 7, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 7, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2497, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:04.445 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-05-08T23:32:04.757 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T23:32:04.757 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T23:32:04.758 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-08T23:32:04.758 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:32:04.758 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:32:04.758 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T23:32:04.758 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T23:32:04.980 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:32:05.977 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-05-08T23:32:05.977 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:32:05.977 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2425, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2425, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 689790, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 753386, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.215741085, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.068002596, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.053937022, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.075147089, 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:05.978 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2424, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.098755838, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000040740 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 4, 2024-05-08T23:32:05.979 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 181, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 876, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 461036, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.011512505, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007186627, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.002004569, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000067353, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 178, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.016622515, 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000093384 2024-05-08T23:32:05.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 460352 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 268, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 257, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1460588, 2024-05-08T23:32:05.981 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 138103, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 3, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.051442400, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.016293283, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.034481501, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.006928977, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 256, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.016174914, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000063183 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:05.982 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1460320, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 137792 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-05-08T23:32:05.983 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4089, 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.119373978, 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000029193 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:05.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-05-08T23:32:05.985 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-05-08T23:32:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-05-08T23:32:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:32:05.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-05-08T23:32:05.989 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-05-08T23:32:05.990 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 7, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-05-08T23:32:05.991 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 1, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4196801, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 4196660, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 7 2024-05-08T23:32:05.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 3, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 3, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 1, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 1, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 49072, 2024-05-08T23:32:05.993 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 207156 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-05-08T23:32:05.994 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.995 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.996 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.997 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.998 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:05.999 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.000 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.001 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-05-08T23:32:06.002 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.003 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.004 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 4, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 3777, 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-05-08T23:32:06.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:32:06.006 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-05-08T23:32:06.007 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 181393, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 49, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-05-08T23:32:06.008 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9648, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 202, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 8320, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 13, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-05-08T23:32:06.009 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 2424, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 2424, 2024-05-08T23:32:06.010 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2424, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1.179977254, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000486789 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2424, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 2425 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 2424, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 2424, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-05-08T23:32:06.011 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 1582, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 840, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-05-08T23:32:06.012 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-05-08T23:32:06.013 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 98, 2024-05-08T23:32:06.014 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 17, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 4, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 4, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-05-08T23:32:06.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 2693, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1892244, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 2693, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2693, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1892244, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.017 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2424, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 2502, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2424, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2502, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.018 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 2424, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 2424, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2424, 2024-05-08T23:32:06.019 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2424, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:06.020 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55df81b46500": { 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:06.021 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:06.022 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-05-08T23:32:06.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:05 smithi189 ceph-mon[46919]: pgmap v390: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-05-08T23:32:06.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:05 smithi155 ceph-mon[47105]: pgmap v390: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-05-08T23:32:06.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:05 smithi016 ceph-mon[42541]: pgmap v390: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-05-08T23:32:06.442 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T23:32:06.442 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T23:32:06.442 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-05-08T23:32:06.442 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:32:06.442 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:32:06.442 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T23:32:06.442 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T23:32:06.664 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:32:07.628 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-05-08T23:32:07.628 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:32:07.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 963, 2024-05-08T23:32:07.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 944, 2024-05-08T23:32:07.628 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 245939, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 806689, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 13, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.095769752, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.037807434, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.052608094, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005786955, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 942, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.066186835, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000070262 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:07.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1687, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000373304, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000221 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 347104 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 383, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 408, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1499396, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 219139, 2024-05-08T23:32:07.630 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 11, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.060826058, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.020184581, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.041974473, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.006682931, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 407, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.030965956, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000076083 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 174, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000036489, 2024-05-08T23:32:07.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000209 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1472416, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 141632 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 72, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 121, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 15115, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 519925, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 51, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 7, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.022636317, 2024-05-08T23:32:07.632 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.003748817, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.016841138, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000243935, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 119, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011618642, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000097635 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 98, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000015471, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000157 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 6816, 2024-05-08T23:32:07.633 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 124848 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 51, 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001835726, 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000035994 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000014816, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000007408 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 5, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 4, 2024-05-08T23:32:07.635 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3.565037842, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.891259460 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 1, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 14, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 14, 2024-05-08T23:32:07.636 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 14, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 4, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 4, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 2, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 6, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 4, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 1, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-05-08T23:32:07.637 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 4, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 234, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T23:32:07.638 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 1, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 1, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-05-08T23:32:07.639 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 1 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 2, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-05-08T23:32:07.640 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-05-08T23:32:07.641 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 7, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 7, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 1, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 1, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4196801, 2024-05-08T23:32:07.642 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 7, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.010916400, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001559485 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 17, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 17, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 15, 2024-05-08T23:32:07.643 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 16, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 1, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 14, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 14, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 4, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 4, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 49808, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 198964 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 8, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-05-08T23:32:07.644 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 5, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 28, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 1, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3.565037842, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.891259460 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.645 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-05-08T23:32:07.646 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-05-08T23:32:07.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.648 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.648 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.649 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-05-08T23:32:07.649 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.649 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.650 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.653 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-05-08T23:32:07.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.655 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.656 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-05-08T23:32:07.657 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 3, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 3, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 737, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:32:07.658 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:32:07.659 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:32:07.660 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 279831, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 102, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9648, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 202, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:32:07.661 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 68085, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 298, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 45, 2024-05-08T23:32:07.662 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 13170, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 27, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 45, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 45, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.231995623, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005155458 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 45, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 68 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 45, 2024-05-08T23:32:07.663 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 45, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 7, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 8, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-05-08T23:32:07.664 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-05-08T23:32:07.665 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-05-08T23:32:07.666 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 98, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 17, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 9, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 9, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 16, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 1 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.667 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-05-08T23:32:07.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1414, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1614490, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1414, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1414, 2024-05-08T23:32:07.669 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1614490, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:07.670 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 45, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 13170, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 45, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 13170, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-05-08T23:32:07.671 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 45, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 45, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 45, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 45, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.672 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-05-08T23:32:07.673 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x562aeb2200a0": { 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 7, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2497, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:07.674 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 7, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 7, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2497, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:07.675 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-05-08T23:32:07.982 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-08T23:32:07.982 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-08T23:32:07.982 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-08T23:32:07.982 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-08T23:32:07.982 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T23:32:07.982 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T23:32:07.983 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 378de17a-0d90-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T23:32:08.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:07 smithi189 ceph-mon[46919]: pgmap v391: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-05-08T23:32:08.206 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T23:32:08.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:07 smithi155 ceph-mon[47105]: pgmap v391: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-05-08T23:32:08.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:07 smithi016 ceph-mon[42541]: pgmap v391: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-05-08T23:32:09.177 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-05-08T23:32:09.177 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T23:32:09.177 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 0, 2024-05-08T23:32:09.177 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2, 2024-05-08T23:32:09.177 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 0, 2024-05-08T23:32:09.177 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 0, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.001278745, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000313839, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:09.178 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T23:32:09.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 251, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 239, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1433152, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 136192, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.050504829, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.016378812, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.034257942, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.006655400, 2024-05-08T23:32:09.180 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 239, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.014435910, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000060401 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1433152, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 136192 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.181 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 182, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 320, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 184272, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.012150108, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.008032940, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.001378597, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T23:32:09.182 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 179, 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.022333898, 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000124770 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:09.183 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 184272 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-05-08T23:32:09.184 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-05-08T23:32:09.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T23:32:09.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T23:32:09.187 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-05-08T23:32:09.188 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-05-08T23:32:09.188 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-05-08T23:32:09.188 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T23:32:09.188 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T23:32:09.188 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-05-08T23:32:09.188 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-05-08T23:32:09.189 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-05-08T23:32:09.189 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-05-08T23:32:09.189 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T23:32:09.189 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-05-08T23:32:09.189 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T23:32:09.189 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 149007, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 30, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-05-08T23:32:09.190 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-05-08T23:32:09.191 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 249, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1395348, 2024-05-08T23:32:09.192 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 249, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 249, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1395348, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-05-08T23:32:09.193 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-05-08T23:32:09.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:09.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-05-08T23:32:09.194 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-05-08T23:32:09.548 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-08T23:32:09.548 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-08T23:32:09.548 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-08T23:32:09.548 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-08T23:32:09.548 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-08T23:32:09.549 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-08T23:32:09.569 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-08T23:32:09.569 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:32:09.569 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:09.596 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:32:09.596 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:09.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 -- 172.21.15.16:0/2191514661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030106ff0 msgr2=0x7f30301073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 --2- 172.21.15.16:0/2191514661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030106ff0 0x7f30301073d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3020009990 tx=0x7f3020009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 -- 172.21.15.16:0/2191514661 shutdown_connections 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 --2- 172.21.15.16:0/2191514661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3030100070 0x7f303010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 --2- 172.21.15.16:0/2191514661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030106ff0 0x7f30301073d0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 --2- 172.21.15.16:0/2191514661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30300ff6b0 0x7f30300ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.854+0000 7f303582d700 1 -- 172.21.15.16:0/2191514661 >> 172.21.15.16:0/2191514661 conn(0x7f3030074fd0 msgr2=0x7f30300753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.855+0000 7f303582d700 1 -- 172.21.15.16:0/2191514661 shutdown_connections 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.855+0000 7f303582d700 1 -- 172.21.15.16:0/2191514661 wait complete. 2024-05-08T23:32:09.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.855+0000 7f303582d700 1 Processor -- start 2024-05-08T23:32:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.855+0000 7f303582d700 1 -- start start 2024-05-08T23:32:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303582d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30300ff6b0 0x7f3030197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303582d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 0x7f3030197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303582d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3030106ff0 0x7f303019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303582d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019c070 con 0x7f3030100070 2024-05-08T23:32:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303582d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019c1e0 con 0x7f3030106ff0 2024-05-08T23:32:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303582d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019c3a0 con 0x7f30300ff6b0 2024-05-08T23:32:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 0x7f3030197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 0x7f3030197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35298/0 (socket says 172.21.15.16:35298) 2024-05-08T23:32:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 -- 172.21.15.16:0/2643137412 learned_addr learned my addr 172.21.15.16:0/2643137412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f302effd700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30300ff6b0 0x7f3030197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 -- 172.21.15.16:0/2643137412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30300ff6b0 msgr2=0x7f3030197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30300ff6b0 0x7f3030197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 -- 172.21.15.16:0/2643137412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3030106ff0 msgr2=0x7f303019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f302f7fe700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3030106ff0 0x7f303019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3030106ff0 0x7f303019ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 -- 172.21.15.16:0/2643137412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30200095f0 con 0x7f3030100070 2024-05-08T23:32:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f30267fc700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 0x7f3030197690 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f301800e9a0 tx=0x7f301800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.856+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018009790 con 0x7f3030100070 2024-05-08T23:32:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.857+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303019c680 con 0x7f3030100070 2024-05-08T23:32:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.857+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f301800cbd0 con 0x7f3030100070 2024-05-08T23:32:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.857+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018018680 con 0x7f3030100070 2024-05-08T23:32:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.857+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30301a3ec0 con 0x7f3030100070 2024-05-08T23:32:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.857+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30301093a0 con 0x7f3030100070 2024-05-08T23:32:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.859+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f301800cd40 con 0x7f3030100070 2024-05-08T23:32:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.860+0000 7f303482b700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f301c0a1290 0x7f301c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.860+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f3018014070 con 0x7f3030100070 2024-05-08T23:32:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.860+0000 7f302effd700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f301c0a1290 0x7f301c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.861+0000 7f302effd700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f301c0a1290 0x7f301c0a3750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30200059d0 tx=0x7f3020005940 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:09.864+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3018096680 con 0x7f3030100070 2024-05-08T23:32:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.083+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3030198450 con 0x7f3030100070 2024-05-08T23:32:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.085+0000 7f303482b700 1 -- 172.21.15.16:0/2643137412 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+25183 (secure 0 0 0) 0x7f3018099ca0 con 0x7f3030100070 2024-05-08T23:32:10.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:32:10.085 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:31:13.733005+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:10.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f301c0a1290 msgr2=0x7f301c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f301c0a1290 0x7f301c0a3750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30200059d0 tx=0x7f3020005940 comp rx=0 tx=0).stop 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 msgr2=0x7f3030197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 0x7f3030197690 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f301800e9a0 tx=0x7f301800ed60 comp rx=0 tx=0).stop 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 shutdown_connections 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f301c0a1290 0x7f301c0a3750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3030106ff0 0x7f303019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30300ff6b0 0x7f3030197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 --2- 172.21.15.16:0/2643137412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3030100070 0x7f3030197690 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 >> 172.21.15.16:0/2643137412 conn(0x7f3030074fd0 msgr2=0x7f30300fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 shutdown_connections 2024-05-08T23:32:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.087+0000 7f303582d700 1 -- 172.21.15.16:0/2643137412 wait complete. 2024-05-08T23:32:10.100 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:09 smithi016 ceph-mon[42541]: pgmap v392: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-05-08T23:32:10.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:32:10.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:10.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:09 smithi155 ceph-mon[47105]: pgmap v392: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 -- 172.21.15.16:0/2942529545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 msgr2=0x7f86e4073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 --2- 172.21.15.16:0/2942529545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e4073750 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009990 tx=0x7f86d8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 -- 172.21.15.16:0/2942529545 shutdown_connections 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 --2- 172.21.15.16:0/2942529545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e410d790 0x7f86e410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 --2- 172.21.15.16:0/2942529545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e4073750 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 --2- 172.21.15.16:0/2942529545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e4073c90 0x7f86e410d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 -- 172.21.15.16:0/2942529545 >> 172.21.15.16:0/2942529545 conn(0x7f86e40fc9e0 msgr2=0x7f86e40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 -- 172.21.15.16:0/2942529545 shutdown_connections 2024-05-08T23:32:10.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.247+0000 7f86eb76f700 1 -- 172.21.15.16:0/2942529545 wait complete. 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 Processor -- start 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 -- start start 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4073c90 0x7f86e4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e410d790 0x7f86e419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419e3f0 con 0x7f86e4073370 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419e560 con 0x7f86e4073c90 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86eb76f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e419e720 con 0x7f86e410d790 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86e950b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86e950b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e41994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35314/0 (socket says 172.21.15.16:35314) 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86e950b700 1 -- 172.21.15.16:0/3401720808 learned_addr learned my addr 172.21.15.16:0/3401720808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.248+0000 7f86e8d0a700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4073c90 0x7f86e4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e9d0c700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e410d790 0x7f86e419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e950b700 1 -- 172.21.15.16:0/3401720808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e410d790 msgr2=0x7f86e419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e950b700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e410d790 0x7f86e419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e950b700 1 -- 172.21.15.16:0/3401720808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4073c90 msgr2=0x7f86e4199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e950b700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4073c90 0x7f86e4199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e950b700 1 -- 172.21.15.16:0/3401720808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80095f0 con 0x7f86e4073370 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86e950b700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e41994d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f86d8004760 tx=0x7f86d8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d801d070 con 0x7f86e4073370 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e419e9a0 con 0x7f86e4073370 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86d8004830 con 0x7f86e4073370 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d80053a0 con 0x7f86e4073370 2024-05-08T23:32:10.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.249+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e41a6240 con 0x7f86e4073370 2024-05-08T23:32:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.250+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e41089a0 con 0x7f86e4073370 2024-05-08T23:32:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.251+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f86d8005500 con 0x7f86e4073370 2024-05-08T23:32:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.252+0000 7f86d67fc700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86d00a1290 0x7f86d00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.252+0000 7f86e8d0a700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86d00a1290 0x7f86d00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.252+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f86d80ce5e0 con 0x7f86e4073370 2024-05-08T23:32:10.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.252+0000 7f86e8d0a700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86d00a1290 0x7f86d00a3750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86e419aa20 tx=0x7f86e0009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.255+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f86d8097720 con 0x7f86e4073370 2024-05-08T23:32:10.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:09 smithi189 ceph-mon[46919]: pgmap v392: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-05-08T23:32:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.472+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86e4067460 con 0x7f86e4073370 2024-05-08T23:32:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.473+0000 7f86d67fc700 1 -- 172.21.15.16:0/3401720808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86d809ad40 con 0x7f86e4073370 2024-05-08T23:32:10.472 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:32:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86d00a1290 msgr2=0x7f86d00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86d00a1290 0x7f86d00a3750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f86e419aa20 tx=0x7f86e0009260 comp rx=0 tx=0).stop 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 msgr2=0x7f86e41994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e41994d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f86d8004760 tx=0x7f86d8004790 comp rx=0 tx=0).stop 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 shutdown_connections 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f86d00a1290 0x7f86d00a3750 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86e410d790 0x7f86e419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86e4073370 0x7f86e41994d0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 --2- 172.21.15.16:0/3401720808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86e4073c90 0x7f86e4199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 >> 172.21.15.16:0/3401720808 conn(0x7f86e40fc9e0 msgr2=0x7f86e40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 shutdown_connections 2024-05-08T23:32:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.475+0000 7f86eb76f700 1 -- 172.21.15.16:0/3401720808 wait complete. 2024-05-08T23:32:10.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 -- 172.21.15.16:0/2690277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c068c90 msgr2=0x7efe2c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 --2- 172.21.15.16:0/2690277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c068c90 0x7efe2c105b60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0099c0 tx=0x7efe1c009cd0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 -- 172.21.15.16:0/2690277734 shutdown_connections 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 --2- 172.21.15.16:0/2690277734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c106290 0x7efe2c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 --2- 172.21.15.16:0/2690277734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe2c0682e0 0x7efe2c0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 --2- 172.21.15.16:0/2690277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c068c90 0x7efe2c105b60 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 -- 172.21.15.16:0/2690277734 >> 172.21.15.16:0/2690277734 conn(0x7efe2c076f40 msgr2=0x7efe2c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 -- 172.21.15.16:0/2690277734 shutdown_connections 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 -- 172.21.15.16:0/2690277734 wait complete. 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.630+0000 7efe3469a700 1 Processor -- start 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 -- start start 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c0682e0 0x7efe2c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe2c106290 0x7efe2c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 0x7efe2c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c19c650 con 0x7efe2c19bbd0 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c19c7c0 con 0x7efe2c106290 2024-05-08T23:32:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe3469a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c19c980 con 0x7efe2c0682e0 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 0x7efe2c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 0x7efe2c19c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35332/0 (socket says 172.21.15.16:35332) 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 -- 172.21.15.16:0/986901811 learned_addr learned my addr 172.21.15.16:0/986901811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32436700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c0682e0 0x7efe2c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe31c35700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe2c106290 0x7efe2c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 -- 172.21.15.16:0/986901811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c0682e0 msgr2=0x7efe2c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c0682e0 0x7efe2c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 -- 172.21.15.16:0/986901811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe2c106290 msgr2=0x7efe2c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe2c106290 0x7efe2c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.631+0000 7efe32c37700 1 -- 172.21.15.16:0/986901811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c0095f0 con 0x7efe2c19bbd0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.632+0000 7efe32c37700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 0x7efe2c19c050 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ef30 tx=0x7efe2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.632+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24018070 con 0x7efe2c19bbd0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.632+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efe2400f040 con 0x7efe2c19bbd0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.632+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c1a3e80 con 0x7efe2c19bbd0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.632+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400cbc0 con 0x7efe2c19bbd0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.632+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c1a4350 con 0x7efe2c19bbd0 2024-05-08T23:32:10.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.633+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c1094f0 con 0x7efe2c19bbd0 2024-05-08T23:32:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.634+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efe2400cd40 con 0x7efe2c19bbd0 2024-05-08T23:32:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.635+0000 7efe237fe700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe180a1310 0x7efe180a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:10.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.635+0000 7efe32436700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe180a1310 0x7efe180a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.635+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9446+0+0 (secure 0 0 0) 0x7efe240ccf70 con 0x7efe2c19bbd0 2024-05-08T23:32:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.636+0000 7efe32436700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe180a1310 0x7efe180a37d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efe28005950 tx=0x7efe2800b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:10.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.639+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efe240960b0 con 0x7efe2c19bbd0 2024-05-08T23:32:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.856+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe2c1985f0 con 0x7efe2c19bbd0 2024-05-08T23:32:10.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.856+0000 7efe237fe700 1 -- 172.21.15.16:0/986901811 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe240996d0 con 0x7efe2c19bbd0 2024-05-08T23:32:10.856 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe180a1310 msgr2=0x7efe180a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe180a1310 0x7efe180a37d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efe28005950 tx=0x7efe2800b410 comp rx=0 tx=0).stop 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 msgr2=0x7efe2c19c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 0x7efe2c19c050 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ef30 tx=0x7efe2400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 shutdown_connections 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efe180a1310 0x7efe180a37d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe2c19bbd0 0x7efe2c19c050 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c0682e0 0x7efe2c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 --2- 172.21.15.16:0/986901811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe2c106290 0x7efe2c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.858+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 >> 172.21.15.16:0/986901811 conn(0x7efe2c076f40 msgr2=0x7efe2c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.859+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 shutdown_connections 2024-05-08T23:32:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:10.859+0000 7efe3469a700 1 -- 172.21.15.16:0/986901811 wait complete. 2024-05-08T23:32:10.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 -- 172.21.15.16:0/3254956092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a4101b90 msgr2=0x7f72a4105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 --2- 172.21.15.16:0/3254956092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a4101b90 0x7f72a4105bd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7294009990 tx=0x7f7294009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 -- 172.21.15.16:0/3254956092 shutdown_connections 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 --2- 172.21.15.16:0/3254956092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72a4106300 0x7f72a410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 --2- 172.21.15.16:0/3254956092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72a41011e0 0x7f72a41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 --2- 172.21.15.16:0/3254956092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a4101b90 0x7f72a4105bd0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 -- 172.21.15.16:0/3254956092 >> 172.21.15.16:0/3254956092 conn(0x7f72a40fca80 msgr2=0x7f72a40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 -- 172.21.15.16:0/3254956092 shutdown_connections 2024-05-08T23:32:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.017+0000 7f72ab697700 1 -- 172.21.15.16:0/3254956092 wait complete. 2024-05-08T23:32:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 Processor -- start 2024-05-08T23:32:11.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 -- start start 2024-05-08T23:32:11.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72a41011e0 0x7f72a41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72a4106300 0x7f72a4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 0x7f72a419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419c6e0 con 0x7f72a419bc60 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419c850 con 0x7f72a4106300 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72ab697700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419ca10 con 0x7f72a41011e0 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72a9c34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 0x7f72a419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72a9c34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 0x7f72a419c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35342/0 (socket says 172.21.15.16:35342) 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72a9c34700 1 -- 172.21.15.16:0/1654878010 learned_addr learned my addr 172.21.15.16:0/1654878010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72a8c32700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72a4106300 0x7f72a4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.018+0000 7f72a9433700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72a41011e0 0x7f72a41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72a9c34700 1 -- 172.21.15.16:0/1654878010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72a41011e0 msgr2=0x7f72a41973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72a9c34700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72a41011e0 0x7f72a41973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72a9c34700 1 -- 172.21.15.16:0/1654878010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72a4106300 msgr2=0x7f72a4197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72a9c34700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72a4106300 0x7f72a4197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72a9c34700 1 -- 172.21.15.16:0/1654878010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940095f0 con 0x7f72a419bc60 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72a9c34700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 0x7f72a419c0e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f729c007ee0 tx=0x7f729c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c017070 con 0x7f72a419bc60 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f729c00f040 con 0x7f72a419bc60 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a41a3f70 con 0x7f72a419bc60 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c005570 con 0x7f72a419bc60 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.019+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41a43f0 con 0x7f72a419bc60 2024-05-08T23:32:11.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.020+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a41095d0 con 0x7f72a419bc60 2024-05-08T23:32:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.022+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f729c00db20 con 0x7f72a419bc60 2024-05-08T23:32:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.023+0000 7f729a7fc700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72900a1310 0x7f72900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.023+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f729c0ccd90 con 0x7f72a419bc60 2024-05-08T23:32:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.023+0000 7f72a9433700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72900a1310 0x7f72900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.023+0000 7f72a9433700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72900a1310 0x7f72900a37d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72a0005950 tx=0x7f72a000b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.027+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f729c095d90 con 0x7f72a419bc60 2024-05-08T23:32:11.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:10 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2643137412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:11.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:10 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3401720808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:32:11.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:10 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/986901811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:32:11.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.253+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72a4198680 con 0x7f72a419bc60 2024-05-08T23:32:11.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.254+0000 7f729a7fc700 1 -- 172.21.15.16:0/1654878010 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f729c0993b0 con 0x7f72a419bc60 2024-05-08T23:32:11.253 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72900a1310 msgr2=0x7f72900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72900a1310 0x7f72900a37d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f72a0005950 tx=0x7f72a000b410 comp rx=0 tx=0).stop 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 msgr2=0x7f72a419c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 0x7f72a419c0e0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f729c007ee0 tx=0x7f729c00d3b0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 shutdown_connections 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f72900a1310 0x7f72900a37d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a419bc60 0x7f72a419c0e0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72a41011e0 0x7f72a41973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.256+0000 7f72ab697700 1 --2- 172.21.15.16:0/1654878010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72a4106300 0x7f72a4197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.257+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 >> 172.21.15.16:0/1654878010 conn(0x7f72a40fca80 msgr2=0x7f72a40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.257+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 shutdown_connections 2024-05-08T23:32:11.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.257+0000 7f72ab697700 1 -- 172.21.15.16:0/1654878010 wait complete. 2024-05-08T23:32:11.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2643137412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:11.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3401720808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:32:11.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:10 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/986901811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:32:11.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 -- 172.21.15.16:0/1909907324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ff410 msgr2=0x7f20440ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 --2- 172.21.15.16:0/1909907324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ff410 0x7f20440ff890 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f2038009990 tx=0x7f2038009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 -- 172.21.15.16:0/1909907324 shutdown_connections 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 --2- 172.21.15.16:0/1909907324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20440ffdd0 0x7f204410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 --2- 172.21.15.16:0/1909907324 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2044105ef0 0x7f20441062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 --2- 172.21.15.16:0/1909907324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ff410 0x7f20440ff890 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 -- 172.21.15.16:0/1909907324 >> 172.21.15.16:0/1909907324 conn(0x7f20440763a0 msgr2=0x7f20440fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 -- 172.21.15.16:0/1909907324 shutdown_connections 2024-05-08T23:32:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.412+0000 7f204a38e700 1 -- 172.21.15.16:0/1909907324 wait complete. 2024-05-08T23:32:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 Processor -- start 2024-05-08T23:32:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 -- start start 2024-05-08T23:32:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20440ff410 0x7f20441005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 0x7f2044100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2044105ef0 0x7f2044101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20441a5760 con 0x7f20440ffdd0 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20441a58d0 con 0x7f20440ff410 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204a38e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20441a5a90 con 0x7f2044105ef0 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f20437fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 0x7f2044100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f20437fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 0x7f2044100b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35368/0 (socket says 172.21.15.16:35368) 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f20437fe700 1 -- 172.21.15.16:0/1292817741 learned_addr learned my addr 172.21.15.16:0/1292817741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f2043fff700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20440ff410 0x7f20441005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.413+0000 7f204892b700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2044105ef0 0x7f2044101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20437fe700 1 -- 172.21.15.16:0/1292817741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2044105ef0 msgr2=0x7f2044101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20437fe700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2044105ef0 0x7f2044101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20437fe700 1 -- 172.21.15.16:0/1292817741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20440ff410 msgr2=0x7f20441005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20437fe700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20440ff410 0x7f20441005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20437fe700 1 -- 172.21.15.16:0/1292817741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380095f0 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20437fe700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 0x7f2044100b30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2038004730 tx=0x7f2038004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203801d070 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2038022470 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20441a5d10 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038017650 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.414+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20441a6190 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.415+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2044109440 con 0x7f20440ffdd0 2024-05-08T23:32:11.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.417+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f20380225e0 con 0x7f20440ffdd0 2024-05-08T23:32:11.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.418+0000 7f20417fa700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f202c0a1310 0x7f202c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.418+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f20380ce5c0 con 0x7f20440ffdd0 2024-05-08T23:32:11.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.418+0000 7f2043fff700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f202c0a1310 0x7f202c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.418+0000 7f2043fff700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f202c0a1310 0x7f202c0a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2034007be0 tx=0x7f20340073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:10 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2643137412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:11.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:10 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3401720808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:32:11.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:10 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/986901811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:32:11.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.421+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2038097760 con 0x7f20440ffdd0 2024-05-08T23:32:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.641+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2044101b00 con 0x7f20440ffdd0 2024-05-08T23:32:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.641+0000 7f20417fa700 1 -- 172.21.15.16:0/1292817741 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) v1 ==== 103+0+11 (secure 0 0 0) 0x7f20380a1070 con 0x7f20440ffdd0 2024-05-08T23:32:11.641 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.643+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f202c0a1310 msgr2=0x7f202c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.643+0000 7f204a38e700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f202c0a1310 0x7f202c0a37d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2034007be0 tx=0x7f20340073d0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.643+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 msgr2=0x7f2044100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.643+0000 7f204a38e700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 0x7f2044100b30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2038004730 tx=0x7f2038004760 comp rx=0 tx=0).stop 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.643+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 shutdown_connections 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.643+0000 7f204a38e700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f202c0a1310 0x7f202c0a37d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.644+0000 7f204a38e700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2044105ef0 0x7f2044101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.644+0000 7f204a38e700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f20440ff410 0x7f20441005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.644+0000 7f204a38e700 1 --2- 172.21.15.16:0/1292817741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20440ffdd0 0x7f2044100b30 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.644+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 >> 172.21.15.16:0/1292817741 conn(0x7f20440763a0 msgr2=0x7f2044104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.644+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 shutdown_connections 2024-05-08T23:32:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.644+0000 7f204a38e700 1 -- 172.21.15.16:0/1292817741 wait complete. 2024-05-08T23:32:11.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 -- 172.21.15.16:0/3135163463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100a10 msgr2=0x7f580c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 --2- 172.21.15.16:0/3135163463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100a10 0x7f580c104a30 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5800009990 tx=0x7f5800009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 -- 172.21.15.16:0/3135163463 shutdown_connections 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 --2- 172.21.15.16:0/3135163463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 0x7f580c10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 --2- 172.21.15.16:0/3135163463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c100060 0x7f580c100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 --2- 172.21.15.16:0/3135163463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100a10 0x7f580c104a30 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 -- 172.21.15.16:0/3135163463 >> 172.21.15.16:0/3135163463 conn(0x7f580c076f40 msgr2=0x7f580c077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 -- 172.21.15.16:0/3135163463 shutdown_connections 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 -- 172.21.15.16:0/3135163463 wait complete. 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.797+0000 7f5812938700 1 Processor -- start 2024-05-08T23:32:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 -- start start 2024-05-08T23:32:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 0x7f580c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 0x7f580c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c198bf0 0x7f580c1a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1a38b0 con 0x7f580c100060 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1a3a20 con 0x7f580c198bf0 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5812938700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1a3be0 con 0x7f580c105160 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 0x7f580c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 0x7f580c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35376/0 (socket says 172.21.15.16:35376) 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 -- 172.21.15.16:0/3849203475 learned_addr learned my addr 172.21.15.16:0/3849203475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f5810ed5700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c198bf0 0x7f580c1a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580b7fe700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 0x7f580c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 -- 172.21.15.16:0/3849203475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 msgr2=0x7f580c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 0x7f580c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 -- 172.21.15.16:0/3849203475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c198bf0 msgr2=0x7f580c1a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c198bf0 0x7f580c1a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580bfff700 1 -- 172.21.15.16:0/3849203475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000095f0 con 0x7f580c100060 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.798+0000 7f580b7fe700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 0x7f580c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.799+0000 7f580bfff700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 0x7f580c198120 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9a0 tx=0x7f57fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.799+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc009790 con 0x7f580c100060 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.799+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57fc00cbd0 con 0x7f580c100060 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.799+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc018680 con 0x7f580c100060 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.799+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c1a3ec0 con 0x7f580c100060 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.799+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c1a42f0 con 0x7f580c100060 2024-05-08T23:32:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.800+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c108430 con 0x7f580c100060 2024-05-08T23:32:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.801+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f57fc00cd40 con 0x7f580c100060 2024-05-08T23:32:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.802+0000 7f58097fa700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57f40a1290 0x7f57f40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.802+0000 7f580b7fe700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57f40a1290 0x7f57f40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.802+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f57fc014070 con 0x7f580c100060 2024-05-08T23:32:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.802+0000 7f580b7fe700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57f40a1290 0x7f57f40a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f580c068840 tx=0x7f5800005c00 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:11.806+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f57fc052d60 con 0x7f580c100060 2024-05-08T23:32:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.044+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f580c0686b0 con 0x7f580c100060 2024-05-08T23:32:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.045+0000 7f58097fa700 1 -- 172.21.15.16:0/3849203475 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v99) v1 ==== 81+0+997 (secure 0 0 0) 0x7f57fc0963a0 con 0x7f580c100060 2024-05-08T23:32:12.044 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4093375965 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.044 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:12.044 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1044015911 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.044 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4149284949 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4028430980 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/745120666 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1837768303 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/592052660 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/972303159 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/30115990 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4190536936 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3133644606 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/781396085 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/781396085 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57f40a1290 msgr2=0x7f57f40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57f40a1290 0x7f57f40a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f580c068840 tx=0x7f5800005c00 comp rx=0 tx=0).stop 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 msgr2=0x7f580c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 0x7f580c198120 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00e9a0 tx=0x7f57fc00ecb0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 shutdown_connections 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f57f40a1290 0x7f57f40a3750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f580c198bf0 0x7f580c1a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c100060 0x7f580c198120 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 --2- 172.21.15.16:0/3849203475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f580c105160 0x7f580c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 >> 172.21.15.16:0/3849203475 conn(0x7f580c076f40 msgr2=0x7f580c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 shutdown_connections 2024-05-08T23:32:12.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.047+0000 7f5812938700 1 -- 172.21.15.16:0/3849203475 wait complete. 2024-05-08T23:32:12.048 INFO:teuthology.orchestra.run.smithi016.stderr:listed 17 entries 2024-05-08T23:32:12.061 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-05-08T23:32:12.061 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T23:32:12.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:12.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:11 smithi155 ceph-mon[47105]: pgmap v393: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-05-08T23:32:12.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:11 smithi155 ceph-mon[47105]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:32:12.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:11 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1654878010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:32:12.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:11 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1292817741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 -- 172.21.15.16:0/330086695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 msgr2=0x7ff1f4100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/330086695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f4100320 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009990 tx=0x7ff1e4009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 -- 172.21.15.16:0/330086695 shutdown_connections 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/330086695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 0x7ff1f410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/330086695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f4100320 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/330086695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f41008f0 0x7ff1f4104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 -- 172.21.15.16:0/330086695 >> 172.21.15.16:0/330086695 conn(0x7ff1f40fb7e0 msgr2=0x7ff1f40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 -- 172.21.15.16:0/330086695 shutdown_connections 2024-05-08T23:32:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.263+0000 7ff1f89a1700 1 -- 172.21.15.16:0/330086695 wait complete. 2024-05-08T23:32:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 Processor -- start 2024-05-08T23:32:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 -- start start 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f41019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 0x7ff1f4101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f4102450 0x7ff1f41a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41a5a10 con 0x7ff1f40fff40 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41a5b80 con 0x7ff1f4105060 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f89a1700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41a5d40 con 0x7ff1f4102450 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.264+0000 7ff1f259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f41019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f41019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35400/0 (socket says 172.21.15.16:35400) 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 -- 172.21.15.16:0/1561909448 learned_addr learned my addr 172.21.15.16:0/1561909448 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f1d9b700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 0x7ff1f4101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f2d9d700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f4102450 0x7ff1f41a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 -- 172.21.15.16:0/1561909448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f4102450 msgr2=0x7ff1f41a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f4102450 0x7ff1f41a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 -- 172.21.15.16:0/1561909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 msgr2=0x7ff1f4101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 0x7ff1f4101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 -- 172.21.15.16:0/1561909448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e40095f0 con 0x7ff1f40fff40 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f1d9b700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 0x7ff1f4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f259c700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f41019d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4004730 tx=0x7ff1e4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e401d070 con 0x7ff1f40fff40 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f41a5fc0 con 0x7ff1f40fff40 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1e4022470 con 0x7ff1f40fff40 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4017650 con 0x7ff1f40fff40 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.265+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f41a6380 con 0x7ff1f40fff40 2024-05-08T23:32:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.266+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f4101410 con 0x7ff1f40fff40 2024-05-08T23:32:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.268+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff1e4022870 con 0x7ff1f40fff40 2024-05-08T23:32:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.268+0000 7ff1e37fe700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1dc0a1290 0x7ff1dc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.268+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7ff1e4022e40 con 0x7ff1f40fff40 2024-05-08T23:32:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.268+0000 7ff1f1d9b700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1dc0a1290 0x7ff1dc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.269+0000 7ff1f1d9b700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1dc0a1290 0x7ff1dc0a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4103100 tx=0x7ff1e8009260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.272+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff1e4097fc0 con 0x7ff1f40fff40 2024-05-08T23:32:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:11 smithi016 ceph-mon[42541]: pgmap v393: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-05-08T23:32:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:11 smithi016 ceph-mon[42541]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:32:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:11 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1654878010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:32:12.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:11 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1292817741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:32:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:11 smithi189 ceph-mon[46919]: pgmap v393: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-05-08T23:32:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:11 smithi189 ceph-mon[46919]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T23:32:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:11 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1654878010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:32:12.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:11 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1292817741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:32:12.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.490+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1f4100890 con 0x7ff1f40fff40 2024-05-08T23:32:12.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.491+0000 7ff1e37fe700 1 -- 172.21.15.16:0/1561909448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+25318 (secure 0 0 0) 0x7ff1e4027090 con 0x7ff1f40fff40 2024-05-08T23:32:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:32:12.491 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:32:10.947520+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1dc0a1290 msgr2=0x7ff1dc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1dc0a1290 0x7ff1dc0a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4103100 tx=0x7ff1e8009260 comp rx=0 tx=0).stop 2024-05-08T23:32:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 msgr2=0x7ff1f41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f41019d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4004730 tx=0x7ff1e4009510 comp rx=0 tx=0).stop 2024-05-08T23:32:12.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 shutdown_connections 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7ff1dc0a1290 0x7ff1dc0a3750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f4102450 0x7ff1f41a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1f40fff40 0x7ff1f41019d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 --2- 172.21.15.16:0/1561909448 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff1f4105060 0x7ff1f4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 >> 172.21.15.16:0/1561909448 conn(0x7ff1f40fb7e0 msgr2=0x7ff1f4068490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 shutdown_connections 2024-05-08T23:32:12.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.494+0000 7ff1f89a1700 1 -- 172.21.15.16:0/1561909448 wait complete. 2024-05-08T23:32:12.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T23:14:33.043373+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T23:17:44.419250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T23:17:45.451965+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T23:17:48.549788+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T23:32:12.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T23:32:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 -- 172.21.15.16:0/1105776826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc105f90 msgr2=0x7f97dc106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 --2- 172.21.15.16:0/1105776826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc105f90 0x7f97dc106370 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0099c0 tx=0x7f97cc009cd0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 -- 172.21.15.16:0/1105776826 shutdown_connections 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 --2- 172.21.15.16:0/1105776826 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97dc100ab0 0x7f97dc10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 --2- 172.21.15.16:0/1105776826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc105f90 0x7f97dc106370 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 --2- 172.21.15.16:0/1105776826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97dc1000f0 0x7f97dc100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 -- 172.21.15.16:0/1105776826 >> 172.21.15.16:0/1105776826 conn(0x7f97dc076ba0 msgr2=0x7f97dc076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 -- 172.21.15.16:0/1105776826 shutdown_connections 2024-05-08T23:32:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.651+0000 7f97e0c77700 1 -- 172.21.15.16:0/1105776826 wait complete. 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 Processor -- start 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 -- start start 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97dc1000f0 0x7f97dc101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 0x7f97dc102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97dc105f90 0x7f97dc1025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1a56b0 con 0x7f97dc100ab0 2024-05-08T23:32:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1a5820 con 0x7f97dc105f90 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97e0c77700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1a59e0 con 0x7f97dc1000f0 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.652+0000 7f97d1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 0x7f97dc102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 0x7f97dc102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35426/0 (socket says 172.21.15.16:35426) 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 -- 172.21.15.16:0/4127414879 learned_addr learned my addr 172.21.15.16:0/4127414879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97da59c700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97dc1000f0 0x7f97dc101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97dad9d700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97dc105f90 0x7f97dc1025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 -- 172.21.15.16:0/4127414879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97dc1000f0 msgr2=0x7f97dc101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97dc1000f0 0x7f97dc101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 -- 172.21.15.16:0/4127414879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97dc105f90 msgr2=0x7f97dc1025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97dc105f90 0x7f97dc1025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 -- 172.21.15.16:0/4127414879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc0095f0 con 0x7f97dc100ab0 2024-05-08T23:32:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d1d9b700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 0x7f97dc102060 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e9a0 tx=0x7f97c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4009790 con 0x7f97dc100ab0 2024-05-08T23:32:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97c400cbd0 con 0x7f97dc100ab0 2024-05-08T23:32:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc1a5cc0 con 0x7f97dc100ab0 2024-05-08T23:32:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4018680 con 0x7f97dc100ab0 2024-05-08T23:32:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.653+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc1a61d0 con 0x7f97dc100ab0 2024-05-08T23:32:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.654+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc101560 con 0x7f97dc100ab0 2024-05-08T23:32:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.655+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f97c400cd40 con 0x7f97dc100ab0 2024-05-08T23:32:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.656+0000 7f97d3fff700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f97c80a1290 0x7f97c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.657+0000 7f97da59c700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f97c80a1290 0x7f97c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.657+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f97c4014070 con 0x7f97dc100ab0 2024-05-08T23:32:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.657+0000 7f97da59c700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f97c80a1290 0x7f97c80a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009fd0 tx=0x7f97cc019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.661+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f97c4096770 con 0x7f97dc100ab0 2024-05-08T23:32:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.875+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97dc10fc00 con 0x7f97dc100ab0 2024-05-08T23:32:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.876+0000 7f97d3fff700 1 -- 172.21.15.16:0/4127414879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97c4026020 con 0x7f97dc100ab0 2024-05-08T23:32:12.875 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-05-08T23:32:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f97c80a1290 msgr2=0x7f97c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f97c80a1290 0x7f97c80a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009fd0 tx=0x7f97cc019040 comp rx=0 tx=0).stop 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 msgr2=0x7f97dc102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 0x7f97dc102060 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e9a0 tx=0x7f97c400ecb0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 shutdown_connections 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f97c80a1290 0x7f97c80a3750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f97dc105f90 0x7f97dc1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f97dc1000f0 0x7f97dc101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 --2- 172.21.15.16:0/4127414879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97dc100ab0 0x7f97dc102060 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 >> 172.21.15.16:0/4127414879 conn(0x7f97dc076ba0 msgr2=0x7f97dc069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 shutdown_connections 2024-05-08T23:32:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:12.878+0000 7f97e0c77700 1 -- 172.21.15.16:0/4127414879 wait complete. 2024-05-08T23:32:12.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T23:32:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 -- 172.21.15.16:0/3483804775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18880682e0 msgr2=0x7f18880686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 --2- 172.21.15.16:0/3483804775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18880682e0 0x7f18880686c0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f18780099c0 tx=0x7f1878009cd0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 -- 172.21.15.16:0/3483804775 shutdown_connections 2024-05-08T23:32:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 --2- 172.21.15.16:0/3483804775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1888106290 0x7f188810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 --2- 172.21.15.16:0/3483804775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18880682e0 0x7f18880686c0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 --2- 172.21.15.16:0/3483804775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1888068c90 0x7f1888105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 -- 172.21.15.16:0/3483804775 >> 172.21.15.16:0/3483804775 conn(0x7f1888076f40 msgr2=0x7f1888077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 -- 172.21.15.16:0/3483804775 shutdown_connections 2024-05-08T23:32:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.035+0000 7f188e8d3700 1 -- 172.21.15.16:0/3483804775 wait complete. 2024-05-08T23:32:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 Processor -- start 2024-05-08T23:32:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 -- start start 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18880682e0 0x7f18880ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1888106290 0x7f18881003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 0x7f18881a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18881a5a90 con 0x7f18881008e0 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18881a5c00 con 0x7f1888106290 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188e8d3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18881a5dc0 con 0x7f18880682e0 2024-05-08T23:32:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188ce70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 0x7f18881a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188ce70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 0x7f18881a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35436/0 (socket says 172.21.15.16:35436) 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f188ce70700 1 -- 172.21.15.16:0/713826571 learned_addr learned my addr 172.21.15.16:0/713826571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.036+0000 7f1887fff700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18880682e0 0x7f18880ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f18877fe700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1888106290 0x7f18881003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188ce70700 1 -- 172.21.15.16:0/713826571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18880682e0 msgr2=0x7f18880ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188ce70700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18880682e0 0x7f18880ffe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188ce70700 1 -- 172.21.15.16:0/713826571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1888106290 msgr2=0x7f18881003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188ce70700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1888106290 0x7f18881003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188ce70700 1 -- 172.21.15.16:0/713826571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780095f0 con 0x7f18881008e0 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188ce70700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 0x7f18881a54c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f188000ef30 tx=0x7f188000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880018070 con 0x7f18881008e0 2024-05-08T23:32:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f188000f040 con 0x7f18881008e0 2024-05-08T23:32:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18881a6040 con 0x7f18881008e0 2024-05-08T23:32:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000cc70 con 0x7f18881008e0 2024-05-08T23:32:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.037+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18881a6550 con 0x7f18881008e0 2024-05-08T23:32:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.038+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18881094d0 con 0x7f18881008e0 2024-05-08T23:32:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.040+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1880003a00 con 0x7f18881008e0 2024-05-08T23:32:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.040+0000 7f18857fa700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18700a1310 0x7f18700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.041+0000 7f1887fff700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18700a1310 0x7f18700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.041+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f18800cd190 con 0x7f18881008e0 2024-05-08T23:32:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.041+0000 7f1887fff700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18700a1310 0x7f18700a37d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f187800b580 tx=0x7f1878019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.045+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1880096280 con 0x7f18881008e0 2024-05-08T23:32:13.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:12 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3849203475' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:32:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:12 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1561909448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:13.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:12 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/4127414879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:32:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.264+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1888101340 con 0x7f18881008e0 2024-05-08T23:32:13.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.265+0000 7f18857fa700 1 -- 172.21.15.16:0/713826571 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18800998a0 con 0x7f18881008e0 2024-05-08T23:32:13.264 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:32:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18700a1310 msgr2=0x7f18700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18700a1310 0x7f18700a37d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f187800b580 tx=0x7f1878019040 comp rx=0 tx=0).stop 2024-05-08T23:32:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 msgr2=0x7f18881a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 0x7f18881a54c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f188000ef30 tx=0x7f188000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 shutdown_connections 2024-05-08T23:32:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f18700a1310 0x7f18700a37d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18881008e0 0x7f18881a54c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18880682e0 0x7f18880ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 --2- 172.21.15.16:0/713826571 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1888106290 0x7f18881003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 >> 172.21.15.16:0/713826571 conn(0x7f1888076f40 msgr2=0x7f18880fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 shutdown_connections 2024-05-08T23:32:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.267+0000 7f188e8d3700 1 -- 172.21.15.16:0/713826571 wait complete. 2024-05-08T23:32:13.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3849203475' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:32:13.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1561909448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:13.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:12 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/4127414879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:32:13.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T23:32:13.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:12 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3849203475' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:32:13.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:12 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1561909448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:13.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:12 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/4127414879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T23:32:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 -- 172.21.15.16:0/4272777052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c100060 msgr2=0x7f908c100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 --2- 172.21.15.16:0/4272777052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c100060 0x7f908c100440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90800098f0 tx=0x7f9080009c00 comp rx=0 tx=0).stop 2024-05-08T23:32:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 -- 172.21.15.16:0/4272777052 shutdown_connections 2024-05-08T23:32:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 --2- 172.21.15.16:0/4272777052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c105180 0x7f908c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 --2- 172.21.15.16:0/4272777052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c100060 0x7f908c100440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 --2- 172.21.15.16:0/4272777052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c100a10 0x7f908c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 -- 172.21.15.16:0/4272777052 >> 172.21.15.16:0/4272777052 conn(0x7f908c076f70 msgr2=0x7f908c077380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 -- 172.21.15.16:0/4272777052 shutdown_connections 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.423+0000 7f9092e6b700 1 -- 172.21.15.16:0/4272777052 wait complete. 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 Processor -- start 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 -- start start 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100060 0x7f908c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c105180 0x7f908c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 0x7f908c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19e870 con 0x7f908c19ddf0 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19e9e0 con 0x7f908c105180 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9092e6b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c19eba0 con 0x7f908c100060 2024-05-08T23:32:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9091408700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 0x7f908c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9091408700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 0x7f908c19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35458/0 (socket says 172.21.15.16:35458) 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f908bfff700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c105180 0x7f908c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9091408700 1 -- 172.21.15.16:0/3768333411 learned_addr learned my addr 172.21.15.16:0/3768333411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9090c07700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100060 0x7f908c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f908bfff700 1 -- 172.21.15.16:0/3768333411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100060 msgr2=0x7f908c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f908bfff700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100060 0x7f908c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f908bfff700 1 -- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 msgr2=0x7f908c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f908bfff700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 0x7f908c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f908bfff700 1 -- 172.21.15.16:0/3768333411 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780095f0 con 0x7f908c105180 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.424+0000 7f9091408700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 0x7f908c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.425+0000 7f908bfff700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c105180 0x7f908c199a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9078005fd0 tx=0x7f907800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.425+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907800cbd0 con 0x7f908c105180 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.425+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9080009520 con 0x7f908c105180 2024-05-08T23:32:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.425+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90780044b0 con 0x7f908c105180 2024-05-08T23:32:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.425+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078003d50 con 0x7f908c105180 2024-05-08T23:32:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.425+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c1a64a0 con 0x7f908c105180 2024-05-08T23:32:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.426+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c101530 con 0x7f908c105180 2024-05-08T23:32:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.428+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9078003ef0 con 0x7f908c105180 2024-05-08T23:32:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.428+0000 7f9089ffb700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f907c0a1290 0x7f907c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.429+0000 7f9090c07700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f907c0a1290 0x7f907c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.429+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f9078014070 con 0x7f908c105180 2024-05-08T23:32:13.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.429+0000 7f9090c07700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f907c0a1290 0x7f907c0a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9080009fd0 tx=0x7f9080019040 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.432+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9078096970 con 0x7f908c105180 2024-05-08T23:32:13.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.647+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f908c19a810 con 0x7f908c105180 2024-05-08T23:32:13.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.648+0000 7f9089ffb700 1 -- 172.21.15.16:0/3768333411 <== mon.2 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9078099f90 con 0x7f908c105180 2024-05-08T23:32:13.648 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f907c0a1290 msgr2=0x7f907c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f907c0a1290 0x7f907c0a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9080009fd0 tx=0x7f9080019040 comp rx=0 tx=0).stop 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c105180 msgr2=0x7f908c199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c105180 0x7f908c199a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9078005fd0 tx=0x7f907800ee70 comp rx=0 tx=0).stop 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 shutdown_connections 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f907c0a1290 0x7f907c0a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f908c19ddf0 0x7f908c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f908c100060 0x7f908c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 --2- 172.21.15.16:0/3768333411 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f908c105180 0x7f908c199a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 >> 172.21.15.16:0/3768333411 conn(0x7f908c076f70 msgr2=0x7f908c0699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 shutdown_connections 2024-05-08T23:32:13.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.651+0000 7f9092e6b700 1 -- 172.21.15.16:0/3768333411 wait complete. 2024-05-08T23:32:13.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T23:32:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 -- 172.21.15.16:0/392686584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 msgr2=0x7f808810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 --2- 172.21.15.16:0/392686584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f808810de20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f808000b0a0 tx=0x7f808000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 -- 172.21.15.16:0/392686584 shutdown_connections 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 --2- 172.21.15.16:0/392686584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f808810de20 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 --2- 172.21.15.16:0/392686584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8088101180 0x7f8088101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 --2- 172.21.15.16:0/392686584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088101b30 0x7f8088105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 -- 172.21.15.16:0/392686584 >> 172.21.15.16:0/392686584 conn(0x7f80880fc9e0 msgr2=0x7f80880fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 -- 172.21.15.16:0/392686584 shutdown_connections 2024-05-08T23:32:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.806+0000 7f8090077700 1 -- 172.21.15.16:0/392686584 wait complete. 2024-05-08T23:32:13.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 Processor -- start 2024-05-08T23:32:13.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 -- start start 2024-05-08T23:32:13.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088101180 0x7f8088197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f80881979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808819bd40 0x7f808819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808819c7c0 con 0x7f80881062a0 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808819c930 con 0x7f8088101180 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f8090077700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808819caf0 con 0x7f808819bd40 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f808d612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f80881979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f808d612700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f80881979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35462/0 (socket says 172.21.15.16:35462) 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f808d612700 1 -- 172.21.15.16:0/2574466444 learned_addr learned my addr 172.21.15.16:0/2574466444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f808de13700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088101180 0x7f8088197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.807+0000 7f808e614700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808819bd40 0x7f808819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f808d612700 1 -- 172.21.15.16:0/2574466444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808819bd40 msgr2=0x7f808819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f808d612700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808819bd40 0x7f808819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f808d612700 1 -- 172.21.15.16:0/2574466444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088101180 msgr2=0x7f8088197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f808d612700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088101180 0x7f8088197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f808d612700 1 -- 172.21.15.16:0/2574466444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009d00 con 0x7f80881062a0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f808d612700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f80881979a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f807800d980 tx=0x7f807800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078009930 con 0x7f80881062a0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8078010460 con 0x7f80881062a0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80881a3ff0 con 0x7f80881062a0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800f620 con 0x7f80881062a0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.808+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80881a44d0 con 0x7f80881062a0 2024-05-08T23:32:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.809+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8088109570 con 0x7f80881062a0 2024-05-08T23:32:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.811+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f80780105d0 con 0x7f80881062a0 2024-05-08T23:32:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.812+0000 7f807effd700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80740a1290 0x7f80740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.812+0000 7f808de13700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80740a1290 0x7f80740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.812+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f80780ccb00 con 0x7f80881062a0 2024-05-08T23:32:13.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.812+0000 7f808de13700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80740a1290 0x7f80740a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8084005950 tx=0x7f808400b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:13.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:13.815+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8078095b00 con 0x7f80881062a0 2024-05-08T23:32:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.035+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8088198760 con 0x7f80881062a0 2024-05-08T23:32:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.035+0000 7f807effd700 1 -- 172.21.15.16:0/2574466444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80780170a0 con 0x7f80881062a0 2024-05-08T23:32:14.035 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-05-08T23:32:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80740a1290 msgr2=0x7f80740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80740a1290 0x7f80740a3750 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8084005950 tx=0x7f808400b410 comp rx=0 tx=0).stop 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 msgr2=0x7f80881979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f80881979a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f807800d980 tx=0x7f807800dc90 comp rx=0 tx=0).stop 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 shutdown_connections 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f80740a1290 0x7f80740a3750 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808819bd40 0x7f808819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8088101180 0x7f8088197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 --2- 172.21.15.16:0/2574466444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80881062a0 0x7f80881979a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 >> 172.21.15.16:0/2574466444 conn(0x7f80880fc9e0 msgr2=0x7f80880fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 shutdown_connections 2024-05-08T23:32:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.038+0000 7f8090077700 1 -- 172.21.15.16:0/2574466444 wait complete. 2024-05-08T23:32:14.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T23:32:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 -- 172.21.15.16:0/3530177989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101b30 msgr2=0x7f22d8105b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 --2- 172.21.15.16:0/3530177989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101b30 0x7f22d8105b70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009990 tx=0x7f22c8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 -- 172.21.15.16:0/3530177989 shutdown_connections 2024-05-08T23:32:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 --2- 172.21.15.16:0/3530177989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 0x7f22d810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 --2- 172.21.15.16:0/3530177989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d8101180 0x7f22d8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 --2- 172.21.15.16:0/3530177989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101b30 0x7f22d8105b70 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 -- 172.21.15.16:0/3530177989 >> 172.21.15.16:0/3530177989 conn(0x7f22d80fc9e0 msgr2=0x7f22d80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 -- 172.21.15.16:0/3530177989 shutdown_connections 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 -- 172.21.15.16:0/3530177989 wait complete. 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 Processor -- start 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.194+0000 7f22dfa58700 1 -- start start 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 0x7f22d8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 0x7f22d8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d819dd70 0x7f22d819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d819e7f0 con 0x7f22d8101180 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d819e960 con 0x7f22d819dd70 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d819eb20 con 0x7f22d81062a0 2024-05-08T23:32:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 0x7f22d8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 0x7f22d8199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35488/0 (socket says 172.21.15.16:35488) 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 -- 172.21.15.16:0/2733636776 learned_addr learned my addr 172.21.15.16:0/2733636776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dcff3700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 0x7f22d8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 -- 172.21.15.16:0/2733636776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 msgr2=0x7f22d8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 0x7f22d8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 -- 172.21.15.16:0/2733636776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d819dd70 msgr2=0x7f22d819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d819dd70 0x7f22d819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 -- 172.21.15.16:0/2733636776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80095f0 con 0x7f22d8101180 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dcff3700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 0x7f22d8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dd7f4700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 0x7f22d8199500 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f22d400d980 tx=0x7f22d400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400bb60 con 0x7f22d8101180 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d81a6080 con 0x7f22d8101180 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22d4007640 con 0x7f22d8101180 2024-05-08T23:32:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.195+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4004d60 con 0x7f22d8101180 2024-05-08T23:32:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.196+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d81a6560 con 0x7f22d8101180 2024-05-08T23:32:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.196+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d8102650 con 0x7f22d8101180 2024-05-08T23:32:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.198+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f22d4004020 con 0x7f22d8101180 2024-05-08T23:32:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.198+0000 7f22ce7fc700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22c40a1290 0x7f22c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.199+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9534+0+0 (secure 0 0 0) 0x7f22d40cd470 con 0x7f22d8101180 2024-05-08T23:32:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.199+0000 7f22dcff3700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22c40a1290 0x7f22c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.199+0000 7f22dcff3700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22c40a1290 0x7f22c40a3750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c8005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.202+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f22d40964d0 con 0x7f22d8101180 2024-05-08T23:32:14.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:13 smithi155 ceph-mon[47105]: pgmap v395: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-05-08T23:32:14.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:13 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/713826571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:32:14.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:13 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3768333411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:32:14.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:13 smithi016 ceph-mon[42541]: pgmap v395: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-05-08T23:32:14.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:13 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/713826571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:32:14.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:13 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3768333411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:32:14.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:13 smithi189 ceph-mon[46919]: pgmap v395: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-05-08T23:32:14.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:13 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/713826571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T23:32:14.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:13 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3768333411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T23:32:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.439+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f22d8067cf0 con 0x7f22d8101180 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.439+0000 7f22ce7fc700 1 -- 172.21.15.16:0/2733636776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v99) v1 ==== 81+0+997 (secure 0 0 0) 0x7f22d4016020 con 0x7f22d8101180 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4093375965 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1044015911 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4149284949 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4028430980 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/745120666 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3684081056 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1837768303 2024-05-09T23:12:19.352750+0000 2024-05-08T23:32:14.439 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/592052660 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1569877251 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/972303159 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/30115990 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/4190536936 2024-05-09T23:12:42.499707+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3133644606 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/781396085 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.440 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/781396085 2024-05-09T23:19:37.917704+0000 2024-05-08T23:32:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22c40a1290 msgr2=0x7f22c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22c40a1290 0x7f22c40a3750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f22c8000c00 tx=0x7f22c8005fb0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 msgr2=0x7f22d8199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 0x7f22d8199500 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f22d400d980 tx=0x7f22d400dd40 comp rx=0 tx=0).stop 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 shutdown_connections 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f22c40a1290 0x7f22c40a3750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22d819dd70 0x7f22d819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22d8101180 0x7f22d8199500 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 --2- 172.21.15.16:0/2733636776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22d81062a0 0x7f22d8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 >> 172.21.15.16:0/2733636776 conn(0x7f22d80fc9e0 msgr2=0x7f22d80fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 shutdown_connections 2024-05-08T23:32:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:14.442+0000 7f22dfa58700 1 -- 172.21.15.16:0/2733636776 wait complete. 2024-05-08T23:32:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:listed 17 entries 2024-05-08T23:32:14.455 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-05-08T23:32:14.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:14.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T23:32:14.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111907]: fuse finished with error 0 and tester_r 0 2024-05-08T23:32:14.499 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T23:32:15.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:14 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2574466444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:32:15.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:14 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/2733636776' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:32:15.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:14 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2574466444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:32:15.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:14 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/2733636776' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:32:15.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:14 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2574466444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T23:32:15.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:14 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/2733636776' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T23:32:16.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:15 smithi016 ceph-mon[42541]: pgmap v396: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:16.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:15 smithi189 ceph-mon[46919]: pgmap v396: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:16.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:15 smithi155 ceph-mon[47105]: pgmap v396: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:18.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:17 smithi016 ceph-mon[42541]: pgmap v397: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:18.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:17 smithi189 ceph-mon[46919]: pgmap v397: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:18.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:17 smithi155 ceph-mon[47105]: pgmap v397: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:19 smithi016 ceph-mon[42541]: pgmap v398: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:20.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:19 smithi016 ceph-mon[42541]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:32:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:19 smithi189 ceph-mon[46919]: pgmap v398: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:20.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:19 smithi189 ceph-mon[46919]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:32:20.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:19 smithi155 ceph-mon[47105]: pgmap v398: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-08T23:32:20.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:19 smithi155 ceph-mon[47105]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T23:32:20.601 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:32:20.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:20.601 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T23:32:20.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:20.624 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-05-08T23:32:20.688 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T23:32:20.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:20.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T23:32:20.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T23:32:20.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-05-08T23:32:20.860 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T23:32:20.870 INFO:tasks.cephadm:Teardown begin 2024-05-08T23:32:20.870 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T23:32:20.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:21.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 -- 172.21.15.16:0/2488410216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04581028c0 msgr2=0x7f045810ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 --2- 172.21.15.16:0/2488410216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04581028c0 0x7f045810ada0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0444009990 tx=0x7f0444009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 -- 172.21.15.16:0/2488410216 shutdown_connections 2024-05-08T23:32:21.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 --2- 172.21.15.16:0/2488410216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045810b3c0 0x7f045810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 --2- 172.21.15.16:0/2488410216 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0458101fa0 0x7f0458102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 --2- 172.21.15.16:0/2488410216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04581028c0 0x7f045810ada0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 -- 172.21.15.16:0/2488410216 >> 172.21.15.16:0/2488410216 conn(0x7f04580fb7e0 msgr2=0x7f04580fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 -- 172.21.15.16:0/2488410216 shutdown_connections 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 -- 172.21.15.16:0/2488410216 wait complete. 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.018+0000 7f045ec14700 1 Processor -- start 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 -- start start 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0458101fa0 0x7f04580ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04581028c0 0x7f0458100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 0x7f0458100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581a5600 con 0x7f045810b3c0 2024-05-08T23:32:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581a5770 con 0x7f0458101fa0 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045ec14700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581a5930 con 0x7f04581028c0 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 0x7f0458100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 0x7f0458100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34060/0 (socket says 172.21.15.16:34060) 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 -- 172.21.15.16:0/1325673767 learned_addr learned my addr 172.21.15.16:0/1325673767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045c9b0700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0458101fa0 0x7f04580ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f0457fff700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04581028c0 0x7f0458100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 -- 172.21.15.16:0/1325673767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04581028c0 msgr2=0x7f0458100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04581028c0 0x7f0458100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 -- 172.21.15.16:0/1325673767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0458101fa0 msgr2=0x7f04580ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0458101fa0 0x7f04580ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.019+0000 7f045d1b1700 1 -- 172.21.15.16:0/1325673767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04440095f0 con 0x7f045810b3c0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.020+0000 7f045d1b1700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 0x7f0458100670 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f045000ef30 tx=0x7f045000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.020+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450018070 con 0x7f045810b3c0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.020+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04581a5bb0 con 0x7f045810b3c0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.020+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f045000f040 con 0x7f045810b3c0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.020+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045000cc70 con 0x7f045810b3c0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.020+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04581a6070 con 0x7f045810b3c0 2024-05-08T23:32:21.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.021+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0458106490 con 0x7f045810b3c0 2024-05-08T23:32:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.022+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0450007500 con 0x7f045810b3c0 2024-05-08T23:32:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.023+0000 7f0455ffb700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04480a1290 0x7f04480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.023+0000 7f045c9b0700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04480a1290 0x7f04480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.023+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f04500ccbd0 con 0x7f045810b3c0 2024-05-08T23:32:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.023+0000 7f045c9b0700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04480a1290 0x7f04480a3750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f044c007be0 tx=0x7f044c0073d0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.027+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0450095d10 con 0x7f045810b3c0 2024-05-08T23:32:21.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.296+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04580688d0 con 0x7f045810b3c0 2024-05-08T23:32:21.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.296+0000 7f0455ffb700 1 -- 172.21.15.16:0/1325673767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f0450099330 con 0x7f045810b3c0 2024-05-08T23:32:21.296 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:32:21.296 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04480a1290 msgr2=0x7f04480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04480a1290 0x7f04480a3750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f044c007be0 tx=0x7f044c0073d0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 msgr2=0x7f0458100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 0x7f0458100670 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f045000ef30 tx=0x7f045000c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 shutdown_connections 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f04480a1290 0x7f04480a3750 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f045810b3c0 0x7f0458100670 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0458101fa0 0x7f04580ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 --2- 172.21.15.16:0/1325673767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04581028c0 0x7f0458100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 >> 172.21.15.16:0/1325673767 conn(0x7f04580fb7e0 msgr2=0x7f04580fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 shutdown_connections 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.299+0000 7f045ec14700 1 -- 172.21.15.16:0/1325673767 wait complete. 2024-05-08T23:32:21.299 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:32:21.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T23:32:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 -- 172.21.15.16:0/2668653303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040fff40 msgr2=0x7f5a04100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 --2- 172.21.15.16:0/2668653303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040fff40 0x7f5a04100320 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009990 tx=0x7f59f8009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 -- 172.21.15.16:0/2668653303 shutdown_connections 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 --2- 172.21.15.16:0/2668653303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a04105060 0x7f5a0410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 --2- 172.21.15.16:0/2668653303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a040fff40 0x7f5a04100320 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 --2- 172.21.15.16:0/2668653303 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a041008f0 0x7f5a04104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 -- 172.21.15.16:0/2668653303 >> 172.21.15.16:0/2668653303 conn(0x7f5a040fb7e0 msgr2=0x7f5a040fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 -- 172.21.15.16:0/2668653303 shutdown_connections 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.450+0000 7f5a0af39700 1 -- 172.21.15.16:0/2668653303 wait complete. 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 Processor -- start 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 -- start start 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a040fff40 0x7f5a04197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 0x7f5a041977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0419bb00 0x7f5a0419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0419c580 con 0x7f5a04105060 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0419c6f0 con 0x7f5a0419bb00 2024-05-08T23:32:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a0af39700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0419c8b0 con 0x7f5a040fff40 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 0x7f5a041977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 0x7f5a041977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34082/0 (socket says 172.21.15.16:34082) 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 -- 172.21.15.16:0/1843962618 learned_addr learned my addr 172.21.15.16:0/1843962618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a08cd5700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a040fff40 0x7f5a04197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a094d6700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0419bb00 0x7f5a0419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 -- 172.21.15.16:0/1843962618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a040fff40 msgr2=0x7f5a04197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a040fff40 0x7f5a04197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 -- 172.21.15.16:0/1843962618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0419bb00 msgr2=0x7f5a0419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0419bb00 0x7f5a0419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.451+0000 7f5a03fff700 1 -- 172.21.15.16:0/1843962618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80095f0 con 0x7f5a04105060 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.452+0000 7f5a03fff700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 0x7f5a041977d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e970 tx=0x7f59f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.452+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0009790 con 0x7f5a04105060 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.452+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59f000eec0 con 0x7f5a04105060 2024-05-08T23:32:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.452+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a041a3e10 con 0x7f5a04105060 2024-05-08T23:32:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.452+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00054b0 con 0x7f5a04105060 2024-05-08T23:32:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.452+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a041a42e0 con 0x7f5a04105060 2024-05-08T23:32:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.453+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a04101410 con 0x7f5a04105060 2024-05-08T23:32:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.454+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f59f0004750 con 0x7f5a04105060 2024-05-08T23:32:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.455+0000 7f5a01ffb700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f59f40a1310 0x7f59f40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.455+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9446+0+0 (secure 0 0 0) 0x7f59f0014070 con 0x7f5a04105060 2024-05-08T23:32:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.455+0000 7f5a08cd5700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f59f40a1310 0x7f59f40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.455+0000 7f5a08cd5700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f59f40a1310 0x7f59f40a37d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006010 tx=0x7f59f800b260 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.459+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f59f0096300 con 0x7f5a04105060 2024-05-08T23:32:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.728+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a04067c10 con 0x7f5a04105060 2024-05-08T23:32:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.729+0000 7f5a01ffb700 1 -- 172.21.15.16:0/1843962618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10877 (secure 0 0 0) 0x7f59f0099920 con 0x7f5a04105060 2024-05-08T23:32:21.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:32:21.728 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14568,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4018978287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":4018978287},{"type":"v1","addr":"172.21.15.16:6835","nonce":4018978287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/495357433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":495357433},{"type":"v1","addr":"172.21.15.16:6839","nonce":495357433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2654183877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2654183877},{"type":"v1","addr":"172.21.15.16:6841","nonce":2654183877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2420649836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2420649836},{"type":"v1","addr":"172.21.15.155:6833","nonce":2420649836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24469,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/3253311334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3253311334},{"type":"v1","addr":"172.21.15.189:6833","nonce":3253311334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/3189374355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":3189374355},{"type":"v1","addr":"172.21.15.16:6837","nonce":3189374355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T23:17:46.078099+0000","modified":"2024-05-08T23:19:39.926299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24563,"mds_1":24550,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24497":{"gid":24497,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6835/1260766455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1260766455},{"type":"v1","addr":"172.21.15.155:6835","nonce":1260766455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24517":{"gid":24517,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3239398256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3239398256},{"type":"v1","addr":"172.21.15.189:6835","nonce":3239398256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.155:6837/1960878524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6836","nonce":1960878524},{"type":"v1","addr":"172.21.15.155:6837","nonce":1960878524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.189:6837/2098584238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2098584238},{"type":"v1","addr":"172.21.15.189:6837","nonce":2098584238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.189:6839/3614950817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":3614950817},{"type":"v1","addr":"172.21.15.189:6839","nonce":3614950817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6839/773665585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6838","nonce":773665585},{"type":"v1","addr":"172.21.15.155:6839","nonce":773665585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f59f40a1310 msgr2=0x7f59f40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f59f40a1310 0x7f59f40a37d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006010 tx=0x7f59f800b260 comp rx=0 tx=0).stop 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 msgr2=0x7f5a041977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 0x7f5a041977d0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f59f000e970 tx=0x7f59f000ec80 comp rx=0 tx=0).stop 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 shutdown_connections 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f59f40a1310 0x7f59f40a37d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a0419bb00 0x7f5a0419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a040fff40 0x7f5a04197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 --2- 172.21.15.16:0/1843962618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a04105060 0x7f5a041977d0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 >> 172.21.15.16:0/1843962618 conn(0x7f5a040fb7e0 msgr2=0x7f5a04068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 shutdown_connections 2024-05-08T23:32:21.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.732+0000 7f5a0af39700 1 -- 172.21.15.16:0/1843962618 wait complete. 2024-05-08T23:32:21.732 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-05-08T23:32:21.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T23:32:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 -- 172.21.15.16:0/2902408848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c101fa0 msgr2=0x7fab1c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 --2- 172.21.15.16:0/2902408848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c101fa0 0x7fab1c102380 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fab18009990 tx=0x7fab18009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 -- 172.21.15.16:0/2902408848 shutdown_connections 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 --2- 172.21.15.16:0/2902408848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab1c10b3c0 0x7fab1c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 --2- 172.21.15.16:0/2902408848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c101fa0 0x7fab1c102380 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 --2- 172.21.15.16:0/2902408848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab1c1028c0 0x7fab1c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 -- 172.21.15.16:0/2902408848 >> 172.21.15.16:0/2902408848 conn(0x7fab1c0fb7e0 msgr2=0x7fab1c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 -- 172.21.15.16:0/2902408848 shutdown_connections 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.883+0000 7fab2467d700 1 -- 172.21.15.16:0/2902408848 wait complete. 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 Processor -- start 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 -- start start 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab1c101fa0 0x7fab1c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab1c1028c0 0x7fab1c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 0x7fab1c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c19c100 con 0x7fab1c10b3c0 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c19c270 con 0x7fab1c101fa0 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab2467d700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c19c430 con 0x7fab1c1028c0 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab22c1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 0x7fab1c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab22c1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 0x7fab1c19bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34100/0 (socket says 172.21.15.16:34100) 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab22c1a700 1 -- 172.21.15.16:0/3104226202 learned_addr learned my addr 172.21.15.16:0/3104226202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab22419700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab1c101fa0 0x7fab1c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.884+0000 7fab21c18700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab1c1028c0 0x7fab1c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab22c1a700 1 -- 172.21.15.16:0/3104226202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab1c1028c0 msgr2=0x7fab1c197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab22c1a700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab1c1028c0 0x7fab1c197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab22c1a700 1 -- 172.21.15.16:0/3104226202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab1c101fa0 msgr2=0x7fab1c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab22c1a700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab1c101fa0 0x7fab1c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab22c1a700 1 -- 172.21.15.16:0/3104226202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180095f0 con 0x7fab1c10b3c0 2024-05-08T23:32:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab22c1a700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 0x7fab1c19bac0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fab1400ef30 tx=0x7fab1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab14018070 con 0x7fab1c10b3c0 2024-05-08T23:32:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab1400f040 con 0x7fab1c10b3c0 2024-05-08T23:32:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1400cbc0 con 0x7fab1c10b3c0 2024-05-08T23:32:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c19c6b0 con 0x7fab1c10b3c0 2024-05-08T23:32:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.885+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a3f50 con 0x7fab1c10b3c0 2024-05-08T23:32:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.886+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c1064a0 con 0x7fab1c10b3c0 2024-05-08T23:32:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.887+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fab1400cd40 con 0x7fab1c10b3c0 2024-05-08T23:32:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.888+0000 7fab0f7fe700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab080a1310 0x7fab080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.888+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9446+0+0 (secure 0 0 0) 0x7fab140cd080 con 0x7fab1c10b3c0 2024-05-08T23:32:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.888+0000 7fab22419700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab080a1310 0x7fab080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.889+0000 7fab22419700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab080a1310 0x7fab080a37d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab18006010 tx=0x7fab18005960 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:21.892+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fab14096160 con 0x7fab1c10b3c0 2024-05-08T23:32:22.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.110+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab1c198580 con 0x7fab1c10b3c0 2024-05-08T23:32:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.112+0000 7fab0f7fe700 1 -- 172.21.15.16:0/3104226202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+25184 (secure 0 0 0) 0x7fab14099780 con 0x7fab1c10b3c0 2024-05-08T23:32:22.112 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-05-08T23:32:22.112 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"378de17a-0d90-11ef-bc97-c7b262605968","created":"2024-05-08T23:11:54.944695+0000","modified":"2024-05-08T23:32:18.748702+0000","last_up_change":"2024-05-08T23:17:10.291562+0000","last_in_change":"2024-05-08T23:16:52.879214+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T23:14:33.043373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T23:17:44.419250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T23:17:45.451965+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T23:17:48.549788+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c7c44106-3485-4f2e-b9ca-fdd6db13f996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6803","nonce":3035598518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6805","nonce":3035598518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6809","nonce":3035598518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3035598518},{"type":"v1","addr":"172.21.15.16:6807","nonce":3035598518}]},"public_addr":"172.21.15.16:6803/3035598518","cluster_addr":"172.21.15.16:6805/3035598518","heartbeat_back_addr":"172.21.15.16:6809/3035598518","heartbeat_front_addr":"172.21.15.16:6807/3035598518","state":["exists","up"]},{"osd":1,"uuid":"e8db4eaa-2c4b-4792-bb92-584dc7e6dc7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6801","nonce":2944272573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6803","nonce":2944272573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6807","nonce":2944272573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":2944272573},{"type":"v1","addr":"172.21.15.155:6805","nonce":2944272573}]},"public_addr":"172.21.15.155:6801/2944272573","cluster_addr":"172.21.15.155:6803/2944272573","heartbeat_back_addr":"172.21.15.155:6807/2944272573","heartbeat_front_addr":"172.21.15.155:6805/2944272573","state":["exists","up"]},{"osd":2,"uuid":"1a5d9e16-e1c7-4880-900c-e886b3e9c00d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6801","nonce":3853442888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6803","nonce":3853442888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6807","nonce":3853442888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3853442888},{"type":"v1","addr":"172.21.15.189:6805","nonce":3853442888}]},"public_addr":"172.21.15.189:6801/3853442888","cluster_addr":"172.21.15.189:6803/3853442888","heartbeat_back_addr":"172.21.15.189:6807/3853442888","heartbeat_front_addr":"172.21.15.189:6805/3853442888","state":["exists","up"]},{"osd":3,"uuid":"810b101a-3d2e-4761-be92-dc7bc786a11c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6811","nonce":2397977008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6813","nonce":2397977008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6817","nonce":2397977008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":2397977008},{"type":"v1","addr":"172.21.15.16:6815","nonce":2397977008}]},"public_addr":"172.21.15.16:6811/2397977008","cluster_addr":"172.21.15.16:6813/2397977008","heartbeat_back_addr":"172.21.15.16:6817/2397977008","heartbeat_front_addr":"172.21.15.16:6815/2397977008","state":["exists","up"]},{"osd":4,"uuid":"d1578371-9303-4583-95e0-6c7812f396ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6809","nonce":2036488025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6811","nonce":2036488025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6815","nonce":2036488025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":2036488025},{"type":"v1","addr":"172.21.15.155:6813","nonce":2036488025}]},"public_addr":"172.21.15.155:6809/2036488025","cluster_addr":"172.21.15.155:6811/2036488025","heartbeat_back_addr":"172.21.15.155:6815/2036488025","heartbeat_front_addr":"172.21.15.155:6813/2036488025","state":["exists","up"]},{"osd":5,"uuid":"b6855d48-5018-4f4a-a7e6-d97ce06a38e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6809","nonce":925680266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6811","nonce":925680266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6815","nonce":925680266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":925680266},{"type":"v1","addr":"172.21.15.189:6813","nonce":925680266}]},"public_addr":"172.21.15.189:6809/925680266","cluster_addr":"172.21.15.189:6811/925680266","heartbeat_back_addr":"172.21.15.189:6815/925680266","heartbeat_front_addr":"172.21.15.189:6813/925680266","state":["exists","up"]},{"osd":6,"uuid":"6f82436c-9ec7-4fb4-9c5d-5d815b86a408","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6819","nonce":3858453894}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6821","nonce":3858453894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6825","nonce":3858453894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3858453894},{"type":"v1","addr":"172.21.15.16:6823","nonce":3858453894}]},"public_addr":"172.21.15.16:6819/3858453894","cluster_addr":"172.21.15.16:6821/3858453894","heartbeat_back_addr":"172.21.15.16:6825/3858453894","heartbeat_front_addr":"172.21.15.16:6823/3858453894","state":["exists","up"]},{"osd":7,"uuid":"96cf8304-3a9c-49d8-8789-543a1ddcc7d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6817","nonce":283518572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6819","nonce":283518572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6823","nonce":283518572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6820","nonce":283518572},{"type":"v1","addr":"172.21.15.155:6821","nonce":283518572}]},"public_addr":"172.21.15.155:6817/283518572","cluster_addr":"172.21.15.155:6819/283518572","heartbeat_back_addr":"172.21.15.155:6823/283518572","heartbeat_front_addr":"172.21.15.155:6821/283518572","state":["exists","up"]},{"osd":8,"uuid":"2a625c37-f744-4a80-ac8f-44990dbfd83d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6817","nonce":2123508255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6819","nonce":2123508255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6823","nonce":2123508255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2123508255},{"type":"v1","addr":"172.21.15.189:6821","nonce":2123508255}]},"public_addr":"172.21.15.189:6817/2123508255","cluster_addr":"172.21.15.189:6819/2123508255","heartbeat_back_addr":"172.21.15.189:6823/2123508255","heartbeat_front_addr":"172.21.15.189:6821/2123508255","state":["exists","up"]},{"osd":9,"uuid":"1821a996-5473-46aa-8f43-49fadd38fbe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6827","nonce":3426310247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6829","nonce":3426310247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6833","nonce":3426310247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3426310247},{"type":"v1","addr":"172.21.15.16:6831","nonce":3426310247}]},"public_addr":"172.21.15.16:6827/3426310247","cluster_addr":"172.21.15.16:6829/3426310247","heartbeat_back_addr":"172.21.15.16:6833/3426310247","heartbeat_front_addr":"172.21.15.16:6831/3426310247","state":["exists","up"]},{"osd":10,"uuid":"913decb0-6802-4e12-ace9-bdd7f341456e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6824","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6825","nonce":1164143288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6827","nonce":1164143288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6831","nonce":1164143288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6828","nonce":1164143288},{"type":"v1","addr":"172.21.15.155:6829","nonce":1164143288}]},"public_addr":"172.21.15.155:6825/1164143288","cluster_addr":"172.21.15.155:6827/1164143288","heartbeat_back_addr":"172.21.15.155:6831/1164143288","heartbeat_front_addr":"172.21.15.155:6829/1164143288","state":["exists","up"]},{"osd":11,"uuid":"72ca4e93-8e58-41a1-8b0b-e08b2a73d2bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6825","nonce":2978765234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6827","nonce":2978765234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6831","nonce":2978765234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2978765234},{"type":"v1","addr":"172.21.15.189:6829","nonce":2978765234}]},"public_addr":"172.21.15.189:6825/2978765234","cluster_addr":"172.21.15.189:6827/2978765234","heartbeat_back_addr":"172.21.15.189:6831/2978765234","heartbeat_front_addr":"172.21.15.189:6829/2978765234","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:13:55.567527+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:12.585597+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:29.882446+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:14:47.753147+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:04.559447+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:21.936129+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:39.595900+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:15:56.987449+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:14.602200+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:32.630373+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:16:49.869268+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T23:17:07.583407+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/4093375965":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/1044015911":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4149284949":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/4028430980":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/745120666":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/3684081056":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/1837768303":"2024-05-09T23:12:19.352750+0000","172.21.15.16:0/592052660":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/1569877251":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/972303159":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/30115990":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/4190536936":"2024-05-09T23:12:42.499707+0000","172.21.15.16:0/3133644606":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6801/781396085":"2024-05-09T23:19:37.917704+0000","172.21.15.16:6800/781396085":"2024-05-09T23:19:37.917704+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T23:32:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:21 smithi016 ceph-mon[42541]: pgmap v400: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 777 B/s wr, 7 op/s 2024-05-08T23:32:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:21 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1325673767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:21 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1843962618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab080a1310 msgr2=0x7fab080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab080a1310 0x7fab080a37d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fab18006010 tx=0x7fab18005960 comp rx=0 tx=0).stop 2024-05-08T23:32:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 msgr2=0x7fab1c19bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 0x7fab1c19bac0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fab1400ef30 tx=0x7fab1400c3a0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 shutdown_connections 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fab080a1310 0x7fab080a37d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab1c10b3c0 0x7fab1c19bac0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab1c101fa0 0x7fab1c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 --2- 172.21.15.16:0/3104226202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab1c1028c0 0x7fab1c197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 >> 172.21.15.16:0/3104226202 conn(0x7fab1c0fb7e0 msgr2=0x7fab1c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 shutdown_connections 2024-05-08T23:32:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.115+0000 7fab2467d700 1 -- 172.21.15.16:0/3104226202 wait complete. 2024-05-08T23:32:22.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T23:32:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:21 smithi189 ceph-mon[46919]: pgmap v400: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 777 B/s wr, 7 op/s 2024-05-08T23:32:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:21 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1325673767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:21 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1843962618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:21 smithi155 ceph-mon[47105]: pgmap v400: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 777 B/s wr, 7 op/s 2024-05-08T23:32:22.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:21 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1325673767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:21 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1843962618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 -- 172.21.15.16:0/353480174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38105180 msgr2=0x7efc3810cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 --2- 172.21.15.16:0/353480174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38105180 0x7efc3810cd00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7efc3000b0a0 tx=0x7efc3000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 -- 172.21.15.16:0/353480174 shutdown_connections 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 --2- 172.21.15.16:0/353480174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38105180 0x7efc3810cd00 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 --2- 172.21.15.16:0/353480174 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc38100060 0x7efc38100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 --2- 172.21.15.16:0/353480174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc38100a10 0x7efc38104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.268+0000 7efc3e13f700 1 -- 172.21.15.16:0/353480174 >> 172.21.15.16:0/353480174 conn(0x7efc38076fa0 msgr2=0x7efc380773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.269+0000 7efc3e13f700 1 -- 172.21.15.16:0/353480174 shutdown_connections 2024-05-08T23:32:22.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.269+0000 7efc3e13f700 1 -- 172.21.15.16:0/353480174 wait complete. 2024-05-08T23:32:22.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.269+0000 7efc3e13f700 1 Processor -- start 2024-05-08T23:32:22.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 -- start start 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc38100060 0x7efc381981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc38105180 0x7efc38198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 0x7efc381a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381a3970 con 0x7efc38198c40 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381a3ae0 con 0x7efc38100060 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc3e13f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381a3ca0 con 0x7efc38105180 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc37fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 0x7efc381a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc377fe700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc38100060 0x7efc381981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc37fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 0x7efc381a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34120/0 (socket says 172.21.15.16:34120) 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc37fff700 1 -- 172.21.15.16:0/1410728941 learned_addr learned my addr 172.21.15.16:0/1410728941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.270+0000 7efc36ffd700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc38105180 0x7efc38198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc37fff700 1 -- 172.21.15.16:0/1410728941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc38105180 msgr2=0x7efc38198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc37fff700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc38105180 0x7efc38198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc37fff700 1 -- 172.21.15.16:0/1410728941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc38100060 msgr2=0x7efc381981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc37fff700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc38100060 0x7efc381981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc37fff700 1 -- 172.21.15.16:0/1410728941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30009d00 con 0x7efc38198c40 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc37fff700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 0x7efc381a3240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7efc300094c0 tx=0x7efc300094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000e040 con 0x7efc38198c40 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc30007950 con 0x7efc38198c40 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc381a3ec0 con 0x7efc38198c40 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30005040 con 0x7efc38198c40 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.271+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc381a4340 con 0x7efc38198c40 2024-05-08T23:32:22.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.272+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc38108450 con 0x7efc38198c40 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.274+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc3001d480 con 0x7efc38198c40 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.274+0000 7efc34ff9700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc240a1290 0x7efc240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.274+0000 7efc377fe700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc240a1290 0x7efc240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.274+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9446+0+0 (secure 0 0 0) 0x7efc300ce7e0 con 0x7efc38198c40 2024-05-08T23:32:22.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.275+0000 7efc377fe700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc240a1290 0x7efc240a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ac50 tx=0x7efc2800a380 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.278+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc30097950 con 0x7efc38198c40 2024-05-08T23:32:22.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.549+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efc38068140 con 0x7efc38198c40 2024-05-08T23:32:22.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.559+0000 7efc34ff9700 1 -- 172.21.15.16:0/1410728941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7efc3009af70 con 0x7efc38198c40 2024-05-08T23:32:22.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc240a1290 msgr2=0x7efc240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc240a1290 0x7efc240a3750 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc2800ac50 tx=0x7efc2800a380 comp rx=0 tx=0).stop 2024-05-08T23:32:22.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 msgr2=0x7efc381a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 0x7efc381a3240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7efc300094c0 tx=0x7efc300094f0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 shutdown_connections 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7efc240a1290 0x7efc240a3750 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38198c40 0x7efc381a3240 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efc38100060 0x7efc381981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 --2- 172.21.15.16:0/1410728941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc38105180 0x7efc38198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 >> 172.21.15.16:0/1410728941 conn(0x7efc38076fa0 msgr2=0x7efc380fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 shutdown_connections 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.562+0000 7efc3e13f700 1 -- 172.21.15.16:0/1410728941 wait complete. 2024-05-08T23:32:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T23:32:22.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T23:32:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 -- 172.21.15.16:0/2365601618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1581062a0 msgr2=0x7fd15810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/2365601618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1581062a0 0x7fd15810de20 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd15000b0a0 tx=0x7fd15000b3b0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 -- 172.21.15.16:0/2365601618 shutdown_connections 2024-05-08T23:32:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/2365601618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1581062a0 0x7fd15810de20 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/2365601618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd158069420 0x7fd158069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/2365601618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158069dd0 0x7fd158105b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.718+0000 7fd15fd0c700 1 -- 172.21.15.16:0/2365601618 >> 172.21.15.16:0/2365601618 conn(0x7fd158076ba0 msgr2=0x7fd158076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.719+0000 7fd15fd0c700 1 -- 172.21.15.16:0/2365601618 shutdown_connections 2024-05-08T23:32:22.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.719+0000 7fd15fd0c700 1 -- 172.21.15.16:0/2365601618 wait complete. 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.719+0000 7fd15fd0c700 1 Processor -- start 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.719+0000 7fd15fd0c700 1 -- start start 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15fd0c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158069420 0x7fd158197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15fd0c700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1581062a0 0x7fd158197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15fd0c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 0x7fd15819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15fd0c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15819c770 con 0x7fd15819bcf0 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15fd0c700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15819c8e0 con 0x7fd1581062a0 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15fd0c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15819caa0 con 0x7fd158069420 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 0x7fd15819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 0x7fd15819c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34130/0 (socket says 172.21.15.16:34130) 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 -- 172.21.15.16:0/1046180916 learned_addr learned my addr 172.21.15.16:0/1046180916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15daa8700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158069420 0x7fd158197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15d2a7700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1581062a0 0x7fd158197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 -- 172.21.15.16:0/1046180916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158069420 msgr2=0x7fd158197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158069420 0x7fd158197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 -- 172.21.15.16:0/1046180916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1581062a0 msgr2=0x7fd158197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1581062a0 0x7fd158197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.720+0000 7fd15e2a9700 1 -- 172.21.15.16:0/1046180916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150009d00 con 0x7fd15819bcf0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.721+0000 7fd15e2a9700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 0x7fd15819c170 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd1500094c0 tx=0x7fd1500094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.721+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15000e040 con 0x7fd15819bcf0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.721+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd150007950 con 0x7fd15819bcf0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.721+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1581a3fa0 con 0x7fd15819bcf0 2024-05-08T23:32:22.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.721+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150005040 con 0x7fd15819bcf0 2024-05-08T23:32:22.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.721+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1581a4470 con 0x7fd15819bcf0 2024-05-08T23:32:22.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.722+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd158109570 con 0x7fd15819bcf0 2024-05-08T23:32:22.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.723+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd15001d480 con 0x7fd15819bcf0 2024-05-08T23:32:22.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.724+0000 7fd14effd700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd1440a1310 0x7fd1440a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:22.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.724+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9616+0+0 (secure 0 0 0) 0x7fd1500ce5c0 con 0x7fd15819bcf0 2024-05-08T23:32:22.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.724+0000 7fd15daa8700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd1440a1310 0x7fd1440a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:22.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.724+0000 7fd15daa8700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd1440a1310 0x7fd1440a37d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd154005950 tx=0x7fd15400b410 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:22.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.727+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd150097730 con 0x7fd15819bcf0 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a[42518]: 2024-05-08T23:32:22.550+0000 7f656d5f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3104226202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1410728941' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1410728941' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T23:32:22.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:22 smithi016 ceph-mon[42541]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T23:32:22.918 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f[83492]: ignoring --setuser ceph since I am not root 2024-05-08T23:32:22.918 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f[83492]: ignoring --setgroup ceph since I am not root 2024-05-08T23:32:22.918 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f[83492]: starting mds.f at 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3104226202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1410728941' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1410728941' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mon[46919]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T23:32:22.919 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c[83833]: ignoring --setuser ceph since I am not root 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c[83833]: ignoring --setgroup ceph since I am not root 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mds[83856]: -- 172.21.15.189:0/365646623 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e7284d41a0 con 0x55e7284ab000 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c[83833]: starting mds.c at 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-i[84175]: ignoring --setuser ceph since I am not root 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-i[84175]: ignoring --setgroup ceph since I am not root 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:32:22 smithi189 ceph-mds[84198]: -- 172.21.15.189:0/4016714193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580940021a0 con 0x558093fd9000 2024-05-08T23:32:22.920 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:32:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-i[84175]: starting mds.i at 2024-05-08T23:32:22.964 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3104226202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1410728941' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1410728941' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mon[47105]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: ignoring --setuser ceph since I am not root 2024-05-08T23:32:22.965 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: ignoring --setgroup ceph since I am not root 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mds[83862]: -- 172.21.15.155:0/2088984474 <== mon.2 v2:172.21.15.155:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56116fd7c1a0 con 0x56116fd52c00 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: starting mds.b at 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e[83153]: ignoring --setuser ceph since I am not root 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e[83153]: ignoring --setgroup ceph since I am not root 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e[83153]: starting mds.e at 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h[83495]: ignoring --setuser ceph since I am not root 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h[83495]: ignoring --setgroup ceph since I am not root 2024-05-08T23:32:22.966 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:32:22 smithi155 ceph-mds[83518]: -- 172.21.15.155:0/3293126841 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3c36c61a0 con 0x55d3c369cc00 2024-05-08T23:32:22.967 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:32:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h[83495]: starting mds.h at 2024-05-08T23:32:22.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:22.997+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd158198710 con 0x7fd15819bcf0 2024-05-08T23:32:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.562+0000 7fd14effd700 1 -- 172.21.15.16:0/1046180916 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd15009ad50 con 0x7fd15819bcf0 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd1440a1310 msgr2=0x7fd1440a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd1440a1310 0x7fd1440a37d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd154005950 tx=0x7fd15400b410 comp rx=0 tx=0).stop 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 msgr2=0x7fd15819c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 0x7fd15819c170 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd1500094c0 tx=0x7fd1500094f0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 shutdown_connections 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fd1440a1310 0x7fd1440a37d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15819bcf0 0x7fd15819c170 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.566+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158069420 0x7fd158197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.567+0000 7fd15fd0c700 1 --2- 172.21.15.16:0/1046180916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1581062a0 0x7fd158197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.567+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 >> 172.21.15.16:0/1046180916 conn(0x7fd158076ba0 msgr2=0x7fd1580fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.567+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 shutdown_connections 2024-05-08T23:32:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.567+0000 7fd15fd0c700 1 -- 172.21.15.16:0/1046180916 wait complete. 2024-05-08T23:32:23.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 -- 172.21.15.16:0/3215147948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc106290 msgr2=0x7fddbc10de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3215147948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc106290 0x7fddbc10de10 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fddb400b0d0 tx=0x7fddb400b3e0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 -- 172.21.15.16:0/3215147948 shutdown_connections 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3215147948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc106290 0x7fddbc10de10 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3215147948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddbc0682e0 0x7fddbc0686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3215147948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc068c90 0x7fddbc105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 -- 172.21.15.16:0/3215147948 >> 172.21.15.16:0/3215147948 conn(0x7fddbc076f40 msgr2=0x7fddbc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:23.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 -- 172.21.15.16:0/3215147948 shutdown_connections 2024-05-08T23:32:23.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.724+0000 7fddc20b9700 1 -- 172.21.15.16:0/3215147948 wait complete. 2024-05-08T23:32:23.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 Processor -- start 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 -- start start 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc0682e0 0x7fddbc1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddbc106290 0x7fddbc197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 0x7fddbc19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc19c750 con 0x7fddbc19bcd0 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.725+0000 7fddc20b9700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc19c8c0 con 0x7fddbc106290 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddc20b9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc19ca80 con 0x7fddbc0682e0 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 0x7fddbc19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 0x7fddbc19c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34152/0 (socket says 172.21.15.16:34152) 2024-05-08T23:32:23.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 -- 172.21.15.16:0/3888543336 learned_addr learned my addr 172.21.15.16:0/3888543336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbb7fe700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc0682e0 0x7fddbc1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbaffd700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddbc106290 0x7fddbc197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 -- 172.21.15.16:0/3888543336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc0682e0 msgr2=0x7fddbc1973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc0682e0 0x7fddbc1973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 -- 172.21.15.16:0/3888543336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddbc106290 msgr2=0x7fddbc197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddbc106290 0x7fddbc197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 -- 172.21.15.16:0/3888543336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb4009d00 con 0x7fddbc19bcd0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbb7fe700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc0682e0 0x7fddbc1973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddbbfff700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 0x7fddbc19c150 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006070 tx=0x7fddb400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb400e040 con 0x7fddbc19bcd0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddb4003e90 con 0x7fddbc19bcd0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc1a3f80 con 0x7fddbc19bcd0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.726+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb40045b0 con 0x7fddbc19bcd0 2024-05-08T23:32:23.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.727+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc1a4450 con 0x7fddbc19bcd0 2024-05-08T23:32:23.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.727+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc1094f0 con 0x7fddbc19bcd0 2024-05-08T23:32:23.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.729+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fddb4004710 con 0x7fddbc19bcd0 2024-05-08T23:32:23.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.729+0000 7fddb8ff9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdda80a1310 0x7fdda80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:23.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.730+0000 7fddbb7fe700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdda80a1310 0x7fdda80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:23.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.730+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9616+0+0 (secure 0 0 0) 0x7fddb40cee10 con 0x7fddbc19bcd0 2024-05-08T23:32:23.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.730+0000 7fddbb7fe700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdda80a1310 0x7fdda80a37d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddac005950 tx=0x7fddac0058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.733+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fddb4097e70 con 0x7fddbc19bcd0 2024-05-08T23:32:23.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: pgmap v401: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-05-08T23:32:23.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1046180916' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:32:23.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T23:32:23.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T23:32:23.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: Cluster is now healthy 2024-05-08T23:32:23.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6836/3506943982,v1:172.21.15.189:6839/3506943982] up:boot 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6834/1456153657,v1:172.21.15.155:6837/1456153657] up:boot 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6834/3348873916,v1:172.21.15.189:6837/3348873916] up:boot 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6836/143833602,v1:172.21.15.155:6839/143833602] up:boot 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.189:6835/1820655793,v1:172.21.15.189:6838/1820655793] up:boot 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: mds.? [v2:172.21.15.155:6835/2583663737,v1:172.21.15.155:6838/2583663737] up:boot 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1046180916' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: fsmap 12 up:standby 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:32:23.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:23 smithi016 ceph-mon[42541]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:32:23.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:23.953+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fddbc102640 con 0x7fddbc19bcd0 2024-05-08T23:32:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: pgmap v401: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-05-08T23:32:24.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1046180916' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: Cluster is now healthy 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6836/3506943982,v1:172.21.15.189:6839/3506943982] up:boot 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6834/1456153657,v1:172.21.15.155:6837/1456153657] up:boot 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6834/3348873916,v1:172.21.15.189:6837/3348873916] up:boot 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6836/143833602,v1:172.21.15.155:6839/143833602] up:boot 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.189:6835/1820655793,v1:172.21.15.189:6838/1820655793] up:boot 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: mds.? [v2:172.21.15.155:6835/2583663737,v1:172.21.15.155:6838/2583663737] up:boot 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1046180916' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: fsmap 12 up:standby 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:32:24.169 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:32:24.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:32:24.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:32:24.170 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:23 smithi189 ceph-mon[46919]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:32:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: pgmap v401: 129 pgs: 129 active+clean; 24 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-05-08T23:32:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1046180916' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T23:32:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T23:32:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T23:32:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: Cluster is now healthy 2024-05-08T23:32:24.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6836/3506943982,v1:172.21.15.189:6839/3506943982] up:boot 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6834/1456153657,v1:172.21.15.155:6837/1456153657] up:boot 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6834/3348873916,v1:172.21.15.189:6837/3348873916] up:boot 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6836/143833602,v1:172.21.15.155:6839/143833602] up:boot 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.189:6835/1820655793,v1:172.21.15.189:6838/1820655793] up:boot 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: mds.? [v2:172.21.15.155:6835/2583663737,v1:172.21.15.155:6838/2583663737] up:boot 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1046180916' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: fsmap 12 up:standby 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T23:32:24.216 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T23:32:24.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T23:32:24.217 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:23 smithi155 ceph-mon[47105]: from='mgr.25310 172.21.15.16:0/3827573862' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T23:32:24.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.770+0000 7fddb8ff9700 1 -- 172.21.15.16:0/3888543336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7fddb40210d0 con 0x7fddbc19bcd0 2024-05-08T23:32:24.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdda80a1310 msgr2=0x7fdda80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdda80a1310 0x7fdda80a37d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fddac005950 tx=0x7fddac0058e0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 msgr2=0x7fddbc19c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 0x7fddbc19c150 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006070 tx=0x7fddb400cd90 comp rx=0 tx=0).stop 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 shutdown_connections 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7fdda80a1310 0x7fdda80a37d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc19bcd0 0x7fddbc19c150 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fddbc0682e0 0x7fddbc1973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 --2- 172.21.15.16:0/3888543336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddbc106290 0x7fddbc197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 >> 172.21.15.16:0/3888543336 conn(0x7fddbc076f40 msgr2=0x7fddbc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 shutdown_connections 2024-05-08T23:32:24.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.775+0000 7fddc20b9700 1 -- 172.21.15.16:0/3888543336 wait complete. 2024-05-08T23:32:24.776 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-05-08T23:32:24.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 -- 172.21.15.16:0/8704855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc100060 msgr2=0x7f00dc100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 --2- 172.21.15.16:0/8704855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc100060 0x7f00dc100440 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f00cc009990 tx=0x7f00cc009ca0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 -- 172.21.15.16:0/8704855 shutdown_connections 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 --2- 172.21.15.16:0/8704855 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00dc105160 0x7f00dc10cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 --2- 172.21.15.16:0/8704855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc100060 0x7f00dc100440 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 --2- 172.21.15.16:0/8704855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00dc100a10 0x7f00dc104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 -- 172.21.15.16:0/8704855 >> 172.21.15.16:0/8704855 conn(0x7f00dc076f40 msgr2=0x7f00dc077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 -- 172.21.15.16:0/8704855 shutdown_connections 2024-05-08T23:32:24.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.943+0000 7f00e1d23700 1 -- 172.21.15.16:0/8704855 wait complete. 2024-05-08T23:32:24.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 Processor -- start 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 -- start start 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00dc100060 0x7f00dc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00dc105160 0x7f00dc1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 0x7f00dc19afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc19b5e0 con 0x7f00dc19ab60 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.944+0000 7f00e1d23700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc19b750 con 0x7f00dc105160 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00e1d23700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc19b910 con 0x7f00dc100060 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 0x7f00dc19afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 0x7f00dc19afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34170/0 (socket says 172.21.15.16:34170) 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 -- 172.21.15.16:0/501016780 learned_addr learned my addr 172.21.15.16:0/501016780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:24.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00db7fe700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00dc100060 0x7f00dc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00daffd700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00dc105160 0x7f00dc1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 -- 172.21.15.16:0/501016780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00dc100060 msgr2=0x7f00dc196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00dc100060 0x7f00dc196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 -- 172.21.15.16:0/501016780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00dc105160 msgr2=0x7f00dc1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00dc105160 0x7f00dc1967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 -- 172.21.15.16:0/501016780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0095f0 con 0x7f00dc19ab60 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00dbfff700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 0x7f00dc19afe0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f00d400b970 tx=0x7f00d400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400c930 con 0x7f00dc19ab60 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00d4014440 con 0x7f00dc19ab60 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc1a2e70 con 0x7f00dc19ab60 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.945+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40075d0 con 0x7f00dc19ab60 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.946+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc1a32f0 con 0x7f00dc19ab60 2024-05-08T23:32:24.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.946+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc108380 con 0x7f00dc19ab60 2024-05-08T23:32:24.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.948+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f00d40145b0 con 0x7f00dc19ab60 2024-05-08T23:32:24.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.949+0000 7f00d8ff9700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f00c40a1290 0x7f00c40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:24.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.949+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9083+0+0 (secure 0 0 0) 0x7f00d4006e40 con 0x7f00dc19ab60 2024-05-08T23:32:24.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.949+0000 7f00db7fe700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f00c40a1290 0x7f00c40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:24.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.950+0000 7f00db7fe700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f00c40a1290 0x7f00c40a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00b5c0 tx=0x7f00cc005fb0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:24.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:24.953+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f00d4096060 con 0x7f00dc19ab60 2024-05-08T23:32:25.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:24 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3888543336' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:25.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:24 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3888543336' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:25.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.169+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f00dc101510 con 0x7f00dc19ab60 2024-05-08T23:32:25.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:24 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3888543336' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.780+0000 7f00d8ff9700 1 -- 172.21.15.16:0/501016780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7f00d40190d0 con 0x7f00dc19ab60 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f00c40a1290 msgr2=0x7f00c40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f00c40a1290 0x7f00c40a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00b5c0 tx=0x7f00cc005fb0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 msgr2=0x7f00dc19afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 0x7f00dc19afe0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f00d400b970 tx=0x7f00d400bc80 comp rx=0 tx=0).stop 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 shutdown_connections 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f00c40a1290 0x7f00c40a3750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc19ab60 0x7f00dc19afe0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00dc100060 0x7f00dc196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 --2- 172.21.15.16:0/501016780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00dc105160 0x7f00dc1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.784+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 >> 172.21.15.16:0/501016780 conn(0x7f00dc076f40 msgr2=0x7f00dc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.785+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 shutdown_connections 2024-05-08T23:32:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.785+0000 7f00e1d23700 1 -- 172.21.15.16:0/501016780 wait complete. 2024-05-08T23:32:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-05-08T23:32:25.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T23:32:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 -- 172.21.15.16:0/822367501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 msgr2=0x7f26a010cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 --2- 172.21.15.16:0/822367501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a010cce0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f269800b0a0 tx=0x7f269800b3b0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 -- 172.21.15.16:0/822367501 shutdown_connections 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 --2- 172.21.15.16:0/822367501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a010cce0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 --2- 172.21.15.16:0/822367501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0100060 0x7f26a0100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 --2- 172.21.15.16:0/822367501 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100a10 0x7f26a0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 -- 172.21.15.16:0/822367501 >> 172.21.15.16:0/822367501 conn(0x7f26a0076f40 msgr2=0x7f26a0077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 -- 172.21.15.16:0/822367501 shutdown_connections 2024-05-08T23:32:25.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.947+0000 7f26a48a8700 1 -- 172.21.15.16:0/822367501 wait complete. 2024-05-08T23:32:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 Processor -- start 2024-05-08T23:32:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 -- start start 2024-05-08T23:32:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100060 0x7f26a0196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a0196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0196a90 0x7f26a01a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01a5bd0 con 0x7f26a0105160 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01a5d10 con 0x7f26a0100060 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f26a48a8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a01a5e80 con 0x7f26a0196a90 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f269dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a0196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f269dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a0196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34188/0 (socket says 172.21.15.16:34188) 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f269dd9b700 1 -- 172.21.15.16:0/1718467929 learned_addr learned my addr 172.21.15.16:0/1718467929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f269e59c700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100060 0x7f26a0196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.948+0000 7f269ed9d700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0196a90 0x7f26a01a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269dd9b700 1 -- 172.21.15.16:0/1718467929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0196a90 msgr2=0x7f26a01a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269dd9b700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0196a90 0x7f26a01a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269dd9b700 1 -- 172.21.15.16:0/1718467929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100060 msgr2=0x7f26a0196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269dd9b700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100060 0x7f26a0196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269dd9b700 1 -- 172.21.15.16:0/1718467929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698009d00 con 0x7f26a0105160 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269ed9d700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0196a90 0x7f26a01a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269e59c700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100060 0x7f26a0196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f269dd9b700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a0196550 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f269400d950 tx=0x7f269400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269400bb60 con 0x7f26a0105160 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2694007620 con 0x7f26a0105160 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940053e0 con 0x7f26a0105160 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a01a6160 con 0x7f26a0105160 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.949+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a01a6620 con 0x7f26a0105160 2024-05-08T23:32:25.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.950+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a0108430 con 0x7f26a0105160 2024-05-08T23:32:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.951+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2694005540 con 0x7f26a0105160 2024-05-08T23:32:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.952+0000 7f268f7fe700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26880a1290 0x7f26880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T23:32:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.952+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8553+0+0 (secure 0 0 0) 0x7f26940cc9f0 con 0x7f26a0105160 2024-05-08T23:32:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.952+0000 7f269e59c700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26880a1290 0x7f26880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T23:32:25.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.952+0000 7f269e59c700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26880a1290 0x7f26880a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2690005950 tx=0x7f26900058e0 comp rx=0 tx=0).ready entity=mgr.25310 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T23:32:25.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:25.955+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2694095dc0 con 0x7f26a0105160 2024-05-08T23:32:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:25 smithi016 ceph-mon[42541]: pgmap v403: 129 pgs: 129 active+clean; 24 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T23:32:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/3888543336' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:32:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:25 smithi016 ceph-mon[42541]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:32:26.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:25 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/501016780' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:25 smithi189 ceph-mon[46919]: pgmap v403: 129 pgs: 129 active+clean; 24 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T23:32:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/3888543336' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:32:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:25 smithi189 ceph-mon[46919]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:32:26.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:25 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/501016780' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.179+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f26a0068590 con 0x7f26a0105160 2024-05-08T23:32:26.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:25 smithi155 ceph-mon[47105]: pgmap v403: 129 pgs: 129 active+clean; 24 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 7 op/s 2024-05-08T23:32:26.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/3888543336' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:32:26.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:25 smithi155 ceph-mon[47105]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T23:32:26.215 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:25 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/501016780' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.787+0000 7f268f7fe700 1 -- 172.21.15.16:0/1718467929 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) v1 ==== 178+0+0 (secure 0 0 0) 0x7f26940160d0 con 0x7f26a0105160 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26880a1290 msgr2=0x7f26880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26880a1290 0x7f26880a3750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2690005950 tx=0x7f26900058e0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 msgr2=0x7f26a0196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a0196550 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f269400d950 tx=0x7f269400dd10 comp rx=0 tx=0).stop 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 shutdown_connections 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:6800/198235485,v1:172.21.15.16:6801/198235485] conn(0x7f26880a1290 0x7f26880a3750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a0196a90 0x7f26a01a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f26a0100060 0x7f26a0196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 --2- 172.21.15.16:0/1718467929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26a0105160 0x7f26a0196550 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T23:32:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 >> 172.21.15.16:0/1718467929 conn(0x7f26a0076f40 msgr2=0x7f26a00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T23:32:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 shutdown_connections 2024-05-08T23:32:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-05-08T23:32:26.792+0000 7f26a48a8700 1 -- 172.21.15.16:0/1718467929 wait complete. 2024-05-08T23:32:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-05-08T23:32:26.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:32:26.840 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:32:26.879 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:32:26.919 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T23:32:26.919 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:32:26.941 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:32:26.962 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T23:32:26.982 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T23:32:26.982 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T23:32:26.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a 2024-05-08T23:32:27.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:26 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/501016780' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:32:27.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:26 smithi016 ceph-mon[42541]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:32:27.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:26 smithi016 ceph-mon[42541]: from='client.? 172.21.15.16:0/1718467929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:27.113 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:27 smithi016 systemd[1]: Stopping Ceph mon.a for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:27.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/501016780' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:32:27.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:26 smithi189 ceph-mon[46919]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:32:27.168 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:26 smithi189 ceph-mon[46919]: from='client.? 172.21.15.16:0/1718467929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:27.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/501016780' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T23:32:27.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:26 smithi155 ceph-mon[47105]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T23:32:27.214 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:26 smithi155 ceph-mon[47105]: from='client.? 172.21.15.16:0/1718467929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T23:32:27.575 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:27 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a[42518]: 2024-05-08T23:32:27.246+0000 7f6572e04700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:27.575 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:27 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a[42518]: 2024-05-08T23:32:27.246+0000 7f6572e04700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T23:32:27.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:May 08 23:32:27 smithi016 bash[129785]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-a 2024-05-08T23:32:27.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T23:32:27.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:27.947 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T23:32:27.948 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T23:32:27.948 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.b 2024-05-08T23:32:28.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:27 smithi155 systemd[1]: Stopping Ceph mon.b for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:28.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:28 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-b[47082]: 2024-05-08T23:32:28.188+0000 7f2cada15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:28.464 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:28 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-b[47082]: 2024-05-08T23:32:28.188+0000 7f2cada15700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T23:32:28.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:May 08 23:32:28 smithi155 bash[90364]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-b 2024-05-08T23:32:28.957 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T23:32:29.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:29.007 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T23:32:29.007 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T23:32:29.007 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.c 2024-05-08T23:32:29.312 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:29 smithi189 systemd[1]: Stopping Ceph mon.c for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:29.641 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-c[46896]: 2024-05-08T23:32:29.311+0000 7fd3061f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:29.641 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-c[46896]: 2024-05-08T23:32:29.311+0000 7fd3061f3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T23:32:29.918 INFO:journalctl@ceph.mon.c.smithi189.stdout:May 08 23:32:29 smithi189 bash[90088]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mon-c 2024-05-08T23:32:29.962 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T23:32:30.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:30.010 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T23:32:30.010 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T23:32:30.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x 2024-05-08T23:32:30.293 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:32:30 smithi016 systemd[1]: Stopping Ceph mgr.x for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:30.293 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:32:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x[42792]: 2024-05-08T23:32:30.261+0000 7f420eddc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:30.613 INFO:journalctl@ceph.mgr.x.smithi016.stdout:May 08 23:32:30 smithi016 bash[129923]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-x 2024-05-08T23:32:30.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T23:32:30.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:30.749 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T23:32:30.749 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T23:32:30.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.y 2024-05-08T23:32:31.092 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:32:30 smithi155 systemd[1]: Stopping Ceph mgr.y for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:31.464 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:32:31 smithi155 bash[90506]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-y 2024-05-08T23:32:31.464 INFO:journalctl@ceph.mgr.y.smithi155.stdout:May 08 23:32:31 smithi155 systemd[1]: ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-08T23:32:31.503 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T23:32:31.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:31.540 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T23:32:31.540 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T23:32:31.541 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.z 2024-05-08T23:32:31.918 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:32:31 smithi189 systemd[1]: Stopping Ceph mgr.z for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:32.242 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:32:31 smithi189 bash[90226]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mgr-z 2024-05-08T23:32:32.242 INFO:journalctl@ceph.mgr.z.smithi189.stdout:May 08 23:32:32 smithi189 systemd[1]: ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T23:32:32.248 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T23:32:32.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:32.285 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T23:32:32.285 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T23:32:32.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.0 2024-05-08T23:32:32.613 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:32:32 smithi016 systemd[1]: Stopping Ceph osd.0 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:32.613 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:32:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0[61610]: 2024-05-08T23:32:32.527+0000 7fb40093d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:32.614 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:32:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0[61610]: 2024-05-08T23:32:32.527+0000 7fb40093d700 -1 osd.0 104 *** Got signal Terminated *** 2024-05-08T23:32:32.614 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:32:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0[61610]: 2024-05-08T23:32:32.527+0000 7fb40093d700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:32:38.363 INFO:journalctl@ceph.osd.0.smithi016.stdout:May 08 23:32:38 smithi016 bash[130065]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-0 2024-05-08T23:32:39.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T23:32:39.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:39.344 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T23:32:39.344 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T23:32:39.344 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.1 2024-05-08T23:32:39.714 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:32:39 smithi155 systemd[1]: Stopping Ceph osd.1 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:39.714 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:32:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1[52095]: 2024-05-08T23:32:39.577+0000 7f5117567700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:39.714 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:32:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1[52095]: 2024-05-08T23:32:39.577+0000 7f5117567700 -1 osd.1 104 *** Got signal Terminated *** 2024-05-08T23:32:39.714 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:32:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1[52095]: 2024-05-08T23:32:39.577+0000 7f5117567700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:32:45.464 INFO:journalctl@ceph.osd.1.smithi155.stdout:May 08 23:32:45 smithi155 bash[90646]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-1 2024-05-08T23:32:46.399 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T23:32:46.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:46.441 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T23:32:46.441 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T23:32:46.441 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.2 2024-05-08T23:32:46.767 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:32:46 smithi189 systemd[1]: Stopping Ceph osd.2 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:47.168 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:32:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2[52700]: 2024-05-08T23:32:46.767+0000 7ff2f68f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:47.168 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:32:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2[52700]: 2024-05-08T23:32:46.767+0000 7ff2f68f3700 -1 osd.2 104 *** Got signal Terminated *** 2024-05-08T23:32:47.168 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:32:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2[52700]: 2024-05-08T23:32:46.767+0000 7ff2f68f3700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:32:52.668 INFO:journalctl@ceph.osd.2.smithi189.stdout:May 08 23:32:52 smithi189 bash[90371]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-2 2024-05-08T23:32:53.637 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T23:32:53.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:32:53.679 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T23:32:53.679 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T23:32:53.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.3 2024-05-08T23:32:54.113 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:32:53 smithi016 systemd[1]: Stopping Ceph osd.3 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:32:54.113 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:32:53 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3[69694]: 2024-05-08T23:32:53.940+0000 7fb64ccec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:32:54.113 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:32:53 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3[69694]: 2024-05-08T23:32:53.940+0000 7fb64ccec700 -1 osd.3 104 *** Got signal Terminated *** 2024-05-08T23:32:54.113 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:32:53 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3[69694]: 2024-05-08T23:32:53.940+0000 7fb64ccec700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:32:59.363 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:32:58 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:32:58.862+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:32:59.863 INFO:journalctl@ceph.osd.3.smithi016.stdout:May 08 23:32:59 smithi016 bash[130326]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-3 2024-05-08T23:33:00.363 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:32:59 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:32:59.865+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:00.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:00 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:00.005+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:00.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T23:33:00.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:00.834 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T23:33:00.834 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T23:33:00.834 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.4 2024-05-08T23:33:01.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:00 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:00.842+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:01.214 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:33:00 smithi155 systemd[1]: Stopping Ceph osd.4 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:01.214 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:33:01 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4[60168]: 2024-05-08T23:33:01.074+0000 7fdac11a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:01.214 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:33:01 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4[60168]: 2024-05-08T23:33:01.074+0000 7fdac11a6700 -1 osd.4 104 *** Got signal Terminated *** 2024-05-08T23:33:01.214 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:33:01 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4[60168]: 2024-05-08T23:33:01.074+0000 7fdac11a6700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:01.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:00 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:00.982+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:02.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:01 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:01.840+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:02.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:02 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:02.023+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:02.917 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:02 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:02.422+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:03.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:02.808+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:03.395 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:03 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:03.026+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:03.668 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:03 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:03.396+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:04.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:03 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:03.799+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:04.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:04 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:04.013+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:04.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:04 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:04.420+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:05.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:04 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:04.814+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:05.374 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:04 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:04.983+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:05.668 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:05 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:05.374+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:05.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:05 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:05.391+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:06.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:05 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:05.806+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:06.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:05 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:05.806+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:06.417 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:05 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:05.962+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:06.668 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:06 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:06.630+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:06.668 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:06 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:06.418+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:06.684 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:06 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:06.389+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:06.964 INFO:journalctl@ceph.osd.4.smithi155.stdout:May 08 23:33:06 smithi155 bash[90906]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-4 2024-05-08T23:33:07.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:06 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:06.856+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:07.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:06 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:06.856+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:07.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:07 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:06.999+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:07.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:07 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:06.999+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:07.464 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:07 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:07.392+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:07.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:07.250+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:07.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:07 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:07.627+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:07.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:07 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:07.455+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:07.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T23:33:07.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:07.994 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T23:33:07.994 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T23:33:07.994 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.5 2024-05-08T23:33:08.246 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:07 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:07.962+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:08.246 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:07 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:07.962+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:08.246 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:08 smithi189 systemd[1]: Stopping Ceph osd.5 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:08.268 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:07.890+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:08.268 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:07 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:07.890+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:08.582 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:08.482+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:08.583 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:08.246+0000 7f919f67f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:08.583 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:08.246+0000 7f919f67f700 -1 osd.5 104 *** Got signal Terminated *** 2024-05-08T23:33:08.583 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:08.246+0000 7f919f67f700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:08.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:08 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:08.269+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:08.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:08 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:08.421+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:08.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:08 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:08.291+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:08.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:08.583+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:09.241 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:08 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:08.885+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:09.241 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:08 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:08.885+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:09.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:08.954+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:09.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:08 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:08.954+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:09.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:09 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:09.242+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:09.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:09 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:09.454+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:09.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:09 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:09.314+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:09.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:09.596+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:09.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:09.521+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:09.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:09.521+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:10.284 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:09 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:09.891+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:10.284 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:09 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:09.891+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:10.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:09.946+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:10.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:09 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:09.946+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:10.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:10 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:10.285+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:10.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:10 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:10.476+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:10.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:10 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:10.317+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:10.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:10 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:10.556+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:10.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:10 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:10.542+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:10.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:10 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:10.542+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:11.324 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:10 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:10.874+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:11.324 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:10 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:10.874+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:11.417 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:10 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:10.964+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:11.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:10 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:10.964+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:11.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:11 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:11.325+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:11.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:11 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:11.449+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:11.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:11 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:11.341+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:11.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:11 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:11.548+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:11.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:11 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:11.495+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:11.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:11 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:11.495+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:12.280 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:11 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:11.888+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:12.280 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:11 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:11.888+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:12.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:11 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:11.972+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:12.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:11 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:11.972+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:12.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:12.280+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:12.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:12 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:12.433+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:12.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:12 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:12.315+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:12.917 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:12 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:12.595+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:12.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:12 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:12.450+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:12.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:12 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:12.450+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:13.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:12.849+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:13.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:12.849+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:13.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:12 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:12.849+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:13.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:13.013+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:36.497066+0000 front 2024-05-08T23:32:36.497091+0000 (oldest deadline 2024-05-08T23:32:59.397272+0000) 2024-05-08T23:33:13.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:13.013+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:39.397851+0000 front 2024-05-08T23:32:39.397946+0000 (oldest deadline 2024-05-08T23:33:06.398461+0000) 2024-05-08T23:33:13.418 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5[61114]: 2024-05-08T23:33:13.013+0000 7f9197c6d700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.499452+0000 front 2024-05-08T23:32:50.499535+0000 (oldest deadline 2024-05-08T23:33:12.799663+0000) 2024-05-08T23:33:13.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:13.326+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:13.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:13 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:13.462+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:13.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:13 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:13.333+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:13.918 INFO:journalctl@ceph.osd.5.smithi189.stdout:May 08 23:33:13 smithi189 bash[90627]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-5 2024-05-08T23:33:13.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:13.633+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:13.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:13.495+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:13.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:13 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:13.495+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:14.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:13.825+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:14.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:13.825+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:14.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:13 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:13.825+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:14.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:14.332+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:14.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:14 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:14.443+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:14.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:14 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:14.443+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:14.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:14 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:14.289+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:14.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:14 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:14.289+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:14.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:14 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:14.608+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:14.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:14 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:14.503+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:14.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:14 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:14.503+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:14.934 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T23:33:14.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:14.975 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T23:33:14.975 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T23:33:14.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.6 2024-05-08T23:33:15.113 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:15 smithi016 systemd[1]: Stopping Ceph osd.6 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:15.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:14.792+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:15.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:14.792+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:15.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:14 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:14.792+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:15.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:15.239+0000 7f5c28311700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:15.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:15.239+0000 7f5c28311700 -1 osd.6 104 *** Got signal Terminated *** 2024-05-08T23:33:15.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:15.239+0000 7f5c28311700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:15.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:15.317+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:15.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:15.449+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:15.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:15.449+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:15.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:15.264+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:15.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:15.264+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:15.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:15 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:15.579+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:15.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:15 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:15.514+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:15.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:15 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:15.514+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:16.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:15.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:16.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:15.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:16.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:15 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:15.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:16.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:16.327+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:16.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:16.327+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.061765+0000 front 2024-05-08T23:32:50.061798+0000 (oldest deadline 2024-05-08T23:33:15.361800+0000) 2024-05-08T23:33:16.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:16 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:16.407+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:16.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:16 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:16.407+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:16.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:16 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:16.269+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:16.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:16 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:16.269+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:16.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:16 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:16.594+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:16.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:16 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:16.594+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:16.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:16 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:16.558+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:16.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:16 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:16.558+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:16.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:16 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:16.558+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:17.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:16.743+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:17.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:16.743+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:17.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:16 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:16.743+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:17.612 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:17.346+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:17.613 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:17.346+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.061765+0000 front 2024-05-08T23:32:50.061798+0000 (oldest deadline 2024-05-08T23:33:15.361800+0000) 2024-05-08T23:33:17.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:17 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:17.361+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:17.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:17 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:17.361+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:17.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:17 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:17.315+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:17.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:17 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:17.315+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:17.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:17 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:17.612+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:17.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:17 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:17.612+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:17.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:17 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:17.606+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:17.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:17 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:17.606+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:17.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:17 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:17.606+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:18.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:17.725+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:18.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:17.725+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:18.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:17 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:17.725+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:18.688 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:18 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:18.365+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:18.688 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:18 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:18.365+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.061765+0000 front 2024-05-08T23:32:50.061798+0000 (oldest deadline 2024-05-08T23:33:15.361800+0000) 2024-05-08T23:33:18.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:18 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:18.393+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:18.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:18 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:18.393+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:18.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:18 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:18.284+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:18.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:18 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:18.284+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:18.917 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:18 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:18.650+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:18.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:18 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:18.650+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:18.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:18 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:18.580+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:18.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:18 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:18.580+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:18.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:18 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:18.580+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:19.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:18 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:18.689+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:19.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:18 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:18.689+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:19.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:18 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:18.689+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:19.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:19 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:19.386+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:19.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:19 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:19.386+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:19.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:19 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:19.326+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:19.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:19 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:19.326+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:19.724 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:19.364+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:44.260531+0000 front 2024-05-08T23:32:44.260412+0000 (oldest deadline 2024-05-08T23:33:06.560523+0000) 2024-05-08T23:33:19.724 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6[78459]: 2024-05-08T23:33:19.364+0000 7f5c208ff700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.061765+0000 front 2024-05-08T23:32:50.061798+0000 (oldest deadline 2024-05-08T23:33:15.361800+0000) 2024-05-08T23:33:19.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:19 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:19.644+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:19.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:19 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:19.644+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:19.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:19 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:19.536+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:19.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:19 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:19.536+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:19.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:19 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:19.536+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:20.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:19.725+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:20.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:19.725+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:20.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:19 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:19.725+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:20.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:20 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:20.348+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:20.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:20 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:20.348+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:20.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:20 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:20.357+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:20.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:20 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:20.357+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:20.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:20 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:20.357+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:20.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:20 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:20.608+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:20.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:20 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:20.608+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:20.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:20 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:20.524+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:20.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:20 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:20.524+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:20.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:20 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:20.524+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:21.113 INFO:journalctl@ceph.osd.6.smithi016.stdout:May 08 23:33:20 smithi016 bash[130591]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-6 2024-05-08T23:33:21.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:20 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:20.686+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:21.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:20 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:20.686+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:21.114 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:20 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:20.686+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:21.114 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:20 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:20.686+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:21.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:21 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:21.345+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:21.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:21 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:21.345+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:21.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:21 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:21.345+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:21.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:21 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:21.321+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:21.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:21 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:21.321+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:21.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:21 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:21.321+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:21.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:21 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:21.593+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:21.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:21 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:21.593+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:21.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:21 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:21.560+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:21.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:21 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:21.560+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:21.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:21 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:21.560+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:22.009 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:21 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:21.660+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:22.009 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:21 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:21.660+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:22.009 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:21 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:21.660+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:22.009 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:21 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:21.660+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:22.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T23:33:22.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:22.073 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T23:33:22.073 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T23:33:22.073 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.7 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 systemd[1]: Stopping Ceph osd.7 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:22.312+0000 7fc4a9396700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:22.312+0000 7fc4a9396700 -1 osd.7 104 *** Got signal Terminated *** 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:22.312+0000 7fc4a9396700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:22.338+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:22.338+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:22.361 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:22.338+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:22.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:22.361+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:22.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:22.361+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:22.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:22 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:22.361+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:22.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:22.598+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:22.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:22.598+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:22.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:22.601+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:22.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:22.601+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:22.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:22 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:22.601+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:23.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:22 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:22.685+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:23.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:22 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:22.685+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:23.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:22 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:22.685+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:23.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:22 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:22.685+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:23.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:23.293+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:23.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:23.293+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:23.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:23.293+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:23.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:23.338+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:23.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:23.338+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:23.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:23.338+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:23.917 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:23.605+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:23.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:23.605+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:23.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:23.575+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:23.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:23.575+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:23.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:23 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:23.575+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:24.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:23 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:23.732+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:24.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:23 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:23.732+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:24.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:23 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:23.732+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:24.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:23 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:23.732+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:24.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:24 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:24.258+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:24.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:24 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:24.258+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:24.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:24 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:24.258+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:24.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:24 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:24.316+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:24.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:24 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:24.316+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:24.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:24 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:24.316+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:24.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:24 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:24.556+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:24.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:24 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:24.556+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:24.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:24 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:24.599+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:24.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:24 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:24.599+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:24.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:24 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:24.599+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:25.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:24 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:24.760+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:25.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:24 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:24.760+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:25.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:24 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:24.760+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:25.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:24 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:24.760+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:25.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:25 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:25.241+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:25.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:25 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:25.241+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:25.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:25 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:25.241+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:25.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:25 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:25.364+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:25.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:25 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:25.364+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:25.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:25 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:25.364+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:25.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:25 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:25.582+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:25.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:25 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:25.582+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:25.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:25 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:25.582+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:25.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:25 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:25.586+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:25.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:25 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:25.586+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:25.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:25 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:25.586+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:26.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:25.794+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:26.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:25.794+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:26.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:25.794+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:26.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:25 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:25.794+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:26.464 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:26 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:26.195+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:26.464 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:26 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:26.195+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:26.464 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:26 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:26.195+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:26.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:26 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:26.380+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:26.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:26 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:26.380+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:26.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:26 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:26.380+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:26.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:26 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:26.632+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:26.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:26 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:26.632+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:26.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:26 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:26.632+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:26.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:26 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:26.540+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:26.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:26 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:26.540+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:26.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:26 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:26.540+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:27.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:26 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:26.750+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:27.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:26 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:26.750+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:27.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:26 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:26.750+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:27.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:26 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:26.750+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:27.714 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:27.225+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.205330+0000 front 2024-05-08T23:32:42.205368+0000 (oldest deadline 2024-05-08T23:33:05.105580+0000) 2024-05-08T23:33:27.715 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:27.225+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:49.206707+0000 front 2024-05-08T23:32:49.206773+0000 (oldest deadline 2024-05-08T23:33:13.906604+0000) 2024-05-08T23:33:27.715 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7[68799]: 2024-05-08T23:33:27.225+0000 7fc4a1984700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:58.007693+0000 front 2024-05-08T23:32:58.007647+0000 (oldest deadline 2024-05-08T23:33:20.907657+0000) 2024-05-08T23:33:27.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:27.418+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:27.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:27.418+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:27.715 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:27 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:27.418+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:27.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:27.674+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:27.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:27.674+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:27.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:27.674+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:27.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:27.581+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:27.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:27.581+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:27.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:27.581+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:27.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:27.581+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:28.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:27 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:27.730+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:28.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:27 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:27.730+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:28.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:27 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:27.730+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:28.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:27 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:27.730+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:28.214 INFO:journalctl@ceph.osd.7.smithi155.stdout:May 08 23:33:27 smithi155 bash[91165]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-7 2024-05-08T23:33:28.628 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:28 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:28.440+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:28.628 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:28 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:28.440+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:28.628 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:28 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:28.440+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:28.628 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:28 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:28.440+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:28.917 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:28.671+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:28.671+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:28.671+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:28.671+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.579884+0000 front 2024-05-08T23:33:05.579816+0000 (oldest deadline 2024-05-08T23:33:28.479722+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:28.596+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:28.596+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:28.596+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:28.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:28 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:28.596+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:28.974 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T23:33:29.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:29.015 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T23:33:29.016 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T23:33:29.016 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.8 2024-05-08T23:33:29.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:28 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:28.695+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:29.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:28 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:28.695+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:29.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:28 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:28.695+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:29.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:28 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:28.695+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:29.418 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 systemd[1]: Stopping Ceph osd.8 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:29.418 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.244+0000 7f3878456700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:29.418 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.244+0000 7f3878456700 -1 osd.8 104 *** Got signal Terminated *** 2024-05-08T23:33:29.418 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.244+0000 7f3878456700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:29.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:29 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:29.451+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:29.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:29 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:29.451+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:29.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:29 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:29.451+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:29.714 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:29 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:29.451+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:29.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:29.621+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:29.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:29.621+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:29.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:29.621+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:29.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:29.621+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:29.919 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.712+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:29.919 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.712+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:29.919 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.712+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:29.919 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:29 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:29.712+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.579884+0000 front 2024-05-08T23:33:05.579816+0000 (oldest deadline 2024-05-08T23:33:28.479722+0000) 2024-05-08T23:33:30.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:29 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:29.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:30.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:29 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:29.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:30.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:29 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:29.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:30.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:29 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:29.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:30.698+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:30.698+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:30.699+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:30.699+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.579884+0000 front 2024-05-08T23:33:05.579816+0000 (oldest deadline 2024-05-08T23:33:28.479722+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:30.667+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:30.667+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:30.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:30.667+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:30.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:30 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:30.667+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:30.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:30.478+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:30.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:30.479+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:30.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:30.479+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:30.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:30 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:30.479+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:31.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:30.697+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:31.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:30.697+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:31.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:30.697+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:31.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:30.697+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:31.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:30 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:30.697+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:31.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:31.496+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:31.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:31.496+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:31.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:31.496+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:31.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:31 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:31.496+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:32.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:31.662+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:32.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:31.662+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:32.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:31.662+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:32.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:31.662+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:32.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:31 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:31.662+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:31.719+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:31.720+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:31.720+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:31.720+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.579884+0000 front 2024-05-08T23:33:05.579816+0000 (oldest deadline 2024-05-08T23:33:28.479722+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:31.701+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:31.701+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:31.701+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:32.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:31.701+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:32.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:32.534+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:32.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:32.534+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:32.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:32.534+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:32.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:32 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:32.534+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:33.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:32.684+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:33.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:32.684+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:33.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:32.684+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:33.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:32.684+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:33.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:32 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:32.684+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:32.689+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:32.689+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:32.689+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:32.689+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.579884+0000 front 2024-05-08T23:33:05.579816+0000 (oldest deadline 2024-05-08T23:33:28.479722+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:32.700+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:32.700+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:32.700+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:33.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:32 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:32.700+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:33.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:33.559+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:33.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:33.559+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:33.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:33.559+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:33.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:33 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:33.559+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:34.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:33.652+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:34.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:33.652+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:34.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:33.652+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:34.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:33.652+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:34.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:33.652+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:34.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:33 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:33.652+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:33.677+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.075945+0000 front 2024-05-08T23:32:41.075997+0000 (oldest deadline 2024-05-08T23:33:05.776004+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:33.677+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:50.377790+0000 front 2024-05-08T23:32:50.377679+0000 (oldest deadline 2024-05-08T23:33:15.677877+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:33.677+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:55.678594+0000 front 2024-05-08T23:32:55.678511+0000 (oldest deadline 2024-05-08T23:33:25.078982+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8[69640]: 2024-05-08T23:33:33.677+0000 7f3874a4c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.579884+0000 front 2024-05-08T23:33:05.579816+0000 (oldest deadline 2024-05-08T23:33:28.479722+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:33.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:33.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:34.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:33.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:34.169 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:33 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:33.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:34.918 INFO:journalctl@ceph.osd.8.smithi189.stdout:May 08 23:33:34 smithi189 bash[90892]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-8 2024-05-08T23:33:34.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:34.639+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:34.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:34.639+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:34.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:34.639+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:34.919 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:34 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:34.639+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:34.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:34.514+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:34.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:34.514+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:34.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:34.514+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:34.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:34 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:34.514+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:35.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:34.669+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:35.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:34.669+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:35.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:34.669+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:35.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:34.669+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:35.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:34.669+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:35.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:34 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:34.669+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:35.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:35.616+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:35.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:35.616+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:35.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:35.616+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:35.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:35.616+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:35.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:35.616+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:35.963 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T23:33:35.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:35 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:35.478+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:35.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:35 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:35.478+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:35.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:35 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:35.478+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:35.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:35 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:35.478+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:36.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:36.004 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T23:33:36.005 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T23:33:36.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.9 2024-05-08T23:33:36.038 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:35.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:36.038 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:35.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:36.038 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:35.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:36.038 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:35.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:36.038 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:35.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:36.038 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:35 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:35.705+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:36.363 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 systemd[1]: Stopping Ceph osd.9 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:36.363 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.252+0000 7faf8de03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:36.363 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.252+0000 7faf8de03700 -1 osd.9 104 *** Got signal Terminated *** 2024-05-08T23:33:36.363 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.252+0000 7faf8de03700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:36.917 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:36.570+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:36.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:36.570+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:36.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:36.570+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:36.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:36.570+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:36.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:36 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:36.570+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:36.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:36.525+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:36.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:36.525+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:36.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:36.525+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:36.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:36.525+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:36.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:36 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:36.525+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:37.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.691+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:37.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.691+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:37.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.691+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:37.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.691+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:37.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.691+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:37.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:36 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:36.691+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:37.917 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:37.605+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:37.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:37.605+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:37.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:37.605+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:37.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:37.605+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:37.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:37 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:37.605+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:37.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:37.558+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:37.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:37.558+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:37.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:37.558+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:37.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:37.558+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:37.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:37 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:37.558+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:38.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:37.700+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:38.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:37.700+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:38.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:37.700+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:38.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:37.700+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:38.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:37.700+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:38.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:37 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:37.700+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:38.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:38.632+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:38.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:38.633+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:38.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:38.633+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:38.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:38.633+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:38.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:38 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:38.633+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:38.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:38.512+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:38.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:38.512+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:38.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:38.512+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:38.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:38.512+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:38.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:38 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:38.512+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:39.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:38.712+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:39.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:38.712+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:39.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:38.712+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:39.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:38.712+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:39.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:38.712+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:39.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:38 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:38.712+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:39.917 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:39.663+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:39.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:39.663+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:39.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:39.663+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:39.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:39.663+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:39.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:39.663+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:39.963 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:39.543+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:39.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:39.543+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:39.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:39.543+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:39.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:39.543+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:39.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:39 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:39.543+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:40.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:39.753+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:40.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:39.753+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:40.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:39.753+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:40.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:39.753+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:40.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:39.753+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:40.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:39 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:39.753+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:40.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:40.511+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:40.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:40.511+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:40.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:40.511+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:40.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:40.511+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:40.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:40 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:40.511+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:41.112 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:40.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:31.863577+0000 front 2024-05-08T23:32:31.863547+0000 (oldest deadline 2024-05-08T23:32:58.264424+0000) 2024-05-08T23:33:41.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:40.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:41.165558+0000 front 2024-05-08T23:32:41.165517+0000 (oldest deadline 2024-05-08T23:33:05.265668+0000) 2024-05-08T23:33:41.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:40.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.667123+0000 front 2024-05-08T23:32:48.667142+0000 (oldest deadline 2024-05-08T23:33:12.767172+0000) 2024-05-08T23:33:41.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:40.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:57.368933+0000 front 2024-05-08T23:32:57.368906+0000 (oldest deadline 2024-05-08T23:33:20.269103+0000) 2024-05-08T23:33:41.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:40.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:05.470734+0000 front 2024-05-08T23:33:05.470777+0000 (oldest deadline 2024-05-08T23:33:30.170555+0000) 2024-05-08T23:33:41.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:40 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9[87250]: 2024-05-08T23:33:40.771+0000 7faf8a3f9700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:11.271605+0000 front 2024-05-08T23:33:11.271541+0000 (oldest deadline 2024-05-08T23:33:33.571558+0000) 2024-05-08T23:33:41.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:40.702+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:41.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:40.702+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:41.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:40.702+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:41.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:40.702+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:41.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:40.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:41.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:40 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:40.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:41.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:41.518+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:41.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:41.518+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:41.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:41.518+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:41.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:41.518+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:41.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:41 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:41.518+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:42.113 INFO:journalctl@ceph.osd.9.smithi016.stdout:May 08 23:33:41 smithi016 bash[130850]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-9 2024-05-08T23:33:42.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:41.748+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:42.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:41.748+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:42.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:41.748+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:42.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:41.748+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:42.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:41.748+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:42.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:41 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:41.748+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:42.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:42.519+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:42.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:42.519+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:42.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:42.519+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:42.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:42.519+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:42.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:42 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:42.519+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:42.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T23:33:43.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:43.085 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T23:33:43.085 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T23:33:43.085 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.10 2024-05-08T23:33:43.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:42 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:42.735+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:43.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:42 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:42.735+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:43.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:42 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:42.735+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:43.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:42 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:42.735+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:43.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:42 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:42.735+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:43.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:42 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:42.735+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:43.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 systemd[1]: Stopping Ceph osd.10 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:43.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.326+0000 7ff214e42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:43.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.326+0000 7ff214e42700 -1 osd.10 104 *** Got signal Terminated *** 2024-05-08T23:33:43.464 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.326+0000 7ff214e42700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:43.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.513+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:43.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.513+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:43.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.513+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:43.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.513+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:43.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:43 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:43.513+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:44.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:43 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:43.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:44.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:43 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:43.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:44.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:43 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:43.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:44.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:43 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:43.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:44.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:43 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:43.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:44.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:43 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:43.703+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:44.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:44 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:44.501+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:44.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:44 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:44.501+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:44.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:44 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:44.501+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:44.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:44 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:44.501+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:44.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:44 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:44.501+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:44.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:44 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:44.501+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:20.118992+0000 front 2024-05-08T23:33:20.119022+0000 (oldest deadline 2024-05-08T23:33:43.619133+0000) 2024-05-08T23:33:45.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:44 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:44.688+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:45.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:44 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:44.688+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:45.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:44 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:44.688+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:45.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:44 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:44.688+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:45.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:44 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:44.688+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:45.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:44 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:44.688+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:45.917 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:45 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:45.654+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:45.917 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:45 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:45.654+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:45.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:45 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:45.654+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:45.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:45 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:45.654+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:45.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:45 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:45.654+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:45.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:45 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:45.654+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:45.963 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:45 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:45.486+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:45.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:45 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:45.486+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:45.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:45 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:45.486+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:45.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:45 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:45.486+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:45.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:45 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:45.486+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:45.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:45 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:45.486+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:20.118992+0000 front 2024-05-08T23:33:20.119022+0000 (oldest deadline 2024-05-08T23:33:43.619133+0000) 2024-05-08T23:33:46.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:46 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:46.483+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:46.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:46 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:46.483+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:46.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:46 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:46.483+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:46.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:46 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:46.483+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:46.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:46 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:46.483+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:46.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:46 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:46.483+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:20.118992+0000 front 2024-05-08T23:33:20.119022+0000 (oldest deadline 2024-05-08T23:33:43.619133+0000) 2024-05-08T23:33:47.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:46.681+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:47.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:46.681+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:47.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:46.681+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:47.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:46.681+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:47.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:46.681+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:47.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:46 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:46.681+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:47.964 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:47 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:47.516+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.411888+0000 front 2024-05-08T23:32:42.411905+0000 (oldest deadline 2024-05-08T23:33:07.612709+0000) 2024-05-08T23:33:48.021 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:47 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:47.516+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:47.613484+0000 front 2024-05-08T23:32:47.613407+0000 (oldest deadline 2024-05-08T23:33:13.513442+0000) 2024-05-08T23:33:48.021 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:47 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:47.516+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-05-08T23:32:54.014345+0000 front 2024-05-08T23:32:54.014312+0000 (oldest deadline 2024-05-08T23:33:19.914237+0000) 2024-05-08T23:33:48.022 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:47 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:47.516+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:04.515741+0000 front 2024-05-08T23:33:04.515797+0000 (oldest deadline 2024-05-08T23:33:28.015985+0000) 2024-05-08T23:33:48.022 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:47 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:47.516+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:12.017616+0000 front 2024-05-08T23:33:12.017665+0000 (oldest deadline 2024-05-08T23:33:35.517722+0000) 2024-05-08T23:33:48.022 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:47 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10[76814]: 2024-05-08T23:33:47.516+0000 7ff20d430700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:20.118992+0000 front 2024-05-08T23:33:20.119022+0000 (oldest deadline 2024-05-08T23:33:43.619133+0000) 2024-05-08T23:33:48.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:47 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:47.669+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:48.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:47 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:47.669+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:48.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:47 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:47.669+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:48.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:47 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:47.669+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:48.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:47 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:47.669+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:48.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:47 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:47.669+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:49.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:48 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:48.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:49.213 INFO:journalctl@ceph.osd.10.smithi155.stdout:May 08 23:33:48 smithi155 bash[91429]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-10 2024-05-08T23:33:50.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:50.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:50.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:50.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:50.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:50.169 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:50.169 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:49 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:49.738+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:50.187 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T23:33:50.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:50.228 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T23:33:50.228 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T23:33:50.228 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.11 2024-05-08T23:33:50.598 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 systemd[1]: Stopping Ceph osd.11 for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.598+0000 7f2d20d35700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.598+0000 7f2d20d35700 -1 osd.11 104 *** Got signal Terminated *** 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.598+0000 7f2d20d35700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:50.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:50 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:50.714+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:52.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:52.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:52.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:52.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:52.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:52.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:52.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:51 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:51.711+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:52.918 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:52 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:52.665+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:54.167 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:54.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:54.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:54.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:54.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:54.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:54.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:53 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:53.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-05-08T23:32:37.384491+0000 front 2024-05-08T23:32:37.384531+0000 (oldest deadline 2024-05-08T23:33:02.084530+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6804 osd.1 since back 2024-05-08T23:32:42.585816+0000 front 2024-05-08T23:32:42.585682+0000 (oldest deadline 2024-05-08T23:33:08.485630+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-05-08T23:32:48.485998+0000 front 2024-05-08T23:32:48.485958+0000 (oldest deadline 2024-05-08T23:33:16.086510+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6812 osd.4 since back 2024-05-08T23:33:01.987512+0000 front 2024-05-08T23:33:01.987726+0000 (oldest deadline 2024-05-08T23:33:27.287482+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-05-08T23:33:10.089309+0000 front 2024-05-08T23:33:10.089393+0000 (oldest deadline 2024-05-08T23:33:34.789455+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-05-08T23:33:14.789903+0000 front 2024-05-08T23:33:14.789925+0000 (oldest deadline 2024-05-08T23:33:40.689809+0000) 2024-05-08T23:33:55.168 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:54 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11[77597]: 2024-05-08T23:33:54.674+0000 7f2d19323700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.155:6820 osd.7 since back 2024-05-08T23:33:26.491469+0000 front 2024-05-08T23:33:26.491457+0000 (oldest deadline 2024-05-08T23:33:48.191254+0000) 2024-05-08T23:33:56.417 INFO:journalctl@ceph.osd.11.smithi189.stdout:May 08 23:33:56 smithi189 bash[91148]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-osd-11 2024-05-08T23:33:57.259 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T23:33:57.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:33:57.301 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T23:33:57.301 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T23:33:57.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.a 2024-05-08T23:33:57.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:33:57 smithi016 systemd[1]: Stopping Ceph mds.a for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:33:57.613 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:33:57 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-a[95944]: 2024-05-08T23:33:57.549+0000 7f8cba931700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:33:57.613 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:33:57 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-a[95944]: 2024-05-08T23:33:57.549+0000 7f8cba931700 -1 mds.a *** got signal Terminated *** 2024-05-08T23:34:01.985 INFO:journalctl@ceph.mds.a.smithi016.stdout:May 08 23:34:01 smithi016 bash[131102]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-a 2024-05-08T23:34:01.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T23:34:02.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:02.032 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T23:34:02.032 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T23:34:02.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.d 2024-05-08T23:34:02.306 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:34:02 smithi016 systemd[1]: Stopping Ceph mds.d for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:02.612 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:34:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-d[95036]: 2024-05-08T23:34:02.306+0000 7f8fa976f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:02.612 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:34:02 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-d[95036]: 2024-05-08T23:34:02.306+0000 7f8fa976f700 -1 mds.d *** got signal Terminated *** 2024-05-08T23:34:06.362 INFO:journalctl@ceph.mds.d.smithi016.stdout:May 08 23:34:06 smithi016 bash[131243]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-d 2024-05-08T23:34:06.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T23:34:06.488 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:06.489 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T23:34:06.489 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T23:34:06.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.g 2024-05-08T23:34:06.764 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:34:06 smithi016 systemd[1]: Stopping Ceph mds.g for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:07.112 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:34:06 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-g[95560]: 2024-05-08T23:34:06.764+0000 7eff0f695700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:07.112 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:34:06 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-g[95560]: 2024-05-08T23:34:06.764+0000 7eff0f695700 -1 mds.g *** got signal Terminated *** 2024-05-08T23:34:09.112 INFO:journalctl@ceph.mds.g.smithi016.stdout:May 08 23:34:08 smithi016 bash[131390]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-g 2024-05-08T23:34:09.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T23:34:09.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:09.217 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T23:34:09.217 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T23:34:09.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.j 2024-05-08T23:34:09.498 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:34:09 smithi016 systemd[1]: Stopping Ceph mds.j for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:09.862 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:34:09 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-j[94650]: 2024-05-08T23:34:09.498+0000 7fd87db6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:09.862 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:34:09 smithi016 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-j[94650]: 2024-05-08T23:34:09.498+0000 7fd87db6b700 -1 mds.j *** got signal Terminated *** 2024-05-08T23:34:12.519 INFO:journalctl@ceph.mds.j.smithi016.stdout:May 08 23:34:12 smithi016 bash[131529]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-j 2024-05-08T23:34:12.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T23:34:12.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:12.563 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T23:34:12.563 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T23:34:12.563 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.b 2024-05-08T23:34:12.963 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:34:12 smithi155 systemd[1]: Stopping Ceph mds.b for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:12.964 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:34:12 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: 2024-05-08T23:34:12.821+0000 7fe0d3d8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:12.964 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:34:12 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b[83839]: 2024-05-08T23:34:12.821+0000 7fe0d3d8f700 -1 mds.b *** got signal Terminated *** 2024-05-08T23:34:15.029 INFO:journalctl@ceph.mds.b.smithi155.stdout:May 08 23:34:14 smithi155 bash[91683]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-b 2024-05-08T23:34:15.036 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T23:34:15.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:15.071 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T23:34:15.071 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T23:34:15.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.e 2024-05-08T23:34:15.337 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:34:15 smithi155 systemd[1]: Stopping Ceph mds.e for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:15.713 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:34:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e[83153]: 2024-05-08T23:34:15.337+0000 7f9bba031700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:15.713 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:34:15 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e[83153]: 2024-05-08T23:34:15.337+0000 7f9bba031700 -1 mds.e *** got signal Terminated *** 2024-05-08T23:34:19.106 INFO:journalctl@ceph.mds.e.smithi155.stdout:May 08 23:34:18 smithi155 bash[91823]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-e 2024-05-08T23:34:19.112 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T23:34:19.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:19.150 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T23:34:19.150 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T23:34:19.150 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.h 2024-05-08T23:34:19.411 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:34:19 smithi155 systemd[1]: Stopping Ceph mds.h for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:19.713 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:34:19 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h[83495]: 2024-05-08T23:34:19.411+0000 7f704de0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:19.714 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:34:19 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h[83495]: 2024-05-08T23:34:19.411+0000 7f704de0a700 -1 mds.h *** got signal Terminated *** 2024-05-08T23:34:22.990 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T23:34:23.014 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:34:22 smithi155 bash[91963]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-h 2024-05-08T23:34:23.014 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:34:22 smithi155 systemd[1]: ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.h.service: Succeeded. 2024-05-08T23:34:23.014 INFO:journalctl@ceph.mds.h.smithi155.stdout:May 08 23:34:22 smithi155 systemd[1]: Stopped Ceph mds.h for 378de17a-0d90-11ef-bc97-c7b262605968. 2024-05-08T23:34:23.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:23.024 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T23:34:23.024 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T23:34:23.024 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.k 2024-05-08T23:34:23.285 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:34:23 smithi155 systemd[1]: Stopping Ceph mds.k for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:23.713 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:34:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-k[82805]: 2024-05-08T23:34:23.286+0000 7f149df36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:23.714 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:34:23 smithi155 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-k[82805]: 2024-05-08T23:34:23.286+0000 7f149df36700 -1 mds.k *** got signal Terminated *** 2024-05-08T23:34:26.713 INFO:journalctl@ceph.mds.k.smithi155.stdout:May 08 23:34:26 smithi155 bash[92101]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-k 2024-05-08T23:34:26.747 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T23:34:26.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:26.783 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T23:34:26.783 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T23:34:26.783 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.c 2024-05-08T23:34:27.101 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:34:26 smithi189 systemd[1]: Stopping Ceph mds.c for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:27.417 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:34:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c[83833]: 2024-05-08T23:34:27.102+0000 7f1a04de5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:27.417 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:34:27 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c[83833]: 2024-05-08T23:34:27.102+0000 7f1a04de5700 -1 mds.c *** got signal Terminated *** 2024-05-08T23:34:31.092 INFO:journalctl@ceph.mds.c.smithi189.stdout:May 08 23:34:30 smithi189 bash[91410]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-c 2024-05-08T23:34:31.098 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T23:34:31.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:31.133 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T23:34:31.133 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T23:34:31.133 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.f 2024-05-08T23:34:31.383 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:34:31 smithi189 systemd[1]: Stopping Ceph mds.f for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:31.667 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:34:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f[83492]: 2024-05-08T23:34:31.384+0000 7f10eeacc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:31.667 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:34:31 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f[83492]: 2024-05-08T23:34:31.384+0000 7f10eeacc700 -1 mds.f *** got signal Terminated *** 2024-05-08T23:34:35.080 INFO:journalctl@ceph.mds.f.smithi189.stdout:May 08 23:34:34 smithi189 bash[91552]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-f 2024-05-08T23:34:35.085 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T23:34:35.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:35.120 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T23:34:35.120 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T23:34:35.121 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.i 2024-05-08T23:34:35.365 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:34:35 smithi189 systemd[1]: Stopping Ceph mds.i for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:35.667 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:34:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-i[84175]: 2024-05-08T23:34:35.366+0000 7f63c6b30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:35.667 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:34:35 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-i[84175]: 2024-05-08T23:34:35.366+0000 7f63c6b30700 -1 mds.i *** got signal Terminated *** 2024-05-08T23:34:39.098 INFO:journalctl@ceph.mds.i.smithi189.stdout:May 08 23:34:38 smithi189 bash[91691]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-i 2024-05-08T23:34:39.103 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T23:34:39.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:39.140 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T23:34:39.140 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T23:34:39.140 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.l 2024-05-08T23:34:39.388 INFO:journalctl@ceph.mds.l.smithi189.stdout:May 08 23:34:39 smithi189 systemd[1]: Stopping Ceph mds.l for 378de17a-0d90-11ef-bc97-c7b262605968... 2024-05-08T23:34:39.667 INFO:journalctl@ceph.mds.l.smithi189.stdout:May 08 23:34:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-l[83149]: 2024-05-08T23:34:39.389+0000 7f620c76a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T23:34:39.668 INFO:journalctl@ceph.mds.l.smithi189.stdout:May 08 23:34:39 smithi189 ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-l[83149]: 2024-05-08T23:34:39.389+0000 7f620c76a700 -1 mds.l *** got signal Terminated *** 2024-05-08T23:34:41.118 INFO:journalctl@ceph.mds.l.smithi189.stdout:May 08 23:34:40 smithi189 bash[91830]: ceph-378de17a-0d90-11ef-bc97-c7b262605968-mds-l 2024-05-08T23:34:41.124 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-378de17a-0d90-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T23:34:41.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T23:34:41.158 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T23:34:41.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T23:34:41.364 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:34:42.937 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T23:34:43.127 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:34:44.511 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T23:34:44.698 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:34:46.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:34:46.131 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:34:46.168 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T23:34:46.198 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T23:34:46.203 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi016/crash 2024-05-08T23:34:46.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash -- . 2024-05-08T23:34:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T23:34:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-05-08T23:34:46.249 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi155/crash 2024-05-08T23:34:46.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash -- . 2024-05-08T23:34:46.295 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T23:34:46.295 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-05-08T23:34:46.297 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi189/crash 2024-05-08T23:34:46.299 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash -- . 2024-05-08T23:34:46.372 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/378de17a-0d90-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T23:34:46.372 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-05-08T23:34:46.373 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T23:34:46.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/378de17a-0d90-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T23:34:46.415 INFO:teuthology.orchestra.run.smithi016.stderr:grep: /var/log/ceph/378de17a-0d90-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-08T23:34:46.418 INFO:tasks.cephadm:Compressing logs... 2024-05-08T23:34:46.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:46.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:46.463 DEBUG:teuthology.orchestra.run.smithi189:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:34:46.491 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T23:34:46.493 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T23:34:46.493 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T23:34:56.030 INFO:tasks.cephadm:Archiving logs... 2024-05-08T23:34:56.030 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi016/log 2024-05-08T23:34:56.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:34:57.042 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi155/log 2024-05-08T23:34:57.043 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:34:58.347 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi189/log 2024-05-08T23:34:58.349 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T23:34:59.570 INFO:tasks.cephadm:Removing cluster... 2024-05-08T23:34:59.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --force 2024-05-08T23:34:59.774 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:35:00.043 DEBUG:teuthology.orchestra.run.smithi155:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --force 2024-05-08T23:35:00.226 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:35:00.476 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 378de17a-0d90-11ef-bc97-c7b262605968 --force 2024-05-08T23:35:00.661 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 378de17a-0d90-11ef-bc97-c7b262605968 2024-05-08T23:35:00.908 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T23:35:00.908 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T23:35:00.932 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T23:35:00.953 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T23:35:00.971 INFO:tasks.cephadm:Teardown complete 2024-05-08T23:35:00.971 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T23:35:00.983 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T23:35:00.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:35:00.986 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:35:00.995 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T23:35:01.114 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T23:35:01.127 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T23:35:01.127 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:35:01.130 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:35:01.132 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T23:35:01.148 INFO:teuthology.orchestra.run.smithi155.stderr:bash: ntpq: command not found 2024-05-08T23:35:01.148 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2024-05-08T23:35:01.149 INFO:teuthology.orchestra.run.smithi189.stderr:bash: ntpq: command not found 2024-05-08T23:35:01.154 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:35:01.154 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-05-08T23:35:01.154 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 +1169us[+1170us] +/- 47ms 2024-05-08T23:35:01.154 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -1198us[-1198us] +/- 55ms 2024-05-08T23:35:01.154 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -352us[ -352us] +/- 71ms 2024-05-08T23:35:01.154 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:35:01.155 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:35:01.155 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-05-08T23:35:01.155 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +1166us[+1164us] +/- 47ms 2024-05-08T23:35:01.155 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -1176us[-1178us] +/- 55ms 2024-05-08T23:35:01.155 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -344us[ -344us] +/- 71ms 2024-05-08T23:35:01.155 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:35:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T23:35:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-05-08T23:35:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 8 377 14 +1184us[+1191us] +/- 47ms 2024-05-08T23:35:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 12 -1159us[-1159us] +/- 55ms 2024-05-08T23:35:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 75 -332us[ -324us] +/- 70ms 2024-05-08T23:35:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T23:35:01.157 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T23:35:01.168 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T23:35:01.168 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T23:35:01.186 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:35:01.288 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:35:01.360 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T23:35:01.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:35:01.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:35:01.469 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:35:01.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:35:01.500 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T23:35:01.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:35:01.530 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T23:35:01.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T23:35:01.560 INFO:teuthology.task.internal:Duration was 2358.369604 seconds 2024-05-08T23:35:01.561 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T23:35:01.584 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T23:35:01.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:35:01.587 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:35:01.590 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T23:35:01.635 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:35:01.637 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:35:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T23:35:01.976 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T23:35:01.976 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-05-08T23:35:01.976 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:35:02.005 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-05-08T23:35:02.005 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:35:02.027 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-05-08T23:35:02.028 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T23:35:02.050 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T23:35:02.050 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:35:02.053 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:35:02.070 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T23:35:02.144 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T23:35:02.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:35:02.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:35:02.149 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T23:35:02.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T23:35:02.285 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T23:35:02.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:35:02.316 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:35:02.344 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T23:35:02.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T23:35:02.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:35:02.389 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:35:02.392 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:35:02.415 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-05-08T23:35:02.419 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-05-08T23:35:02.438 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-05-08T23:35:02.481 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:35:02.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:35:02.502 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:35:02.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:35:02.519 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T23:35:02.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T23:35:02.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T23:35:02.551 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T23:35:02.551 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi016 2024-05-08T23:35:02.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:35:02.606 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi155 2024-05-08T23:35:02.606 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:35:02.648 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698148/remote/smithi189 2024-05-08T23:35:02.648 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T23:35:02.689 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T23:35:02.689 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:35:02.692 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:35:02.694 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T23:35:02.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T23:35:02.756 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T23:35:02.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T23:35:02.820 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T23:35:02.820 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:35:02.823 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:35:02.825 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T23:35:02.839 INFO:teuthology.orchestra.run.smithi155.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:35 /home/ubuntu/cephtest 2024-05-08T23:35:02.841 INFO:teuthology.orchestra.run.smithi189.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:35 /home/ubuntu/cephtest 2024-05-08T23:35:02.842 INFO:teuthology.orchestra.run.smithi016.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 23:35 /home/ubuntu/cephtest 2024-05-08T23:35:02.843 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T23:35:02.856 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T23:35:02.895 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}} duration: 2358.369604110718 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T23:35:02.896 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T23:35:02.977 INFO:teuthology.run:pass